Temporal scalability in geometry point cloud compression (g-pcc)

Abstract

Example devices and techniques for incorporating temporal identification information into a geometry point cloud compression syntax structure are disclosed. An example device for decoding point cloud data includes a memory configured to store the point cloud data and one or more processors implemented in circuitry and communicatively coupled to the memory. The one or more processors are configured to determine a value of a temporal identifier (ID) of a syntax structure and decode the point cloud data based on the value of the temporal ID. The temporal ID of the syntax structure identifies a temporal layer to which point cloud data associated with the syntax structure belongs.

Description

[0001] This application claims priority to U.S. Patent Application No. 17 / 482,012, filed September 22, 2021, and U.S. Provisional Patent Application No. 63 / 087,809, filed October 5, 2020, the entire contents of each of which are incorporated herein by reference. U.S. Patent Application No. 17 / 482,012, filed September 22, 2021, claims the benefit of U.S. Provisional Patent Application No. 63 / 087,809, filed October 5, 2020. Technical Field

[0002] This disclosure pertains to point cloud encoding and decoding. Background Technology

[0003] A point cloud is a collection of points in three-dimensional space. These points can correspond to points on objects within that space. Therefore, point clouds can be used to represent the physical content of three-dimensional space. Point clouds can be useful in a wide variety of situations. For example, in the context of autonomous vehicles, point clouds can be used to represent the positions of objects on a road. In another example, in the context of representing the physical content of an environment, point clouds can be used for the purpose of locating virtual objects in augmented reality (AR) or mixed reality (MR) applications. Point cloud compression is the process of encoding and decoding point clouds. Encoding point clouds reduces the amount of data required for storing and transmitting them. Summary of the Invention

[0004] Generally, this disclosure describes techniques for achieving and / or improving the temporal scalability of geometry-based point cloud compression (G-PCC) bitstreams. Temporal scalability allows bitstreams or point cloud frames to be divided into temporal layers. By receiving different subsets or all of these temporal layers, a G-PCC decoder can decode point cloud data at different frame rates. To correctly decode temporal layers, the G-PCC decoder may need to identify which point cloud data belongs to which temporal layer. This disclosure describes techniques that facilitate the G-PCC decoder's identification of which point cloud data belongs to which temporal layer.

[0005] In one example, this disclosure describes a method for decoding point cloud data, comprising: determining the value of a time identifier (ID) of a syntax structure; and decoding the point cloud data based on the value of the time ID, wherein the time ID of the syntax structure identifies the time layer to which the point cloud data associated with the syntax structure belongs.

[0006] In another example, this disclosure describes a method for encoding point cloud data, comprising: determining the value of a time identifier (ID) of a syntax structure; and encoding the point cloud data to include the time ID in the syntax structure, wherein the time ID of the syntax structure identifies the time layer to which the point cloud data associated with the syntax structure belongs.

[0007] In another example, this disclosure describes an apparatus for decoding point cloud data, comprising: a memory configured to store the point cloud data; and one or more processors implemented in a circuit and communicatively coupled to the memory, the one or more processors being configured to: determine a value of a time identifier (ID) of a syntax structure; and decode the point cloud data based on the value of the time ID, wherein the time ID of the syntax structure identifies the time layer to which the point cloud data associated with the syntax structure belongs.

[0008] In another example, this disclosure describes an apparatus for encoding point cloud data, comprising: a memory configured to store the point cloud data; and one or more processors implemented in a circuit and communicatively coupled to the memory, the one or more processors being configured to: determine a value of a time identifier (ID) of a syntax structure; and encode the point cloud data to include the time ID in the syntax structure, wherein the time ID of the syntax structure identifies the time layer to which the point cloud data associated with the syntax structure belongs.

[0009] Details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description, drawings, and claims. Attached Figure Description

[0010] Figure 1 This is a block diagram illustrating an example encoding and decoding system that can perform the techniques described in this disclosure.

[0011] Figure 2 This is a block diagram illustrating an example geometry point cloud compression (G-PCC) encoder.

[0012] Figure 3 This is a block diagram showing an example G-PCC decoder.

[0013] Figure 4 This is a flowchart illustrating an example G-PCC decoding technique using time identifiers according to the technology disclosed herein.

[0014] Figure 5 This is a flowchart illustrating an example G-PCC encoding technique for using time identifiers according to the technology disclosed herein. Detailed Implementation

[0015] Temporal scalability is a desirable characteristic of Geometric Point Cloud Compression (G-PCC) bitstreams. Utilizing temporal scalability, bitstreams or point cloud frames can be divided into temporal layers. By receiving different subsets or all of these temporal layers, the G-PCC decoder can decode the point cloud data at different frame rates. Typically (though not necessary), if a frame is split into N temporal layers, frames belonging to temporal layer 1 will represent the point cloud at one frame rate, frames belonging to temporal layers 1 and 2 will represent the point cloud at a higher frame rate, frames belonging to temporal layers 1, 2, and 3 will represent the point cloud at an even higher frame rate, and so on. Therefore, it is necessary to identify the individual temporal layers of the point cloud frame. Without such identification information, it is impossible to distinguish which slices / frames belong to which temporal layer, and the G-PCC decoder will not be able to correctly decode the point cloud data at various frame rates.

[0016] Figure 1 This is a block diagram illustrating an example encoding and decoding system 100 capable of implementing the techniques of this disclosure. The techniques of this disclosure are generally directed to decoding (encoding and / or decoding) point cloud data, i.e., to support point cloud compression. Generally, point cloud data includes any data used for processing point clouds. Decoding can efficiently compress and / or decompress point cloud data.

[0017] like Figure 1 As shown, system 100 includes a source device 102 and a target device 116. The source device 102 provides encoded point cloud data to be decoded by the target device 116. Specifically, in Figure 1 In this example, source device 102 provides point cloud data to target device 116 via computer-readable medium 110. Source device 102 and target device 116 can include any of a wide range of devices, including desktop computers, laptops, tablets, set-top boxes, handset phones (such as smartphones), televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, land or sea vehicles, spacecraft, aircraft, robots, LiDAR devices, satellites, etc. In some cases, source device 102 and target device 116 may be equipped for wireless communication.

[0018] exist Figure 1In the example, source device 102 includes a data source 104, a memory 106, a G-PCC encoder 200, and an output interface 108. Target device 116 includes an input interface 122, a G-PCC decoder 300, a memory 120, and a data consumer 118. According to this disclosure, the G-PCC encoder 200 of source device 102 and the G-PCC decoder 300 of target device 116 can be configured to apply techniques related to achieving and / or improving the temporal scalability of G-PCC bitstreams.

[0019] Therefore, source device 102 represents an example of an encoding device, while target device 116 represents an example of a decoding device. In other examples, source device 102 and target device 116 may include other components or arrangements. For example, source device 102 may receive data (e.g., point cloud data) from an internal or external source. Similarly, target device 116 may be connected to an external data consumer, rather than including the data consumer in the same device.

[0020] like Figure 1 The system 100 shown is merely an example. In general, other digital encoding and / or decoding devices can perform the techniques of this disclosure related to implementing and / or improving the temporal scalability of G-PCC bitstreams.

[0021] Source device 102 and target device 116 are merely examples of such devices, in which source device 102 generates decoded data for transmission to target device 116. This disclosure refers to a “decoding” device as a device that performs the decoding (encoding and / or decoding) of data. Thus, G-PCC encoder 200 and G-PCC decoder 300 represent examples of decoding devices (specifically, encoder and decoder). In some examples, source device 102 and target device 116 may operate in a substantially symmetrical manner, such that each of source device 102 and target device 116 includes encoding and decoding components. Therefore, system 100 can support unidirectional or bidirectional transmission between source device 102 and target device 116, for example, for streaming, playback, broadcasting, telephone, navigation, and other applications.

[0022] Generally, data source 104 represents the source of data (i.e., raw, unencoded point cloud data) and provides a continuous sequence of "frames" of data to G-PCC encoder 200, which encodes the data for each frame. Data source 104 of source device 102 may include point cloud capture devices, such as any of a variety of cameras or sensors (e.g., a 3D scanner or a light detection and ranging (LIDAR) device), one or more video cameras, archives containing previously captured data, and / or data feed interfaces for receiving data from data content providers. Alternatively or additionally, point cloud data may be computer data or other data generated from a scanner, camera, sensor, or other source. For example, data source 104 may generate computer graphics-based data as source data, or a combination of real-time data, archived data, and computer-generated data. In each case, G-PCC encoder 200 encodes the captured, pre-captured, or computer-generated data. G-PCC encoder 200 may rearrange frames from the received order (sometimes referred to as "display order") to a decoding order for decoding. The G-PCC encoder 200 can generate one or more bit streams including encoded data. The source device 102 can then output the encoded data to a computer-readable medium 110 via the output interface 108 for reception and / or retrieval via, for example, the input interface 122 of the target device 116.

[0023] The memory 106 of the source device 102 and the memory 120 of the target device 116 can represent general-purpose memory. In some examples, memory 106 and memory 120 can store raw data, such as raw data from data source 104 and raw, decoded data from G-PCC decoder 300. Alternatively or additionally, memory 106 and memory 120 can store software instructions executable by, for example, G-PCC encoder 200 and G-PCC decoder 300, respectively. Although in this example, memory 106 and memory 120 are shown separately from G-PCC encoder 200 and G-PCC decoder 300, it should be understood that G-PCC encoder 200 and G-PCC decoder 300 may also include internal memory for functionally similar or equivalent purposes. Furthermore, memory 106 and memory 120 can store encoded data (e.g., output from G-PCC encoder 200 and input to G-PCC decoder 300). In some examples, a portion of memory 106 and memory 120 may be allocated as one or more buffers, for example, to store raw, decoded, and / or encoded data. For instance, memory 106 and memory 120 may store data representing point clouds.

[0024] Computer-readable medium 110 can represent any type of medium or device capable of transmitting encoded data from source device 102 to target device 116. In one example, computer-readable medium 110 represents a communication medium enabling source device 102 to transmit encoded data directly to target device 116 in real time, for example, via a radio frequency network or a computer-based network. Output interface 108 can modulate the transmitted signal, including encoded data, according to a communication standard such as a wireless communication protocol, and input interface 122 can demodulate the received transmitted signal. The communication medium can include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form part of a packet-based network, such as a local area network, wide area network, or global network (such as the Internet). The communication medium can include a router, switch, base station, or any other equipment that may be useful for facilitating communication from source device 102 to target device 116.

[0025] In some examples, source device 102 can output encoded data from output interface 108 to storage device 112. Similarly, target device 116 can access encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media (such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile memory, or non-volatile memory) or any other suitable digital storage media for storing encoded data.

[0026] In some examples, source device 102 may output encoded data to file server 114 or another intermediate storage device that may store the encoded data generated by source device 102. Target device 116 may access the stored data from file server 114 via streaming or downloading. File server 114 may be any type of server device capable of storing encoded data and sending that encoded data to target device 116. File server 114 may represent a web server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Target device 116 may access the encoded data from file server 114 via any standard data connection, including an internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both, suitable for accessing the encoded data stored on file server 114. File server 114 and input interface 122 may be configured to operate according to a streaming protocol, a download protocol, or a combination thereof.

[0027] Output interface 108 and input interface 122 may represent a wireless transmitter / receiver, a modem, a wired network component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transmit data, such as encoded data, according to cellular communication standards such as 4G, 4G-LTE (Long Term Evolution), improved LTE, 5G, etc. In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to transmit data, such as encoded data, according to other wireless standards (e.g., the IEEE 802.11 specification, the IEEE 802.15 specification (e.g., Purple Bee™), the Bluetooth™ standard, etc.). In some examples, source device 102 and / or target device 116 may include corresponding system-on-chip (SoC) devices. For example, source device 102 may include a SoC device to perform functions belonging to G-PCC encoder 200 and / or output interface 108, and target device 116 may include a SoC device to perform functions belonging to G-PCC decoder 300 and / or input interface 122.

[0028] The technology disclosed herein can be applied to encoding and decoding for any application in a variety of applications, such as communication between autonomous vehicles, communication between scanners, cameras, sensors and processing devices (such as local or remote servers), geographic mapping or other applications.

[0029] The input interface 122 of the target device 116 receives an encoded bitstream from a computer-readable medium 110 (e.g., a communication medium, storage device 112, file server 114, etc.). The encoded bitstream may include signaling information defined by the G-PCC encoder 200 (which is also used by the G-PCC decoder 300), such as syntax elements having values ​​describing the characteristics of the decoded units (e.g., slices, pictures, picture groups, sequences, etc.) and / or the processing of the decoded units, such as time identifiers (IDs), which will be discussed further later in this disclosure. The data consumer 118 uses the decoded data. For example, the data consumer 118 may use the decoded data to determine the location of physical objects. In some examples, the data consumer 118 may include a display to show images based on point clouds.

[0030] Both the G-PCC encoder 200 and the G-PCC decoder 300 can be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is partially implemented in software, the device may store instructions for the software in a suitable, non-transitory computer-readable medium, and use one or more processors to execute said instructions in hardware to perform the technology of this disclosure. Each of the G-PCC encoder 200 and the G-PCC decoder 300 may be included in one or more encoders or decoders, and either encoder or decoder may be integrated as part of a combined encoder / decoder (CODEC) in the respective device. Devices including the G-PCC encoder 200 and / or the G-PCC decoder 300 may include one or more integrated circuits, microprocessors, and / or other types of devices.

[0031] The G-PCC encoder 200 and G-PCC decoder 300 can operate according to decoding standards such as Video Point Cloud Compression (V-PCC) or Geometric Point Cloud Compression (G-PCC). This disclosure generally relates to the decoding (e.g., encoding and decoding) of images, including the process of encoding or decoding data. The encoded bitstream typically includes a series of values ​​for syntax elements representing decoding decisions (e.g., decoding modes).

[0032] This disclosure may generally refer to "signaling" certain information (such as syntax elements). The term "signaling" can generally refer to the transmission of values ​​for syntax elements and / or other data for decoding encoded data. In some examples, syntax elements such as time IDs may be included in syntax structures such as slice headers or data unit headers. Slice headers or data unit headers may include syntax elements applicable to slices or data units associated with that header. G-PCC encoder 200 may signal values ​​for syntax elements in the bitstream. G-PCC decoder 300 may parse these syntax elements to determine the signal values. Generally, signaling refers to generating values ​​in the bitstream. As described above, source device 102 may transmit the bitstream to target device 116 substantially in real time or not in real time (such as when syntax elements are stored to storage device 112 for later retrieval by target device 116).

[0033] ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) and more recently ISO / IEC 3DG (JTC1 / SC29 / WG 7) are investigating the potential need to standardize point cloud decoding techniques with compression capabilities significantly exceeding current methods, and are working towards creating standards. This exploration is being conducted collaboratively within a group called the 3D Graphics Group (3DG) to evaluate compression technology designs proposed by experts in the field.

[0034] Point cloud compression activities can be categorized into two distinct approaches. The first approach is “Video Point Cloud Compression” (V-PCC), which segments a 3D object and projects these segments onto multiple 2D planes (represented as “patches” in 2D frames), further decoded by legacy 2D video codecs such as the High Efficiency Video Decoding (HEVC) (ITU-T H.265) codec. The second approach is “Geometry-Based Point Cloud Compression” (G-PCC), which directly compresses 3D geometry (i.e., the location of a set of points in 3D space) and associated attribute values ​​(for each point associated with the 3D geometry). G-PCC addresses the compression of point clouds in both Category 1 (static point clouds) and Category 3 (dynamically acquired point clouds). The latest draft of the G-PCC standard is available in G-PCC DIS, ISO / IEC JTC1 / SC29 / WG11 w19328, via teleconference (previously the Alpbach meeting), in mid-April 2020. The description of the codec is available in G-PCC Codec Description v8, ISO / IEC JTC1 / SC29 / WG11 w19525, via teleconference, in mid-June 2020. Some recent changes to the G-PCC specification are in G-PCC Future Enhancements, ISO / IEC JTC1 / SC29 / WG11 w19522, via teleconference, in mid-June 2020.

[0035] A point cloud comprises a set of points in 3D space, and may have attributes associated with those points. These attributes may be color information such as R, G, B or Y, Cb, Cr, or reflection information, or other attributes. Point clouds can be captured by various cameras or sensors (such as LiDAR sensors and 3D scanners), and can also be computer-generated. Point cloud data is used in a variety of applications, including but not limited to: architecture (modeling), graphics (3D models for visualization and animation), and the automotive industry (LiDAR sensors for navigation aids).

[0036] The 3D space occupied by the point cloud can be surrounded by a virtual bounding box. The positions of the points within the bounding box can be represented with a certain precision. Therefore, the positions of one or more points can be quantized based on this precision. At the smallest level, the bounding box is subdivided into voxels, which are the smallest spatial units represented by a unit cube. A voxel within the bounding box can be associated with zero, one, or more points. The bounding box can be subdivided into multiple cubic / cuboid regions, which can be called tiles. Each tile can be decoded into one or more slices. The partitioning of the bounding box into slices and tiles can be based on the number of points in each partition, or on other considerations (e.g., decoding a specific region into a tile). Sliced ​​regions can be further subdivided using a partitioning decision similar to that in a video codec.

[0037] Figure 2 Provides an overview of the G-PCC encoder 200. Figure 3 An overview of the G-PCC decoder 300 is provided. The modules shown are logical, and do not necessarily correspond one-to-one with the code implemented in the reference implementation of the G-PCC codec (i.e., the TMC13 test model software studied by ISO / IEC MPEG (JTC 1 / SC 29 / WG 11)).

[0038] In both the G-PCC encoder 200 and the G-PCC decoder 300, the point cloud locations are decoded first. Attribute decoding depends on the decoded geometry. Figure 2 and Figure 3 In this configuration, surface approximation analysis unit 212, RAHT unit 218, surface approximation synthesis unit 310, and RAHT unit 314 are options typically used for Category 1 data. LOD generation unit 220, lifting unit 222, LOD generation unit 316, and reverse lifting unit 318 are options typically used for Category 3 data. All other modules are shared between Category 1 and Category 3.

[0039] For Category 3 data, the compressed geometry is typically represented as an octree at the leaf level, descending from the root down to individual voxels. For Category 1 data, the compressed geometry is typically represented by a pruned octree (i.e., an octree at the leaf level, descending from the root down to blocks larger than voxels) plus a model that approximates the surface within each leaf of the pruned octree. In this way, both Category 1 and Category 3 data share the octree decoding mechanism, while Category 1 data can additionally utilize a surface model to approximate the voxels within each leaf. The surface model used is a triangular partitioning of each block comprising 1-10 triangles, thus forming a triangular soup. Therefore, the Category 1 geometry codec is called a Trisoup geometry codec, while the Category 3 geometry codec is called an octree geometry codec.

[0040] At each node in the octree, occupancy is signaled (when not inferred) for one or more of its child nodes (up to eight nodes). Multiple neighborhoods are specified, including: (a) nodes sharing a face with the current octree node, (b) nodes sharing a face, edge, or vertex with the current octree node, etc. Within each neighborhood, occupancy for a node and / or its child nodes can be used to predict occupancy for the current node or its child nodes. For points sparsely filled in some nodes of the octree, the codec also supports a direct decoding mode, where the 3D position of the point is directly encoded. A flag can be signaled to indicate that the direct mode is signaled. At the lowest level, the number of points associated with an octree node / leaf node can also be decoded.

[0041] Once the geometry is decoded, the attributes corresponding to the geometric points are also decoded. When there are multiple attribute points corresponding to a reconstructed / decoded geometric point, the attribute values ​​representing the reconstructed point can be derived.

[0042] There are three attribute decoding methods in G-PCC: Region Adaptive Hierarchical Transform (RAHT) decoding, interpolation-based hierarchical nearest neighbor prediction (prediction transform), and interpolation-based hierarchical nearest neighbor prediction (lifting transform) with an update / lifting step. RAHT and lifting are typically used for Class 1 data, while prediction transform is typically used for Class 3 data. However, any method can be used for any data, and similar to the geometry codec in G-PCC, the attribute decoding method used to decode point clouds is specified in the bitstream.

[0043] Decoding of attributes can be performed at the level of detail (LoD), where a finer representation of the point cloud attributes can be obtained for each level of detail. Each level of detail can be specified based on a distance metric to neighboring nodes or based on sampling distance.

[0044] At the G-PCC encoder 200, the residuals obtained as the output of the attribute-specific decoding method are quantized. Context-adaptive arithmetic decoding can be used to decode the quantized residuals.

[0045] exist Figure 2 In the example, the G-PCC encoder 200 may include a coordinate transformation unit 202, a color transformation unit 204, a voxelization unit 206, an attribute transfer unit 208, an octree analysis unit 210, a surface approximation analysis unit 212, an arithmetic coding unit 214, a geometric reconstruction unit 216, a RAHT unit 218, a LOD generation unit 220, a lifting unit 222, a coefficient quantization unit 224, and an arithmetic coding unit 226.

[0046] like Figure 2 As shown in the example, the G-PCC encoder 200 can receive a set of locations and a set of attributes. The locations may include the coordinates of points in the point cloud. The attributes may include information about the points in the point cloud, such as the colors associated with the points in the point cloud.

[0047] The coordinate transformation unit 202 can apply a transformation to the coordinates of a point to transform the coordinates from the initial domain to the transformation domain. The transformed coordinates can be referred to as transformed coordinates in this disclosure. The color transformation unit 204 can apply a transformation to transform the color information of an attribute to a different domain. For example, the color transformation unit 204 can transform color information from the RGB color space to the YCbCr color space.

[0048] In addition, Figure 2 In the example, voxelization unit 206 can voxelize the transformed coordinates. Voxelization of the transformed coordinates can include quantization and removal of some points in the point cloud. In other words, multiple points in the point cloud can be contained in a single "voxel," which can subsequently be considered as a single point in some respects. Furthermore, octree analysis unit 210 can generate an octree based on the voxelized transformed coordinates. Additionally, in Figure 2 In the example, the surface approximation analysis unit 212 can analyze points to potentially determine a surface representation of the point set. The arithmetic coding unit 214 can entropy encode the syntax elements representing information about the octree and / or surface determined by the surface approximation analysis unit 212. The G-PCC encoder 200 can output these syntax elements in a geometric bitstream.

[0049] The geometric reconstruction unit 216 can reconstruct the transformed coordinates of points in the point cloud based on an octree, data indicating the surface determined by the surface approximation analysis unit 212, and / or other information. Due to voxelization and surface approximation, the number of transformed coordinates reconstructed by the geometric reconstruction unit 216 may differ from the number of original points in the point cloud. The resulting points can be referred to as reconstructed points. The attribute transfer unit 208 can transfer attributes of the original points in the point cloud to the reconstructed points in the point cloud.

[0050] Furthermore, RAHT unit 218 can apply RAHT decoding to the attributes of the reconstructed point. Alternatively or additionally, LOD generation unit 220 and lifting unit 222 can respectively apply LOD processing and lifting to the attributes of the reconstructed point. RAHT unit 218 and lifting unit 222 can generate coefficients based on the attributes. Coefficient quantization unit 224 can quantize the coefficients generated by RAHT unit 218 or lifting unit 222. Arithmetic encoding unit 226 can apply arithmetic decoding to the syntax elements representing the quantized coefficients. G-PCC encoder 200 can output these syntax elements in the attribute bitstream.

[0051] exist Figure 3 In the example, the G-PCC decoder 300 may include a geometric arithmetic decoding unit 302, an attribute arithmetic decoding unit 304, an octree synthesis unit 306, an inverse quantization unit 308, a surface approximation synthesis unit 310, a geometric reconstruction unit 312, a RAHT unit 314, an LOD generation unit 316, an inverse lifting unit 318, an inverse coordinate transformation unit 320, and an inverse color transformation unit 322.

[0052] The G-PCC decoder 300 can obtain a geometric bitstream and an attribute bitstream. The geometric arithmetic decoding unit 302 of the G-PCC decoder 300 can apply arithmetic decoding (e.g., context-adaptive binary arithmetic decoding (CABAC) or other types of arithmetic decoding) to syntax elements in the geometric bitstream, such as time IDs. Similarly, the attribute arithmetic decoding unit 304 can apply arithmetic decoding to syntax elements in the attribute bitstream, such as time IDs.

[0053] Octree synthesis unit 306 can synthesize an octree based on syntax elements parsed from the geometric bitstream. When surface approximation is used in the geometric bitstream, surface approximation synthesis unit 310 can determine the surface model based on syntax elements parsed from the geometric bitstream and based on the octree.

[0054] Furthermore, the geometric reconstruction unit 312 can perform reconstruction to determine the coordinates of points in the point cloud. The inverse coordinate transformation unit 320 can apply an inverse transformation to the reconstructed coordinates to transform the reconstructed coordinates (positions) of points in the point cloud from the transformation domain back to the initial domain.

[0055] In addition, Figure 3 In the example, the inverse quantization unit 308 can inverse quantize the attribute value. The attribute value can be based on syntax elements obtained from the attribute bitstream (e.g., including syntax elements decoded by the attribute arithmetic decoding unit 304).

[0056] Depending on how the attribute values ​​are encoded, RAHT unit 314 can perform RAHT decoding to determine the color value for a point in the point cloud based on the inversely quantized attribute values. Alternatively, LOD generation unit 316 and inverse lifting unit 318 can use level-of-detail techniques to determine the color value for a point in the point cloud.

[0057] In addition, Figure 3 In the example, the inverse color transformation unit 322 can apply an inverse color transformation to color values. The inverse color transformation can be the inverse operation of the color transformation applied by the color transformation unit 204 of the G-PCC encoder 200. For example, the color transformation unit 204 can transform color information from the RGB color space to the YCbCr color space. Therefore, the inverse color transformation unit 322 can transform color information from the YCbCr color space to the RGB color space.

[0058] Show Figure 2 and Figure 3 The various units help to understand the operations performed by the G-PCC encoder 200 and the G-PCC decoder 300. These units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide specific functions and are pre-configured for the operations they can perform. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functionality in the operations they can perform. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by instructions from software or firmware. Fixed-function circuits can execute software instructions (e.g., to receive or output parameters), but the type of operation performed by a fixed-function circuit is generally immutable. In some examples, one or more units in the unit may be different circuit blocks (fixed-function or programmable), and in some examples, one or more units in the unit may be integrated circuits.

[0059] In several applications, point cloud data, or points within point cloud data, are associated with a temporal aspect. This temporal aspect may correspond to the moment the point was captured, or it may correspond to the time when the point cloud was rendered / displayed. Typically, the temporal aspect may not be absolute time, but rather relative to a reference time. This temporal component can be specified as a specific attribute associated with time, or it can be decoded as a frame index. Frame indices can be used to specify the relationship between associated time points. For example, when point A has a frame index of 3 and another point B has a frame index of 4, assuming the frame index size is proportional to time, point B is associated with a later time point than point A.

[0060] It should be noted that, in addition, some point clouds can be stored / represented as separate frames. Unless there is explicit signaling for a frame index or time associated with each point within a frame, for practical purposes, all points in a frame can be associated with the same time instance.

[0061] The G-PCC codec supports "fusing" multiple frames, making the G-PCC encoder 200 or G-PCC decoder 300 more efficient in decoding multiple frames. In the case of fusing multiple frames, the frame index is added as an attribute to distinguish the point from the different frames being fused. In G-PCC, this is called "frame fusion decoding" or "combined frame decoding."

[0062] Temporal scalability is a desirable characteristic of G-PCC bitstreams. Utilizing temporal scalability, the G-PCC encoder 200 can divide a bitstream or point cloud frame into temporal layers. Receiving different subsets of temporal layers allows the G-PCC decoder 300 to decode the point cloud data at different frame rates. Typically (though not necessary), if the G-PCC encoder 200 splits a frame into N temporal layers, frames belonging to temporal layer 1 will represent the point cloud at one frame rate (e.g., 30 frames per second), frames belonging to temporal layers 1 and 2 will represent the point cloud at a higher frame rate (e.g., 60 frames per second), frames belonging to temporal layers 1, 2, and 3 will represent the point cloud at an even higher frame rate (e.g., 120 frames per second), and so on. In some examples, temporal layers can be referred to by other numbers (e.g., 0, 1, and 2) or by other names. It may be desirable to identify the individual temporal layers of a point cloud frame. Without any such identification information, it may be impossible to distinguish which data units, slices, or frames belong to which temporal layer.

[0063] The techniques disclosed herein can solve one or more problems related to distinguishing which data units, slices, or frames belong to which time layer. The techniques disclosed herein can be applied independently or in any combination.

[0064] In one example, the G-PCC encoder 200 may signal a value for a time ID associated with a syntax structure (or for derivation). Using the time ID makes it possible to determine the time layer to which the syntax structure belongs. For example, a syntax structure may be associated with a slice or data unit (such as a slice header or data unit header), and the G-PCC encoder 200 may signal a value for the time ID of a slice or data unit. In G-PCC, a slice may include geometric data units and zero or more associated attribute data units. As used herein, a geometric data unit may also be referred to as a geometric slice, and an attribute data unit may also be referred to as an attribute slice. The time ID may be in the slice syntax structure (such as a slice header), the geometric data unit syntax structure (such as a geometric data unit header), and / or the attribute data unit syntax structure (such as an attribute data unit header).

[0065] For example, G-PCC encoder 200 can determine the value of the time ID of the syntax structure and encode the point cloud data to include the time ID in the syntax structure (e.g., a slice header or a data unit header). Similarly, G-PCC decoder 300 can determine the value of the time ID of the syntax structure and decode the point cloud data based on the time ID value. For instance, if the time ID value indicates a time layer to be decoded at the decoding frame rate being used by G-PCC decoder 300, then G-PCC decoder 300 can decode the point cloud data.

[0066] In some examples, the G-PCC encoder 200 may signal the value only in some syntax structures (e.g., syntax structures associated with geometric slices or data units), and the G-PCC decoder 300 may derive the corresponding value for other syntax structures (e.g., syntax structures associated with attribute slices or data units) from the associated syntax structure containing the value. For example, the G-PCC encoder 200 may signal the value using geometric slices or data units but not attribute slices or data units. In G-PCC, a geometric slice or data unit includes data describing the location of a set of points in the slice, while an associated attribute slice or data unit includes data describing the attributes of the same set of points. In an example where the G-PCC encoder 200 signals the value using geometric slices or data units but not attribute slices or data units, the G-PCC decoder 300 may derive that the value associated with an attribute slice or data unit is equal to the value associated with the geometric slice or data unit corresponding to the attribute slice, where typically the attribute slice or data unit contains attribute information of the points signaled in the corresponding geometric slice. For example, the G-PCC encoder 200 can avoid signaling the value of the time ID associated with the attribute data unit corresponding to the geometric data unit. For example, the G-PCC decoder 300 can infer that the value of the time ID associated with the attribute data unit corresponding to the geometric data unit is equal to the value of the time ID associated with the geometric data unit. Compared to the example where the time ID associated with the attribute data is signaled in the attribute bitstream, inferring that the time ID associated with the attribute data unit is the same as the time ID associated with the corresponding geometric data unit can reduce the size of the attribute bitstream.

[0067] In some examples, the G-PCC encoder 200 may signal the time ID value only in the independent slice (or slice header), and not in the dependent slice (or slice header). A dependent slice may be a slice that may depend on a previous slice for decoding. For example, the G-PCC decoder 300 may not be able to decode a dependent slice without first decoding the slice the dependent slice depends on. In some examples, the time ID of the dependent slice may be explicitly signaled, or it may be inferred by the G-PCC decoder 300 from other syntax elements (e.g., entropy continuation flags). For example, the G-PCC encoder 200 may explicitly signal the time ID for the dependent slice, in which case the G-PCC decoder 300 can determine the value of the time ID for the dependent slice by parsing that time ID. In another example, the G-PCC encoder 200 may avoid signaling the time ID for the dependent slice, in which case the G-PCC decoder 300 can infer the time ID based on another syntax element.

[0068] In some examples, a constraint can be specified, and the G-PCC encoder 200 can apply, that two or more slices belonging to a point cloud frame have the same time ID. For example, this constraint ensures that slices belonging to the same frame have the same time ID, and therefore are all available to the G-PCC decoder 300 when decoding that frame. For example, if this constraint is not present, and if the first slice of a frame has a time ID of t1, and the second slice of the frame has a time ID of t2 (where t1 is less than t2), the second slice may not be available to the G-PCC decoder 300 at a lower frame rate. This could lead to decoding errors.

[0069] For example, the G-PCC encoder 200 can determine the value of a first time ID of a first syntax structure associated with a first slice of a frame. The G-PCC encoder 200 can encode point cloud data to include the first time ID in the first syntax structure. The G-PCC encoder 200 can also determine that a second time ID associated with a second slice of a frame is equal to the first time ID, and encode a second syntax structure associated with the second slice to include the second time ID.

[0070] For example, the G-PCC decoder 300 can determine the value of a first time ID of a first syntax structure associated with a first slice of a frame. The G-PCC decoder 300 can then decode the first slice of the frame based on the value of the first time ID. The G-PCC decoder 300 can also determine that a second time ID associated with a second slice of the frame is equal to the first time ID, and then decode the second slice of the frame based on the second time ID.

[0071] In some examples, the following constraint can be specified, and the G-PCC encoder 200 and G-PCC decoder 300 can apply the constraint that the reconstruction of any point in a slice with time ID t1 does not depend on any slice with time ID t2, where t1 is less than t2. This constraint ensures that the G-PCC decoder 300 receives and decodes any slice that another slice depends on. If this constraint is not present, for example, if the first slice has time ID t2 and the second slice has time ID t1 (where t1 is less than t2), and the second slice depends on the first slice, then the first slice may be unavailable to the G-PCC decoder 300 when decoding the second slice (e.g., this could happen if a bitstream containing all syntax structures with time IDs less than t2 is extracted). This could lead to decoding errors.

[0072] For example, the G-PCC encoder 200 can determine the value of a first time ID for a first grammatical structure associated with an independent slice. The G-PCC encoder 200 can encode point cloud data to include the first time ID in the first grammatical structure. The G-PCC encoder 200 can also determine a second time ID associated with a dependent slice, which depends on the independent slice. The G-PCC encoder 200 can perform the following operations: determine the second time ID associated with a dependent slice, which depends on the independent slice; and encode point cloud data to include the second time ID in a second grammatical structure for the dependent slice, wherein the second time ID is equal to or greater than the first time ID.

[0073] For example, the G-PCC decoder 300 can determine the value of a first time ID of a first syntax structure associated with an independent slice. The G-PCC decoder 300 can decode the independent slice based on the value of the first time ID. The G-PCC decoder 300 can also perform the following operations: determine a second time ID associated with a dependent slice, the dependent slice depending on the independent slice; and decode the dependent slice based on the independent slice and the second time ID, wherein the second time ID is equal to or greater than the first time ID.

[0074] In some examples, when using combined frame decoding, restrictions can be added so that the G-PCC encoder 200 and G-PCC decoder 300 can combine only frames with the same time ID into a single frame. In some examples, the G-PCC encoder 200 and G-PCC decoder 300 can apply the time ID of the combined frames to the combined frame. In some examples, for each frame associated with a combined frame having time ID tC, the G-PCC decoder 300 can infer that the frame has time ID tC.

[0075] In some examples, the time ID can also be associated with a parameter set (and in some cases, the time ID is signaled by the G-PCC encoder 200). For example, a condition can be added, and the G-PCC encoder 200 can apply, that a slice with time ID t1 may not reference a parameter set with time ID t2, where t2 > t1. In some examples, the G-PCC decoder 300 can infer that the time ID of a slice is equal to the time ID of the parameter set associated with that slice. If more than one parameter set can be associated with a slice, the maximum value of the time ID of the associated parameter sets can be applied to that slice.

[0076] In another example, a fixed number of bits or bytes can be reserved in the slice header of the G-PCC stream, allowing future versions of G-PCC to use one or more of these bits to indicate useful information (e.g., time ID). For example, G-PCC encoder 200 can reserve a fixed number of bits or bytes in the slice header of the G-PCC stream for future versions of G-PCC to indicate useful information. In such a case, G-PCC encoder 200 may not encode the reserved bits or bytes, or may encode the reserved bits or bytes in a predetermined manner, such as all 0s, all 1s, or a combination of 0s and 1s. G-PCC decoder 300 may not decode the reserved bits or bytes, or may decode the reserved bits or bytes and ignore the decoded reserved bits or bytes.

[0077] In some examples, the G-PCC encoder 200 may signal a flag indicating the presence or absence of time ID information in one or more slices. The G-PCC decoder 300 may parse this flag to determine whether time ID information exists in one or more slices. When the flag indicates the presence of a time ID, the G-PCC encoder 200 may signal the time ID in one or more slices, and the G-PCC decoder 300 may parse the time ID in one or more slices to determine the time ID. In some examples, the G-PCC encoder 200 may signal a flag indicating the presence of reserved bits in one or more slices, and the G-PCC decoder 300 may parse this flag to determine whether reserved bits exist in one or more slices.

[0078] The term "slice" or "slices" as used above refers to a set of points that are decoded together, and can also be applied to other sets of points that are decoded together, such as blocks, tiles, frames, bricks, data units, etc. Additionally, the location of the signaling for any syntax element may differ from the indicated location, as any location indicated herein is provided as an example.

[0079] The following example illustrates how the G-PCC encoder 200 signals time ID information in the geometry slice header and attribute slice header. The geometry slice header may include syntax elements applicable to geometry slices, and the attribute slice header may include syntax elements applicable to attribute slices.

[0080] The following text is in the following grammatical structure <add> and< / add> The signaling techniques in geometric data units of this disclosure are illustrated below:

[0081]

[0082]

[0083] <add> gsh_temporal_id specifies the time ID of the data unit. The value of gsh_temporal_id should be in the range of 0 to MaxTemporalId (inclusive).< / add> MaxTemporalId can be a value equal to the highest allowed time ID.

[0084] The value of MaxTemporalId can be specified by a profile / level limit, or it can be derived from a value signaled in the bitstream (e.g., the maximum time ID can be signaled in the Sequence Parameter Set (SPS)). In G-PCC, a profile or level can specify a limit on the number of bits required to represent geometry and attribute information. The Sequence Parameter Set can include syntax elements applicable to frame sequences.

[0085] In some examples, the decoding of gsh_temporal_id can be of variable length (e.g., ue(v)) or based on the value of MaxTemporalId.

[0086] In some examples, the time ID can also be signaled as a syntax element ash_temporal_id in the attribute slice header.

[0087] In some examples, the following constraint may apply: the requirement for bitstream consistency is that all slices belonging to the first point cloud frame should have the same time ID. However, another point cloud frame may have slices containing time IDs different from those of the first point cloud frame.

[0088] Figure 4This is a flowchart illustrating an example G-PCC decoding technique using time identifiers according to the present disclosure. G-PCC decoder 300 can determine the value of a time ID in a syntax structure (400). For example, G-PCC decoder 300 can parse a time ID syntax element in a syntax structure (such as a slice header or a data unit header) to determine the value of the time ID of that syntax structure. G-PCC decoder 300 can decode point cloud data based on the value of the time ID (402). For example, G-PCC decoder 300 can determine that the value of the time ID indicates that a slice or data unit associated with the time ID will be decoded by G-PCC decoder 300 at the decoding frame rate, and that G-PCC decoder 300 can decode that slice or data unit. The time ID of the syntax structure can identify the time layer to which the point cloud data associated with that syntax structure belongs.

[0089] In some examples, the syntax structure is associated with slices or data units. In some examples, the syntax structure is associated with geometric data units. For example, the syntax structure could be a geometric data unit header. In some examples, the G-PCC decoder 300 can infer that the value of the time ID associated with the attribute data unit corresponding to the geometric data unit is equal to the value of the time ID associated with the geometric data unit, wherein decoding the point cloud data based on the value of the time ID includes decoding both the geometric data units and the attribute data units. In such examples, the G-PCC decoder 300 can decode slices belonging to the time layer indicated by the time ID, without the G-PCC encoder 200 needing to signal the time ID in the syntax structure associated with the attribute data unit (e.g., the attribute data unit header). In some examples, the G-PCC decoder 300 can determine the value of a flag indicating whether the time ID is indicated in the syntax structure.

[0090] In some examples, the time ID is a first time ID, and this syntax structure is associated with the first slice of the frame. In such an example, the G-PCC decoder 300 can determine that the second time ID associated with the second slice of the frame is equal to the first time ID, and decode the frame based on the first time ID.

[0091] In some examples, the syntax structure is a first syntax structure associated with an independent slice, and the time ID is a first time ID. In such an example, the G-PCC decoder 300 can perform the following operations: determine a second time ID associated with a dependent slice, which depends on the independent slice; and decode the dependent slice based on the independent slice and the second time ID, wherein the second time ID is equal to or greater than the first time ID.

[0092] In some examples, the point cloud data includes point cloud data from multiple frames, the first frame of which includes multiple slices, and each slice is associated with the same time ID.

[0093] In some examples, the point cloud data includes point cloud data from a first frame and point cloud data from a second frame, wherein the point cloud data from the first frame includes a first slice associated with a first time ID t1, and the point cloud data from the second frame includes a second slice associated with a second time ID t2, where t2 > t1. In such examples, the G-PCC decoder 300 can decode the first slice without relying on the second slice.

[0094] Figure 5 This is a flowchart illustrating an example G-PCC encoding technique using time identifiers according to the present disclosure. The G-PCC encoder 200 can determine the value of the time ID of the syntax structure (500). For example, the G-PCC encoder 200 can determine which time layer a slice or data unit should belong to, such that the slice or data unit will be decoded by the G-PCC decoder 300 at a given frame rate associated with that time layer, and can determine the value of the time ID to indicate the time layer. The G-PCC encoder 200 can encode point cloud data to include the time ID in the syntax structure (502). For example, the G-PCC encoder 200 can encode a slice header for slices or a data unit header for data units to include the time ID value. The time ID of the syntax structure can identify the time layer to which the point cloud data associated with that syntax structure belongs.

[0095] In some examples, the syntax structure is associated with slices or data units. In some examples, the syntax structure is associated with geometric data units. In some examples, the G-PCC encoder 200 may avoid signaling the value of the time ID associated with the attribute data unit corresponding to the geometric data unit. In some examples, the G-PCC encoder 200 may signal the value of the time ID as indicated in the syntax structure.

[0096] In some examples, the time ID is a first time ID, and the syntax structure is a first syntax structure associated with the first slice of the frame. In such an example, the G-PCC encoder 200 can determine that the second time ID associated with the second slice of the frame is equal to the first time ID, and encode the second syntax structure associated with the second slice to include the second time ID.

[0097] In some examples, the syntax structure is a first syntax structure associated with an independent slice, and the time ID is a first time ID. In such an example, the G-PCC encoder 200 may perform the following operations: determine a second time ID associated with a dependent slice, which depends on the independent slice; and encode the point cloud data to include the second time ID in the second syntax structure used for the dependent slice, wherein the second time ID is equal to or greater than the first time ID.

[0098] In some examples, the point cloud data includes point cloud data from multiple frames, the first frame of which includes multiple slices, and each slice is associated with the same time ID.

[0099] In some examples, the point cloud data includes point cloud data of a first frame and point cloud data of a second frame, wherein the point cloud data of the first frame includes a first slice associated with a first time ID t1, and the point cloud data of the second frame includes a second slice associated with a second time ID t2, where t2 > t1. In such examples, the G-PCC encoder 200 can encode the first slice such that the decoder (e.g., G-PCC decoder 300) decodes the first slice without relying on the second slice.

[0100] Without a time ID in the syntax structure of the G-PCC bitstream, the G-PCC decoder will not be able to correctly identify which frames, slices, or data units belong to which time layer, and therefore will not be able to correctly decode the G-PCC bitstream at different frame rates; instead, it will have to decode all frames in the bitstream and then determine which frames will be rendered at a lower frame rate—decoding all frames is computationally expensive and requires additional bandwidth. Without a time ID in the syntax structure of the G-PCC bitstream, intermediate network devices sending the bitstream to the G-PCC decoder will not be able to correctly identify which frames, slices, or data units belong to which time layer, and therefore must send all frames to the G-PCC decoder (which may require a relatively high bit rate) or the intermediate network device must determine which frames are sufficient for a specific frame rate (which may not result in good quality of the reconstructed point cloud). By adding a time ID to the syntax structure of the G-PCC bitstream, the G-PCC decoder 300 can decode different time layers in the G-PCC bitstream at a variety of different frame rates. Therefore, the technology of this disclosure supports the desired flexible frame rates for the G-PCC decoder. By adding time IDs to the syntax structure of the G-PCC bitstream, intermediate network devices can select slices / data units belonging to the desired time layer within the G-PCC bitstream for a specific frame rate and send only those slices / data units to the G-PCC decoder 300. This use of time IDs saves bandwidth because it eliminates the need to send all slices / data units to the G-PCC decoder 300.

[0101] Examples from various aspects of this disclosure may be used individually or in any combination.

[0102] This disclosure includes the following non-restrictive terms.

[0103] Clause 1A. A method for processing point cloud data, the method comprising: determining a value indicating a time identifier associated with a syntactic structure; and processing the point cloud data based on the value.

[0104] Clause 2A. The method according to Clause 1A, wherein the syntax structure includes slices or data units, and the value includes a time identifier.

[0105] Clause 3A. The method described under Clause 1A or Clause 2A further includes: signaling or resolving the value based on the fact that the syntax structure is a geometric slice.

[0106] Clause 4A. The method described in Clause 3 further includes: determining whether the syntactic structure is a geometric slice.

[0107] Clause 5A. The method described under Clause 1A or Clause 2A further includes: not signaling or inferring the value based on the fact that the syntactic structure is an attribute slice.

[0108] Clause 6A. The method described in Clause 5A further includes: determining whether the syntactic structure is an attribute slice.

[0109] Clause 7A. The method described in Clause 5A or Clause 6A, wherein inferring the value includes inferring that the value is equal to a value associated with a geometry slice corresponding to the attribute slice.

[0110] Clause 8A. The method described under Clause 2A further includes: signaling or parsing the value based on the fact that the slice is independent.

[0111] Clause 9A. The method described under Clause 8A further includes: determining whether the slice is independent.

[0112] Clause 10A. The method described in Clause 2A further includes: based on the fact that the slice is subordinate, not signaling the value or inferring the value from at least one other syntax element.

[0113] Clause 11A. The method described in Clause 10A further includes: determining whether the slice is dependent.

[0114] Clause 12A. A method for processing point cloud data, the method comprising: determining whether a frame of the point cloud data includes two or more slices; determining, based on the fact that the frame includes two or more slices, that time IDs for the two or more slices are the same time ID; and processing the frame based on the time IDs.

[0115] Clause 13A. A method for processing point cloud data, the method comprising: determining whether the reconstruction of any point in a slice having a time identifier t1 depends on any slice having a time identifier t2, wherein t1 is less than t2; and avoiding processing the slice having the time identifier t1 as dependent on any slice having the time identifier t2 based on the fact that any point in the slice having the time identifier t1 depends on any slice having the time identifier t2.

[0116] Clause 14A. A method for processing point cloud data, the method comprising: determining whether two frames have the same time ID; avoiding combining the two frames into one frame based on the fact that the two frames do not have the same time ID; and processing the two frames without combining them.

[0117] Clause 15A. The method according to Clause 14, wherein the two frames are first two frames, the method further comprising: determining whether the latter two frames have the same time ID; combining the latter two frames into one frame based on the fact that the latter two frames have the same time ID; applying the same time ID to the one frame; and processing the one frame.

[0118] Clause 16A. The method according to Clause 15A further includes: inferring that the same time ID of the one frame is the time ID of each frame associated with the one frame.

[0119] Clause 17A. A method for processing point cloud data, the method comprising: determining a value indicating a time ID associated with a set of parameters; and processing the point cloud data based on the value.

[0120] Clause 18A. The method according to Clause 17A further includes: determining that the slice has a time ID of t1; determining that the parameter set has a time ID of t2, where t2>t1; and based on t2>t1, avoiding processing the slice as referencing the parameter set.

[0121] Clause 19A. The method according to Clause 17A, wherein the parameter set is associated with a slice, the method further comprising: inferring that the time ID associated with the slice is equal to the time ID associated with the parameter set.

[0122] Clause 20A. The method according to Clause 17A, wherein the parameter set is a first parameter set, and wherein the first parameter set and a second parameter set are associated with a slice, the method further comprising: determining a maximum value among the time IDs associated with the first parameter set and the time IDs associated with the second parameter set; and applying the maximum value to the slice.

[0123] Clause 21A. A method for processing point cloud data, the method comprising: determining whether time ID information exists in one or more slices of the point cloud data; signaling a flag indicating the presence of the time ID information in the one or more slices based on the presence of the time ID information in the one or more slices; and processing the one or more slices.

[0124] Clause 22A. The method according to Clause 21A further includes: determining whether there are reserved bits in the one or more slices; and signaling a flag indicating the presence of reserved bits in the one or more slices based on the presence of reserved bits in the one or more slices.

[0125] Clause 23A. The method according to any one of Clauses 1A-22A further includes: generating the point cloud.

[0126] Clause 24A. An apparatus for processing point clouds, said apparatus comprising one or more units for performing the method according to any one of Clauses 1A-22A.

[0127] Clause 25A. The device according to Clause 24A, wherein the one or more units include one or more processors implemented in a circuit.

[0128] Clause 26A. The device according to Clause 24A or Clause 25A further includes: a memory for storing data representing the point cloud.

[0129] Clause 27A. The device according to any one of Clauses 24A-26A, wherein the device includes a decoder.

[0130] Clause 28A. The device according to any one of Clauses 24A-26A, wherein the device includes an encoder.

[0131] Clause 29A. The apparatus according to any one of Clauses 24A-28A further includes: an apparatus for generating the point cloud.

[0132] Clause 30A. The device according to any one of Clauses 20A-29A further includes: a display for presenting an image based on the point cloud.

[0133] Clause 31A. A computer-readable storage medium having instructions stored thereon, which, when executed, cause one or more processors to perform the method according to any one of Clauses 1A-23A.

[0134] Clause 1B. A method for decoding point cloud data, the method comprising: determining a value of a time identifier (ID) of a syntax structure; and decoding the point cloud data based on the value of the time ID, wherein the time ID of the syntax structure identifies a time layer to which the point cloud data associated with the syntax structure belongs.

[0135] Clause 2B. The method described in Clause 1B, wherein the syntactic structure is associated with a slice or data unit.

[0136] Clause 3B. The method described in Clause 2B, wherein the syntax structure is associated with geometric data units.

[0137] Clause 4B. The method according to Clause 3B further comprises: inferring that the value of a time ID associated with an attribute data unit corresponding to the geometric data unit is equal to the value of the time ID associated with the geometric data unit, wherein decoding the point cloud data based on the value of the time ID includes decoding the geometric data unit and decoding the attribute data unit.

[0138] Clause 5B. The method according to any one of Clauses 1B-4B further includes: determining the value of a flag indicating whether the time ID is within the syntax structure.

[0139] Clause 6B. A method according to any one of Clauses 1B-5B, wherein the time ID is a first time ID and the syntax structure is associated with a first slice of a frame, the method further comprising: determining that a second time ID associated with a second slice of the frame is equal to the first time ID; and decoding the second slice of the frame based on the second time ID.

[0140] Clause 7B. A method according to any one of Clauses 1B-5B, wherein the syntax structure is a first syntax structure associated with an independent slice, and the time ID is a first time ID, the method further comprising: determining a second time ID associated with a dependent slice, the dependent slice depending on the independent slice; and decoding the dependent slice based on the independent slice and the second time ID, wherein the second time ID is equal to or greater than the first time ID.

[0141] Clause 8B. The method according to any one of Clauses 1B-7B, wherein the point cloud data comprises point cloud data of a plurality of frames, wherein the point cloud data of a first frame of the plurality of frames of point cloud data comprises a plurality of slices, and wherein each of the plurality of slices is associated with the same time ID.

[0142] Clause 9B. The method according to any one of Clauses 1B-5B, wherein the point cloud data includes point cloud data of a first frame and point cloud data of a second frame, wherein the point cloud data of the first frame includes a first slice associated with a first time ID t1, and the point cloud data of the second frame includes a second slice associated with a second time ID t2, wherein t2>t1, the method further comprising: decoding the first slice without relying on the second slice.

[0143] Clause 10B. A method for encoding point cloud data, the method comprising: determining a value of a time identifier (ID) of a syntax structure; and encoding the point cloud data to include the time ID in the syntax structure, wherein the time ID of the syntax structure identifies a time layer to which the point cloud data associated with the syntax structure belongs.

[0144] Clause 11B. The method described in Clause 10B, wherein the syntactic structure is associated with a slice or data unit.

[0145] Clause 12B. The method according to Clause 11B, wherein the syntax structure is associated with a geometric data unit.

[0146] Clause 13B. The method described in Clause 12B further includes: avoiding signaling the value of a time ID associated with an attribute data unit corresponding to the geometric data unit.

[0147] Clause 14B. The method according to any one of Clauses 10B-13B further includes: signaling a flag whose value indicates the time ID in the syntax structure.

[0148] Clause 15B. The method according to any one of Clauses 10B-14B, wherein the time ID is a first time ID and the syntax structure is a first syntax structure associated with a first slice of a frame, the method further comprising: determining that a second time ID associated with a second slice of the frame is equal to the first time ID; and encoding a second syntax structure associated with the second slice to include the second time ID.

[0149] Clause 16B. The method according to any one of Clauses 10B-15B, wherein the syntax structure is a first syntax structure associated with an independent slice, and the time ID is a first time ID, the method further comprising: determining a second time ID associated with a dependent slice, the dependent slice depending on the independent slice; and encoding the point cloud data to include the second time ID in a second syntax structure for the dependent slice, wherein the second time ID is equal to or greater than the first time ID.

[0150] Clause 17B. The method according to any one of Clauses 10B-16B, wherein the point cloud data comprises point cloud data of a plurality of frames, wherein the point cloud data of a first frame of the plurality of frames of point cloud data comprises a plurality of slices, and wherein each of the plurality of slices is associated with the same time ID.

[0151] Clause 18B. The method according to any one of Clauses 10B-14B, wherein the point cloud data includes point cloud data of a first frame and point cloud data of a second frame, wherein the point cloud data of the first frame includes a first slice associated with a first time ID t1, and the point cloud data of the second frame includes a second slice associated with a second time ID t2, wherein t2>t1, the method further comprising: encoding the first slice such that a decoder decodes the first slice without relying on the second slice.

[0152] Clause 19B. An apparatus for decoding point cloud data, the apparatus comprising: a memory configured to store the point cloud data; and one or more processors implemented in a circuit and communicatively coupled to the memory, the one or more processors being configured to: determine a value of a time identifier (ID) of a syntax structure; and decode the point cloud data based on the value of the time ID, wherein the time ID of the syntax structure identifies a time layer to which the point cloud data associated with the syntax structure belongs.

[0153] Clause 20B. The device as described in Clause 19B, wherein the syntax structure is associated with a slice or data unit.

[0154] Clause 21B. The device according to Clause 20B, wherein the syntax structure is associated with a geometric data unit.

[0155] Clause 22B. The apparatus according to Clause 21B, wherein the one or more processors are further configured to: infer that the value of a time ID associated with an attribute data unit corresponding to the geometric data unit is equal to the value of the time ID associated with the geometric data unit, wherein, as part of decoding the point cloud data based on the value of the time ID, the one or more processors are configured to: decode the geometric data unit and decode the attribute data unit.

[0156] Clause 23B. The device according to any one of Clauses 19B-22B, wherein the one or more processors are further configured to: determine the value of a flag indicating whether the time ID is within the syntax structure.

[0157] Clause 24B. An apparatus according to any one of Clauses 19B-23B, wherein the time ID is a first time ID, and the syntax structure is associated with a first slice of a frame, and wherein the one or more processors are further configured to: determine that a second time ID associated with a second slice of the frame is equal to the first time ID; and decode the second slice of the frame based on the second time ID.

[0158] Clause 25B. An apparatus according to any one of Clauses 19B-23B, wherein the syntax structure is a first syntax structure associated with an independent slice, and the time ID is a first time ID, and wherein the one or more processors are further configured to: determine a second time ID associated with a dependent slice, the dependent slice depending on the independent slice; and decode the dependent slice based on the independent slice and the second time ID, wherein the second time ID is equal to or greater than the first time ID.

[0159] Clause 26B. The device according to any one of Clauses 19B-25B, wherein the point cloud data comprises point cloud data of a plurality of frames, wherein the point cloud data of a first frame of the plurality of frames of point cloud data comprises a plurality of slices, and wherein each of the plurality of slices is associated with the same time ID.

[0160] Clause 27B. An apparatus according to any one of Clauses 19B-23B, wherein the point cloud data comprises point cloud data of a first frame and point cloud data of a second frame, wherein the point cloud data of the first frame comprises a first slice associated with a first time ID t1, and the point cloud data of the second frame comprises a second slice associated with a second time ID t2, wherein t2 > t1, and wherein the one or more processors are further configured to decode the first slice without relying on the second slice.

[0161] Clause 28B. The device according to any one of Clauses 19B-27B further includes: a display configured to display the point cloud data.

[0162] Clause 29B. An apparatus for encoding point cloud data, the apparatus comprising: a memory configured to store the point cloud data; and one or more processors implemented in a circuit and communicatively coupled to the memory, the one or more processors being configured to: determine a value of a time identifier (ID) of a syntax structure; and encode the point cloud data to include the time ID in the syntax structure, wherein the time ID of the syntax structure identifies a time layer to which the point cloud data associated with the syntax structure belongs.

[0163] Clause 30B. The device as described in Clause 29B, wherein the syntax structure is associated with a slice or data unit.

[0164] Clause 31B. The device according to Clause 30B, wherein the syntax structure is associated with a geometric data unit.

[0165] Clause 32B. The device according to Clause 29B, wherein the one or more processors are further configured to: avoid signaling the value of a time ID associated with an attribute data unit corresponding to the geometric data unit.

[0166] Clause 33B. The device according to Clauses 29B-32B, wherein the one or more processors are further configured to: signal a value for a flag indicating the time ID in the syntax structure.

[0167] Clause 34B. An apparatus according to any one of Clauses 29B-33B, wherein the time ID is a first time ID, and the syntax structure is a first syntax structure associated with a first slice of a frame, and wherein the one or more processors are further configured to: determine that a second time ID associated with a second slice of the frame is equal to the first time ID; and encode a second syntax structure associated with the second slice to include the second time ID.

[0168] Clause 35B. An apparatus according to any one of Clauses 29B-33B, wherein the syntax structure is a first syntax structure associated with an independent slice, and the time ID is a first time ID, and wherein the one or more processors are further configured to: determine a second time ID associated with a dependent slice, the dependent slice depending on the independent slice; and encode the point cloud data to include the second time ID in a second syntax structure for the dependent slice, wherein the second time ID is equal to or greater than the first time ID.

[0169] Clause 36B. The device according to any one of Clauses 29B-35B, wherein the point cloud data comprises point cloud data of a plurality of frames, wherein the point cloud data of a first frame of the plurality of frames of point cloud data comprises a plurality of slices, and wherein each of the plurality of slices is associated with the same time ID.

[0170] Clause 37B. An apparatus according to any one of Clauses 29B-33B, wherein the point cloud data comprises point cloud data of a first frame and point cloud data of a second frame, wherein the point cloud data of the first frame comprises a first slice associated with a first time ID t1, and the point cloud data of the second frame comprises a second slice associated with a second time ID t2, wherein t2 > t1, and wherein the one or more processors are further configured to: encode the first slice such that a decoder decodes the first slice without relying on the second slice.

[0171] Clause 38B. Equipment according to any one of Clauses 29B-37B, wherein said equipment includes a vehicle.

[0172] It should be recognized that, depending on the example, certain actions or events of any of the techniques described herein may be performed in a different order, and certain actions or events of any of the techniques described herein may be added, combined, or omitted entirely (e.g., not all described actions or events are necessary for the implementation of the techniques). Furthermore, in some examples, actions or events may be performed concurrently rather than sequentially, for example, through multithreading, interrupt handling, or multiple processors.

[0173] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored or transmitted as one or more instructions or code on or through a computer-readable medium, and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium or a communication medium, including any medium that facilitates the transfer of a computer program from one place to another, for example, according to a communication protocol. In this way, a computer-readable medium may generally correspond to: (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier waveform. A data storage medium may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, code, and / or data structures used for implementing the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0174] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Additionally, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waveforms, signals, or other transient media, but instead refer to non-transient, tangible storage media. As used herein, disks and optical discs include compact optical discs (CDs), laser optical discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, where disks typically magnetically copy data, while optical discs utilize lasers to optically copy data. The above combinations should also be included within the scope of computer-readable media.

[0175] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, as used herein, the terms "processor" and "processing circuitry" can refer to any of the foregoing structures or any other structures suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into combined codecs. Additionally, the techniques can be fully implemented in one or more circuit or logic elements.

[0176] The techniques disclosed herein can be implemented in a wide variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or collections of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but are not necessarily required to be implemented by different hardware units. Rather, as described above, various units can be combined in codec hardware units or provided via a set of interoperable hardware units (including one or more processors as described above) working in conjunction with suitable software and / or firmware.

[0177] Examples have been described. These and other examples are within the scope of the appended claims.

Claims

1. A method of decoding point cloud data, the method comprising: determining a value of a temporal identifier (ID) of a syntax structure associated with a geometry data unit, wherein the temporal ID of the syntax structure identifies a temporal layer to which point cloud data associated with the syntax structure belongs; inferring that a value of a temporal ID associated with an attribute data unit corresponding to the geometry data unit is equal to the value of the temporal ID associated with the geometry data unit; and decoding the point cloud data based on the value of the temporal ID associated with the geometry data unit and the value of the temporal ID associated with the attribute data unit.

2. The method of claim 1, wherein, decoding the point cloud data based on the value of the temporal ID associated with the geometry data unit and the value of the temporal ID associated with the attribute data unit comprises decoding the geometry data unit and decoding the attribute data unit.

3. The method of claim 1, further comprising: determining a value of a flag indicating whether the temporal ID is in the syntax structure.

4. The method of claim 1, wherein, the temporal ID is a first temporal ID and the syntax structure is associated with a first slice of a frame, the method further comprising: determining that a second temporal ID associated with a second slice of the frame is equal to the first temporal ID; and decoding the second slice of the frame based on the second temporal ID.

5. The method of claim 1, wherein, the syntax structure is a first syntax structure associated with an independent slice and the temporal ID is a first temporal ID, the method further comprising: determining a second temporal ID associated with a dependent slice, the dependent slice being dependent on the independent slice; and decoding the dependent slice based on the independent slice and the second temporal ID, wherein the second temporal ID is equal to or greater than the first temporal ID.

6. The method of claim 1, wherein, the point cloud data comprises point cloud data of a plurality of frames, wherein point cloud data of a first frame of the plurality of frames of point cloud data comprises a plurality of slices, and wherein each slice of the plurality of slices is associated with a same temporal ID.

7. The method of claim 1, wherein, the point cloud data comprises point cloud data of a first frame and point cloud data of a second frame, wherein the point cloud data of the first frame comprises a first slice associated with a first temporal ID tl and the point cloud data of the second frame comprises a second slice associated with a second temporal ID t2, wherein t2>tl, the method further comprising: decoding the first slice without dependence on the second slice.

8. A method of encoding point cloud data, the method comprising: determining a value of a temporal identifier (ID) of a syntax structure associated with a geometry data unit, wherein the temporal ID of the syntax structure identifies a temporal layer to which point cloud data associated with the syntax structure belongs; avoiding signaling a value of a temporal ID associated with an attribute data unit corresponding to the geometry data unit; and encoding the point cloud data to include the temporal ID associated with the geometry data unit in the syntax structure.

9. The method of claim 8, further comprising: signaling a flag whose value indicates that the temporal ID is in the syntax structure.

10. The method of claim 8, wherein, the temporal ID is a first temporal ID and the syntax structure is a first syntax structure associated with a first slice of a frame, the method further comprising: determining that a second temporal ID associated with a second slice of the frame is equal to the first temporal ID; and encoding a second syntax structure associated with the second slice to include the second temporal ID.

11. The method of claim 8, wherein, the syntax structure is a first syntax structure associated with an independent slice and the temporal ID is a first temporal ID, the method further comprising: determining a second temporal ID associated with a dependent slice that depends on the independent slice; and encoding the point cloud data to include the second temporal ID in a second syntax structure for the dependent slice, wherein the second temporal ID is equal to or greater than the first temporal ID.

12. The method of claim 8, wherein, the point cloud data comprises point cloud data of a plurality of frames, wherein point cloud data of a first frame of the plurality of frames comprises a plurality of slices, and wherein each slice of the plurality of slices is associated with a same temporal ID.

13. The method of claim 8, wherein, the point cloud data comprises point cloud data of a first frame and point cloud data of a second frame, wherein the point cloud data of the first frame comprises a first slice associated with a first temporal ID ti and the point cloud data of the second frame comprises a second slice associated with a second temporal ID t2, wherein t2 > ti, the method further comprising: encoding the first slice such that a decoder decodes the first slice without relying on the second slice.

14. A device for decoding point cloud data, the device comprising: a memory configured to store the point cloud data; and one or more processors implemented in circuitry and communicatively coupled to the memory, the one or more processors configured to: determine a value of a temporal identifier (ID) of a syntax structure associated with a geometry data unit, wherein the temporal ID of the syntax structure identifies a temporal layer to which point cloud data associated with the syntax structure belongs; infer that a value of a temporal ID associated with an attribute data unit corresponding to the geometry data unit is equal to the value of the temporal ID associated with the geometry data unit; and decode the point cloud data based on the value of the temporal ID associated with the geometry data unit and the value of the temporal ID associated with the attribute data unit.

15. The apparatus of claim 14, wherein, as part of decoding the point cloud data based on the value of the temporal ID associated with the geometry data unit and the value of the temporal ID associated with the attribute data unit, the one or more processors are configured to decode the geometry data unit and decode the attribute data unit.

16. The apparatus of claim 14, wherein, the one or more processors are further configured to: determining a value of a flag indicating whether the temporal ID is in the syntax structure.

17. The apparatus of claim 14, wherein, the temporal ID is a first temporal ID and the syntax structure is associated with a first slice of a frame, and wherein the one or more processors are further configured to: determine that a second temporal ID associated with a second slice of the frame is equal to the first temporal ID; and decode the second slice of the frame based on the second temporal ID.

18. The apparatus of claim 14, wherein, the syntax structure is a first syntax structure associated with an independent slice and the temporal ID is a first temporal ID, and wherein the one or more processors are further configured to: determine a second temporal ID associated with a dependent slice that depends on the independent slice; and decode the dependent slice based on the independent slice and the second temporal ID, wherein the second temporal ID is equal to or greater than the first temporal ID.

19. The apparatus of claim 14, wherein, the point cloud data includes point cloud data of a plurality of frames, wherein point cloud data of a first frame of the plurality of frames of point cloud data includes a plurality of slices, and wherein each slice of the plurality of slices is associated with a same temporal ID.

20. The apparatus of claim 14, wherein, the point cloud data includes point cloud data of a first frame and point cloud data of a second frame, wherein the point cloud data of the first frame includes a first slice associated with a first temporal ID ti, and the point cloud data of the second frame includes a second slice associated with a second temporal ID t2, where t2>ti, and wherein the one or more processors are further configured to: decode the first slice without dependence on the second slice.

21. The device of claim 14, further comprising a display configured to display the point cloud data.

22. A device for encoding point cloud data, the device comprising: a memory configured to store the point cloud data; and one or more processors implemented in circuitry and communicatively coupled to the memory, the one or more processors configured to: determine a value of a temporal identifier (ID) of a syntax structure associated with a geometry data unit, wherein the temporal ID of the syntax structure identifies a temporal layer to which point cloud data associated with the syntax structure belongs; avoid signaling a value of a temporal ID associated with an attribute data unit corresponding to the geometry data unit; and encode the point cloud data to include the temporal ID associated with the geometry data unit in the syntax structure.

23. The apparatus of claim 22, wherein, the one or more processors are further configured to: signal a flag indicating whether the temporal ID is in the syntax structure with a value of the flag.

24. The apparatus of claim 22, wherein, the temporal ID is a first temporal ID and the syntax structure is a first syntax structure associated with a first slice of a frame, and wherein the one or more processors are further configured to: determine that a second temporal ID associated with a second slice of the frame is equal to the first temporal ID; and encode a second syntax structure associated with the second slice to include the second time ID.

25. The apparatus of claim 22, wherein, The syntax structure is a first syntax structure associated with an independent slice and the time ID is a first time ID, and wherein the one or more processors are further configured to: determine a second time ID associated with a dependent slice that depends on the independent slice; and encode the point cloud data to include the second time ID in a second syntax structure for the dependent slice, wherein the second time ID is equal to or greater than the first time ID.

26. The apparatus of claim 22, wherein, The point cloud data includes point cloud data for a plurality of frames, wherein point cloud data for a first frame of the plurality of frames of point cloud data includes a plurality of slices, and wherein each slice of the plurality of slices is associated with a same time ID.

27. The apparatus of claim 22, wherein, The point cloud data includes point cloud data for a first frame and point cloud data for a second frame, wherein the point cloud data for the first frame includes a first slice associated with a first time ID ti, and the point cloud data for the second frame includes a second slice associated with a second time ID t2, wherein t2 > ti, and wherein the one or more processors are further configured to: encode the first slice such that a decoder decodes the first slice without relying on the second slice.

28. The apparatus of claim 22, wherein, The device includes a vehicle.

Images