Decoding attribute values ​​in geometry-based point cloud compression

Abstract

A device for processing point cloud data includes: determining a first attribute value for a first point of the point cloud, the first point being a previously decoded point closest to a current point of the point cloud; determining second and third attribute values ​​for a second and third point of the point cloud, the second and third points being the second and third nearest previously decoded points; determining a fourth attribute value for a fourth point of the point cloud, the fourth point being a previously decoded point that is farther from the current point than the third point or has the same distance to the current point as the third point; and generating a set of candidate predictors having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of a location of the second point and a location of the third point; The device is configured to:

Description

[Technical Field]

[0001]

[0001] This application claims priority to U.S. Patent Application No. 18 / 488,816, filed October 17, 2023, and U.S. Provisional Patent Application No. 63 / 380,222, filed October 19, 2022, the entire contents of which are incorporated herein by reference. U.S. Patent Application No. 18 / 488,816, filed October 17, 2023, claims the benefit of U.S. Provisional Patent Application No. 63 / 380,222, filed October 19, 2022.

[0002]

[0002] The present disclosure relates to point cloud encoding and decoding. [Background technology]

[0003]

[0003] A point cloud is a collection of points in three-dimensional space. The points may correspond to points on an object in three-dimensional space. Thus, a point cloud may be used to represent the physical content of a three-dimensional space. Point clouds may be useful in a wide variety of situations. For example, a point cloud may be used to represent the location of an object on a road in the context of an autonomous vehicle. In another example, a point cloud may be used to position virtual objects in an augmented reality (AR) or mixed reality (MR) application in the context of representing the physical content of an environment. Point cloud compression is the process of encoding and decoding a point cloud. Encoding a point cloud may reduce the amount of data required for storage and transmission of the point cloud. Summary of the Invention

[0004]

[0004] To predict the value of an attribute, the G-PCC encoder and the G-PCC decoder may be configured to follow the same list-building process so that they each generate the same list of predictor candidates. The G-PCC encoder may then signal to the G-PCC decoder which candidates in the list should be used as predictors. The G-PCC coding device may generate an initial list having M predictor candidates corresponding to the M closest points to the current point, e.g., the closest M points in terms of distance. M may be equal to 3, for example, or some other integer value. Based on the locations of the M predictor candidates relative to the current point and each other, a candidate from the M predictor candidates may be replaced by another candidate that is farther from the current point but may provide a better prediction due to its location. This replacement process generally provides better prediction, and therefore better compression, by generating a list of candidates that is more likely to include predictor candidates with values ​​close to the actual attribute value.

[0005]

[0005] However, in some specific coding scenarios, this substitution process may result in low-quality predictions and therefore low-quality compression. The present disclosure describes techniques for preventing the substitution process from being invoked in scenarios where substitution is more likely to reduce prediction quality, while still performing substitution in scenarios where substitution is more likely to increase prediction quality. For example, according to the techniques of the present disclosure, a G-PCC coding device may be configured to generate a set of predictor candidates based on a comparison between the location of a second point and the location of a third point, where the second point of the point cloud is the previously decoded point that is second-closest to the current point of the point cloud, and the third point of the point cloud is the previously decoded point that is third-closest to the current point of the point cloud. By performing or preventing substitution based on a comparison between the location of the second point and the location of the third point, rather than based solely on the relative location of the previously decoded point that is closest to the current point, a G-PCC coding device configured to perform the techniques of the present disclosure can achieve better predictions and therefore better compression.

[0006]

[0006] According to one example of the present disclosure, a device for processing point cloud data includes a memory configured to store the point cloud data and one or more processors implemented in a circuit, wherein the one or more processors determine a first attribute value for a first point of the point cloud, where the first point of the point cloud is the closest already decoded point to a current point of the point cloud; determine a second attribute value for a second point of the point cloud, where the second point of the point cloud is the second closest already decoded point to the current point of the point cloud; determine a third attribute value for a third point of the point cloud, where the third point of the point cloud is the third closest already decoded point to the current point of the point cloud; and determine a fourth attribute value for a fourth point of the point cloud, where the fourth point of the point cloud is determining a set of predictor candidates for attribute values ​​of the current point of the point cloud, wherein the current point defines an intersection of an x-axis, a y-axis, and a z-axis, the x-axis, the y-axis, and the z-axis forming an xy-plane, an xz-plane, and a yz-plane, and to determine the set of predictor candidates for the current point of the point cloud, the one or more processors are further configured to generate a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point; and decoding the attribute values ​​of the current point based on the set of predictor candidates.

[0007] According to one example of the present disclosure, a method for processing point cloud data includes determining a first attribute value for a first point of the point cloud, where the first point of the point cloud is the closest already decoded point to a current point of the point cloud; determining a second attribute value for a second point of the point cloud, where the second point of the point cloud is the second-closest already decoded point to the current point of the point cloud; determining a third attribute value for a third point of the point cloud, where the third point of the point cloud is the third-closest already decoded point to the current point of the point cloud; and determining a fourth attribute value for a fourth point of the point cloud, where the fourth point of the point cloud is farther from the current point than the third point. determining a set of predictor candidates for attribute values ​​of a current point of the point cloud, where the current point defines an intersection of an x-axis, a y-axis, and a z-axis, the x-axis, the y-axis, and the z-axis forming an xy-plane, an xz-plane, and a yz-plane, to determine the set of predictor candidates for the current point of the point cloud, wherein determining the set of predictor candidates includes generating a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point; and decoding the attribute values ​​of the current point based on the set of predictor candidates.

[0008]

[0008] A computer readable storage medium stores instructions that, when executed by one or more processors, cause the one or more processors to perform the following operations: determine a first attribute value for a first point of the point cloud, where the first point of the point cloud is the closest already decoded point to a current point of the point cloud; determine a second attribute value for a second point of the point cloud, where the second point of the point cloud is the second-closest already decoded point to the current point of the point cloud; determine a third attribute value for a third point of the point cloud, where the third point of the point cloud is the third-closest already decoded point to the current point of the point cloud; and determine a fourth attribute value for a fourth point of the point cloud, where the fourth point of the point cloud is the closest point from the current point to the third point. or a third point is an already decoded point that is the same distance to the current point as the third point; determining a set of predictor candidates for attribute values ​​of the current point of the point cloud, wherein the current point defines an intersection of an x-axis, a y-axis, and a z-axis, the x-axis, y-axis, and z-axis forming an xy-plane, an xz-plane, and a yz-plane; and to determine the set of predictor candidates for the current point of the point cloud, the one or more processors are further configured to generate a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point; and decoding the attribute values ​​of the current point based on the set of predictor candidates.

[0009]

[0009] According to one example of the present disclosure, a device for processing point cloud data includes means for determining a first attribute value for a first point of the point cloud, where the first point of the point cloud is the closest already decoded point to a current point of the point cloud; means for determining a second attribute value for a second point of the point cloud, where the second point of the point cloud is the second closest already decoded point to the current point of the point cloud; means for determining a third attribute value for a third point of the point cloud, where the third point of the point cloud is the third closest already decoded point to the current point of the point cloud; and means for determining a fourth attribute value for a fourth point of the point cloud, where the fourth point of the point cloud is farther from the current point than the third point. the current point of the point cloud is an already decoded point; means for determining a set of predictor candidates for attribute values ​​of a current point of the point cloud, the current point defining an intersection of an x-axis, a y-axis, and a z-axis, the x-axis, y-axis, and z-axis forming an xy-plane, an xz-plane, and a yz-plane, to determine the set of predictor candidates for the current point of the point cloud, the means for determining the set of predictor candidates including means for generating a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point; and means for decoding the attribute values ​​of the current point based on the set of predictor candidates.

[0010]

[0010] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will become apparent from the description, drawings, and claims. [Brief explanation of the drawings]

[0011] [Figure 1]

[0011] FIG. 1 is a block diagram illustrating an exemplary encoding and decoding system that may implement the techniques of this disclosure. [Figure 2]

[0012] FIG. 1 is a block diagram illustrating an example Geometry Point Cloud Compression (G-PCC) encoder. [Figure 3]

[0013] FIG. 2 is a block diagram illustrating an exemplary G-PCC decoder. [Figure 4]

[0014] FIG. 1 is a conceptual diagram illustrating an exemplary octree partitioning for geometry coding. [Figure 5]

[0015] 3 is a block diagram illustrating the example geometry encoding unit of FIG. 2 in more detail. [Figure 6]

[0016] 3 is a block diagram illustrating the exemplary attribute encoding unit of FIG. 2 in more detail. [Figure 7]

[0017] 4 is a block diagram illustrating the example geometry decoding unit of FIG. 3 in more detail. [Figure 8]

[0018] 4 is a block diagram illustrating the example attribute decoding unit of FIG. 3 in more detail. [Figure 9]

[0019] FIG. 1 is a conceptual diagram illustrating an example of a prediction tree. [Figure 10A]

[0020] FIG. 1 is a conceptual diagram illustrating an example of a rotational Light Detection and Ranging (LIDAR) acquisition model. [Figure 10B] FIG. 1 is a conceptual diagram illustrating an example of a rotational Light Detection and Ranging (LIDAR) acquisition model. [Figure 11]

[0021] An example of inter-predicting a current point from a point on a reference frame will be shown. [Figure 12]

[0022] 10 is a flow diagram illustrating an example decoding flow associated with a syntax element indicating whether a node is coded in inter-prediction mode or intra-prediction mode. [Figure 13]

[0023] 10 shows an additional inter predictor point obtained from the first point that has a larger azimuth angle than the inter predictor point. [Figure 14]

[0024] 1 shows a flow diagram illustrating the motion compensation process when reference frames are stored in the spherical domain and motion compensation is performed in the Cartesian domain. [Figure 15]

[0025] 1 shows a reference structure for intra-level-of-detail (LOD) prediction. [Figure 16]

[0026] Here is an example of processing neighbors in subsequent LODs that have the same distance from the current point. [Figure 17]

[0027] Here is an example of processing neighbors in the same LOD that have the same distance from the current point. [Figure 18]

[0028] An example of an actual nearest neighbor with a significant jump in terms of Morton order is shown. [Figure 19]

[0029] 1 shows an example of a voxel neighbor. [Figure 20]

[0030] An example of space division is shown below. [Figure 21]

[0031] An example of a neighborhood point distribution is shown below. [Figure 22]

[0032] 10 shows an example of generating List 1 and List 2 for performing a neighbor discovery process for attribute LOD prediction. [Figure 23]

[0033] Illustrates the concept of opposite directions. [Figure 24]

[0034] 10 is a flowchart illustrating an example operation of a G-PCC decoder, in accordance with one or more techniques of this disclosure. [Figure 25]

[0035] FIG. 1 is a conceptual diagram illustrating an example distance measurement system that may be used with one or more techniques of the present disclosure. [Figure 26]

[0036] FIG. 1 is a conceptual diagram illustrating an example vehicle-based scenario in which one or more techniques of this disclosure may be used. [Figure 27]

[0037] FIG. 1 is a conceptual diagram illustrating an example extended reality system in which one or more techniques of this disclosure may be used. [Figure 28]

[0038] FIG. 1 is a conceptual diagram illustrating an example mobile device system in which one or more techniques of the present disclosure may be used. DETAILED DESCRIPTION OF THE INVENTION

[0012]

[0039] "Geometry-based point cloud compression" (G-PCC) directly compresses 3D geometry, i.e., the locations of a set of points in 3D space. G-PCC also compresses associated attribute values. Attributes can be, for example, color information such as R / G / B, Y / Cb / Cr, reflectance information, or other attributes such as reflectance, temperature values, humidity values, latitude coordinates, and longitude coordinates. Point clouds can be captured by various cameras or sensors, such as light detection and ranging (LIDAR) scanners or 3D scanners, or can also be computer-generated. Point cloud data can be used in a variety of applications, including, but not limited to, construction (e.g., modeling), graphics (e.g., 3D models for visualization and animation), and the automotive industry (e.g., LIDAR sensors used to aid navigation).

[0013]

[0040] According to one technique for compressing attribute values, a coding device (e.g., a G-PCC encoder or decoder) can predict an attribute value of a current point based on the attribute values ​​of previously coded points. The residual value, i.e., the difference between the predicted and actual attribute values ​​of the current point, is then signaled in the bitstream, allowing the G-PCC decoder to determine the decoded attribute value as the sum of the predicted and residual values. When lossless compression is used, the decoded attribute value is equal to the actual attribute value before compression. For example, when lossy compression is used, due to quantizing the residual value, the decoded attribute value may differ from the actual attribute value before compression, but generally may be relatively close to the actual attribute value before compression.

[0014]

[0041] To determine the predicted value of an attribute, the G-PCC encoder and the G-PCC decoder may be configured to follow the same list-building process so that each generates the same list of predictor candidates. The G-PCC encoder may then signal to the G-PCC decoder which candidate in the list should be used as the predictor. The G-PCC coding device may generate an initial list having M predictor candidates corresponding to the M points closest to the current point. M may be equal to 3, for example, or some other integer value. Based on the locations of the M predictor candidates relative to the current point and each other, a candidate from the M predictor candidates may be replaced by another candidate that is farther from the current point but may provide a better prediction due to its location. This replacement process generally provides better prediction, and therefore better compression, by generating a list of candidates that is more likely to include predictor candidates with values ​​close to the actual attribute value.

[0015]

[0042] However, in some specific coding scenarios, this substitution process may result in poor prediction and therefore poor compression. This disclosure describes techniques for preventing the substitution process from being invoked for scenarios where substitution is more likely to reduce prediction quality, while still performing substitution for scenarios where substitution is more likely to improve prediction quality. For example, according to the techniques of this disclosure, a G-PCC coding device may be configured to generate a set of predictor candidates based on a comparison between the location of a second point and the location of a third point, where the second point of the point cloud is the already decoded point second closest to the current point of the point cloud, and the third point of the point cloud is the already decoded point third closest to the current point of the point cloud. By performing or preventing substitution based on a comparison between the location of the second point and the location of the third point, rather than based solely on the relative position of the already decoded point closest to the current point, a G-PCC coding device configured to perform the techniques of this disclosure can achieve better prediction and therefore better compression.

[0016]

[0043] 1 is a block diagram illustrating an example encoding and decoding system 100 that may implement the techniques of this disclosure. The techniques of this disclosure generally relate to coding (encoding and / or decoding) point cloud data, i.e., supporting point cloud compression. Generally, point cloud data includes any data for processing a point cloud. Coding may be useful for compressing and / or decompressing the point cloud data.

[0017]

[0044] As shown in Figure 1, system 100 includes a source device 102 and a destination device 116. Source device 102 provides encoded point cloud data that is decoded by destination device 116. Specifically, in the example of Figure 1, source device 102 provides the point cloud data to destination device 116 via computer-readable medium 110. Source device 102 and destination device 116 may include any of a wide range of devices, including desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, telephone handsets such as smartphones, televisions, cameras, display devices, digital media players, video gaming consoles, video streaming devices, terrestrial or marine vehicles, spacecraft, aircraft, robots, LIDAR devices, satellites, etc. In some cases, source device 102 and destination device 116 may be capable of wireless communication.

[0018]

[0045] In the example of FIG. 1 , source device 102 includes data source 104, memory 106, G-PCC encoder 200, and output interface 108. Destination device 116 includes input interface 122, G-PCC decoder 300, memory 120, and data consumer 118. According to this disclosure, G-PCC encoder 200 of source device 102 and G-PCC decoder 300 of destination device 116 may be configured to apply techniques of this disclosure related to predictive geometry coding. Thus, source device 102 represents an example of an encoding device, and destination device 116 represents an example of a decoding device. In other examples, source device 102 and destination device 116 may include other components or arrangements. For example, source device 102 may receive data (e.g., point cloud data) from an internal or external source. Similarly, destination device 116 may interface with an external data consumer rather than including the data consumer within the same device.

[0019]

[0046] The system 100 as shown in FIG. 1 is merely an example. In general, other digital encoding and / or decoding devices may perform the techniques of this disclosure related to predictive geometry coding. The source device 102 and the destination device 116 are merely examples of devices in which the source device 102 generates coded data for transmission to the destination device 116. This disclosure refers to devices that perform coding (encoding and / or decoding) of data as “coding” devices. Thus, the G-PCC encoder 200 and the G-PCC decoder 300 represent examples of coding devices, specifically, encoders and decoders, respectively. In some examples, the source device 102 and the destination device 116 may operate substantially symmetrically, such that each of the source device 102 and the destination device 116 includes encoding and decoding components. Thus, the system 100 may support one-way or two-way transmission between the source device 102 and the destination device 116, for example, streaming, playback, broadcasting, telephony, navigation, and other uses.

[0020]

[0047] Generally, the data source 104 represents a source of data (i.e., raw, unencoded point cloud data) and may provide a sequential series of “frames” of data to the G-PCC encoder 200, which encodes the frames of data. The data source 104 of the source device 102 may include a point cloud capture device, such as any of a variety of cameras or sensors, e.g., a 3D scanner or light detection and ranging (LIDAR) device, one or more video cameras, an archive containing previously captured data, and / or a data feed interface for receiving data from a data content provider. Alternatively or additionally, the point cloud data may be computer-generated from scanner data, camera data, sensor data, or other data. For example, the data source 104 may generate computer-graphics-based data as source data, or may generate a combination of live data, archived data, and computer-generated data. In each case, the G-PCC encoder 200 encodes the captured data, pre-captured data, or computer-generated data. The G-PCC encoder 200 may reorder frames from reception order (sometimes called "display order") to coding order for coding. The G-PCC encoder 200 may generate one or more bitstreams containing the encoded data. The source device 102 may then output the encoded data onto a computer-readable medium 110 via an output interface 108 for receipt and / or retrieval by, for example, an input interface 122 of a destination device 116.

[0021]

[0048] The memory 106 of the source device 102 and the memory 120 of the destination device 116 may represent general-purpose memory. In some examples, the memory 106 and the memory 120 may store raw data, e.g., raw data from the data source 104 and raw decoded data from the G-PCC decoder 300. Additionally or alternatively, the memory 106 and the memory 120 may store software instructions executable by, e.g., the G-PCC encoder 200 and the G-PCC decoder 300, respectively. While the memory 106 and the memory 120 are shown separate from the G-PCC encoder 200 and the G-PCC decoder 300 in this example, it should be understood that the G-PCC encoder 200 and the G-PCC decoder 300 may also include internal memory for functionally similar or equivalent purposes. Furthermore, the memory 106 and the memory 120 may store, e.g., encoded data output from the G-PCC encoder 200 and input to the G-PCC decoder 300. In some examples, portions of memory 106 and memory 120 may be allocated as one or more buffers, e.g., for storing raw, decoded, and / or encoded data. For example, memory 106 and memory 120 may store data representing a point cloud.

[0022]

[0049] The computer-readable medium 110 may represent any type of medium or device capable of transporting encoded data from the source device 102 to the destination device 116. In one example, the computer-readable medium 110 represents a communication medium that enables the source device 102 to transmit encoded data directly to the destination device 116 in real time, for example, via a radio frequency network or a computer-based network. The output interface 108 may modulate, and the input interface 122 may demodulate, a transmission signal containing the encoded data in accordance with a communication standard such as a wireless communication protocol. The communication medium may include any wireless or wired communication medium, such as the radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful for facilitating communication from the source device 102 to the destination device 116.

[0023]

[0050] In some examples, source device 102 may output encoded data from output interface 108 to storage device 112. Similarly, destination device 116 may access encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded data.

[0024]

[0051] In some examples, source device 102 may output the encoded data to a file server 114 or another intermediate storage device, which may store the encoded data generated by source device 102. Destination device 116 may access the stored data from file server 114 via streaming or download. File server 114 may be any type of server device capable of storing encoded data and transmitting the encoded data to destination device 116. File server 114 may represent a web server (e.g., for a website), a File Transfer Protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Destination device 116 may access the encoded data from file server 114 through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both suitable for accessing encoded data stored on file server 114. The file server 114 and the input interface 122 may be configured to operate according to a streaming transmission protocol, a download transmission protocol, or a combination thereof.

[0025]

[0052] Output interface 108 and input interface 122 may represent a wireless transmitter / receiver, a modem, a wired network component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples in which output interface 108 and input interface 122 comprise wireless components, output interface 108 and input interface 122 may be configured to transfer data, such as encoded data, according to a cellular communication standard such as 4G, 4G-LTE (Long-Term Evolution), LTE Advanced, 5G, etc. In some examples in which output interface 108 comprises a wireless transmitter, output interface 108 and input interface 122 may be configured to transfer data, such as encoded data, according to other wireless standards such as the IEEE 802.11 specification, the IEEE 802.15 specification (e.g., ZigBee™), the Bluetooth™ standard, etc. In some examples, source device 102 and / or destination device 116 may include respective system-on-a-chip (SoC) devices. For example, the source device 102 may include an SoC device for implementing functionality attributed to the G-PCC encoder 200 and / or the output interface 108, and the destination device 116 may include an SoC device for implementing functionality attributed to the G-PCC decoder 300 and / or the input interface 122.

[0026]

[0053] The techniques of this disclosure may be applied to encoding and decoding in support of any of a variety of applications, such as communication between autonomous vehicles, communication between scanners, cameras, sensors and processing devices such as local or remote servers, geographic mapping, or other applications.

[0027]

[0054] The input interface 122 of the destination device 116 receives the encoded bitstream from the computer-readable medium 110 (e.g., a communication medium, a storage device 112, a file server 114, etc.). The encoded bitstream may include signaling information defined by the G-PCC encoder 200, such as syntax elements having values ​​that describe the characteristics and / or processing of the coded unit (e.g., a slice, a picture, a group of pictures, a sequence, etc.), which is also used by the G-PCC decoder 300. The data consumer 118 uses the decoded data. For example, the data consumer 118 may use the decoded data to determine the location of a physical object. In some examples, the data consumer 118 may include a display for displaying imagery based on the point cloud.

[0028]

[0055] The G-PCC encoder 200 and the G-PCC decoder 300 may each be implemented as any of a variety of suitable encoder and / or decoder circuitry, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the techniques are implemented partially in software, a device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to implement the techniques of this disclosure. The G-PCC encoder 200 and the G-PCC decoder 300 may each be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (codec) within the respective device. A device including G-PCC encoder 200 and / or G-PCC decoder 300 may comprise one or more integrated circuits, microprocessors, and / or other types of devices.

[0029]

[0056] The G-PCC encoder 200 and the G-PCC decoder 300 may operate according to a coding standard such as the video point cloud compression (V-PCC) standard or the coordinate-based point cloud compression (G-PCC) standard. This disclosure may generally refer to coding (e.g., encoding and decoding) of pictures, which includes the processes of encoding or decoding data. The encoded bitstream generally includes a series of values ​​of syntax elements that represent coding decisions (e.g., coding modes).

[0030]

[0057] This disclosure may generally refer to "signaling" some information, such as a syntax element. The term "signaling" may generally refer to communicating the value of a syntax element and / or other data used to decode the encoded data. That is, the G-PCC encoder 200 may signal the value of a syntax element in a bitstream. Generally, signaling refers to generating a value within the bitstream. As mentioned above, the source device 102 may transfer the bitstream to the destination device 116 in substantially real time or non-real time, which may occur, for example, when storing the syntax element in the storage device 112 for later retrieval by the destination device 116.

[0031]

[0058] ISO / IEC MPEG (JTC1 / SC29 / WG11) is investigating the potential need for and aims to develop a standard for point cloud coding techniques with compression capabilities significantly beyond those of current methods. The group is working together on this exploration in a collaborative effort called the 3D Graphics Team (3DG) to evaluate compression technology designs proposed by those experts in this field.

[0032]

[0059] Point cloud compression activities fall into two distinct approaches. The first approach, "video point cloud compression" (V-PCC), segments a 3D object and projects the segments into multiple 2D planes (represented as "patches" in a 2D frame), which are further coded by a legacy 2D video codec, such as the High Efficiency Video Coding (HEVC) (ITU-T H.265) codec. The second approach, "coordinate-based point cloud compression" (G-PCC), directly compresses 3D coordinates, i.e., the location of a set of points in 3D space, and associated attribute values ​​(for each point associated with a 3D coordinate). G-PCC addresses the compression of point clouds in both category 1 (static point clouds) and category 3 (dynamically acquired point clouds). A recent draft of the G-PCC standard is available at G-PCC DIS, ISO / IEC JTC1 / SC29 / WG11 w19088, Brussels, Belgium, January 2020, and the codec description is available at G-PCC Codec Description v6, ISO / IEC JTC1 / SC29 / WG11 w19091, Brussels, Belgium, January 2020.

[0033]

[0060] A point cloud includes a set of points in 3D space and may have attributes associated with the points. The attributes may be color information such as R, G, B, or Y, Cb, Cr, or reflectance information, or other attributes. Point clouds may be captured by various cameras or sensors, such as LIDAR sensors and 3D scanners, or may be computer-generated. Point cloud data is used in a variety of applications, including, but not limited to, architecture (modeling), graphics (3D models for visualization and animation), and the automotive industry (LIDAR sensors used to aid navigation).

[0034]

[0061] The 3D space occupied by the point cloud data may be enclosed by a virtual bounding box. The positions of points in the bounding box may be represented with a certain precision, and therefore, the positions of one or more points may be quantized based on that precision. At the smallest level, the bounding box is divided into voxels, which are the smallest units of space represented by a unit cube. A voxel in a bounding box may be associated with zero, one, or more points. The bounding box may be divided into multiple cubic / rectangular regions, sometimes called tiles. Each tile may be coded into one or more slices. The partitioning of the bounding box into slices and tiles may be based on the number of points in each section or other considerations (e.g., a particular region may be coded as a tile). The slice regions may be further partitioned using partitioning decisions similar to those in video codecs.

[0035]

[0062] FIG. 2 provides an overview of the G-PCC encoder 200. FIG. 3 provides an overview of the G-PCC decoder 300. The modules shown are logical and do not necessarily correspond one-to-one to implemented code. In the example of FIG. 2, the G-PCC encoder 200 may include a geometry encoding unit 250 and an attribute encoding unit 260. Generally, the geometry encoding unit 250 is configured to encode positions of points in a point cloud frame to generate a geometry bitstream 203. The attribute encoding unit 260 is configured to encode attributes of points in the point cloud frame to generate an attribute bitstream 205. As described below, the attribute encoding unit 260 may also use the positions as well as the encoded geometry (e.g., reconstruction) from the geometry encoding unit 250 to encode attributes.

[0036]

[0063] In the example of Figure 3, the G-PCC decoder 300 may include a geometry decoding unit 350 and an attribute decoding unit 360. Generally, the geometry encoding unit 350 is configured to decode the geometry bitstream 203 to recover positions of points in a point cloud frame. The attribute decoding unit 360 is configured to decode the attribute bitstream 205 to recover attributes of the points in the point cloud frame. As described below, the attribute decoding unit 360 may also use positions from the decoded geometry (e.g., reconstruction) from the geometry decoding unit 350 to encode attributes.

[0037]

[0064] In both the G-PCC encoder 200 and the G-PCC decoder 300, the point cloud position is coded first. Attribute coding depends on the decoded geometry. In Figures 5-8 of this disclosure, coding units with vertical shading are options typically used for Category 1 data. Coding units with diagonal shading are options typically used for Category 3 data. All other modules are common between Category 1 and Category 3.

[0038]

[0065] For Category 3 data, the compressed geometry is typically represented as an octree from the root to the leaf level of individual voxels. For Category 1 data, the compressed geometry is typically represented by a pruned octree (i.e., an octree from the root to the leaf level of blocks larger than a voxel) plus a model that approximates the surface within each leaf of the pruned octree. In this way, both Category 1 and Category 3 data share the octree coding mechanism, while Category 1 data may additionally approximate the voxels within each leaf using a surface model. The surface model used is a triangulation with 1 to 10 triangles per block, resulting in a triangle soup. Therefore, Category 1 geometry codecs are known as Trisoup geometry codecs, and Category 3 geometry codecs are known as Octree geometry codecs.

[0039]

[0066] At each node in the octree, the occupancy (when not guessed) is signaled for one or more of its child nodes (up to a maximum of eight nodes). Multiple neighboring nodes are specified, including (a) nodes that share a face with the current octree node, and (b) nodes that share a face, edge, or vertex with the current octree node. Within each neighboring node, the occupancy of the node and / or its child nodes can be used to predict the occupancy of the current node or its children. For sparsely populated points at a particular node in the octree, the codec also supports a direct coding mode, in which the 3D position of the point is directly coded. A flag can be signaled to indicate that direct mode is signaled. At the lowest level, the number of points associated with an octree node / leaf node can also be coded.

[0040]

[0067] When geometry is coded, attributes corresponding to the geometry points are coded. When there are multiple attribute points corresponding to one reconstructed / decoded geometry point, attribute values ​​representing the reconstructed points can be derived.

[0041]

[0068] G-PCC has three attribute coding methods: Region Adaptive Hierarchical Transform (RAHT) coding, interpolation-based hierarchical nearest neighbor prediction (prediction transform), and interpolation-based hierarchical nearest neighbor prediction with an update / lifting step (lifting transform). RAHT and lifting are typically used for Category 1 data, and prediction is typically used for Category 3 data. However, either method can be used for any data, and as with geometry codecs in G-PCC, the attribute coding method used to code point clouds is specified in the bitstream.

[0042]

[0069] Attribute coding can be done in levels of detail (LoD), with each level of detail being used to obtain a finer representation of the point cloud attributes, which can be specified based on a distance metric from neighboring nodes or based on a sampling distance.

[0043]

[0070] In the G-PCC encoder 200, the residual obtained as the output of the attribute coding method is quantized. The residual may be obtained by subtracting the attribute value from a prediction derived based on neighboring points of the current point and based on the attribute values ​​of previously coded points. The quantized residual may be coded using context-adaptive arithmetic coding.

[0044]

[0071] The G-PCC encoder 200 and the G-PCC decoder 300 may be configured to code point cloud data using predictive geometry coding as an alternative to octree geometry coding. In predictive tree coding, the nodes of the point cloud are arranged in a tree structure (which defines a prediction structure), and various prediction schemes are used to predict the coordinates of each node in the tree with respect to the tree's predictors. A node that is a root vertex has no predictors. Other nodes may have one, two, three, or more children. Other nodes may be leaf nodes with no children. In one example, each node of the prediction has only one parent node.

[0045]

[0072] The G-PCC encoder 200 can employ any algorithm to generate the prediction tree; the algorithm used can be determined based on the application / use case, and several strategies can be used. For each node, residual coordinate values ​​are coded in the bitstream starting from the root node in a depth-first manner. Predictive geometry coding can be particularly useful for category 3 (LIDAR-acquired) point cloud data, e.g., for low-latency applications.

[0046]

[0073] Figure 4 is a conceptual diagram illustrating an exemplary octree partitioning for geometry coding. For each node in the octree 400, the G-PCC encoder 200 can signal to the G-PCC decoder 300 the occupancy (if occupancy is not inferred by the G-PCC decoder 300) for one or more of the node's child nodes (e.g., up to eight nodes). Multiple neighborhoods are specified, including (a) nodes that share a face with the current octree node, and (b) nodes that share a face, edge, or vertex with the current octree node. Within each neighborhood, the occupancy of the node and / or its children can be used to predict the occupancy of the current node or its children. For sparsely populated points at a particular node in the octree, the codec also supports a direct coding mode in which the 3D position of the point is directly coded. The G-PCC encoder 200 can signal a flag to indicate that direct mode is signaled. At the lowest level, the number of points associated with an octree node / leaf node can also be coded.

[0047]

[0074] When geometry is coded, attributes corresponding to the geometry points are coded. When there are multiple attribute points corresponding to one reconstructed / decoded geometry point, attribute values ​​representing the reconstructed points can be derived.

[0048]

[0075] G-PCC has three attribute coding processes: region-adaptive hierarchical transform (RAHT) coding, interpolation-based hierarchical nearest neighbor prediction (prediction transform), and interpolation-based hierarchical nearest neighbor prediction with an update / lifting step (lifting transform). RAHT and lifting are typically used for Category 1 data, and prediction is typically used for Category 3 data. However, either process may be used for any data, and similar to geometry codecs in G-PCC, the attribute coding process used to code point clouds is specified in the bitstream.

[0049]

[0076] Attribute coding may be done at a certain LOD, with each level of detail used to obtain a more precise representation of the point cloud attributes, which may be specified based on a distance metric from neighboring nodes or based on a sampling distance.

[0050]

[0077] In the G-PCC encoder 200, the residual obtained as the output of the coding process for an attribute is quantized. The residual may be obtained by subtracting the attribute value from a prediction derived based on neighboring points of the current point and based on the attribute values ​​of previously coded points. The quantized residual may be coded using context-adaptive arithmetic coding.

[0051]

[0078] Figure 5 is a block diagram illustrating in more detail an example of the geometry encoding unit 250 of Figure 2. The geometry encoding unit 250 may include a coordinate transformation unit 202, a voxelization unit 206, a prediction tree construction unit 207, an octree analysis unit 210, a surface approximation analysis unit 212, an arithmetic coding unit 214, and a geometry reconstruction unit 216.

[0052]

[0079] As shown in the example of Figure 5, geometry encoding unit 250 may obtain a set of locations of points in a point cloud. In one example, geometry encoding unit 250 may obtain a set of locations and a set of attributes of points in the point cloud from data source 104 (Figure 1). The locations may include coordinates of the points in the point cloud. Geometry encoding unit 250 may generate a geometry bitstream 203 that includes coded representations of the locations of the points in the point cloud.

[0053]

[0080] The coordinate transformation unit 202 may apply a transformation to the coordinates of the points to convert the coordinates from the initial domain to the transformed domain. In this disclosure, the transformed coordinates may be referred to as transformed coordinates. The voxelization unit 206 may voxelize the transformed coordinates. Voxelizing the transformed coordinates may include quantization and removing some points of the point cloud. In other words, multiple points of the point cloud may be contained within a single "voxel" and then treated as one point at some points.

[0054]

[0081] The prediction tree construction unit 207 may be configured to generate a prediction tree based on the voxelized transform coordinates. The prediction tree construction unit 207 may be configured to perform any of the predictive tree coding techniques described above in either intra-prediction mode or inter-prediction mode. To perform predictive tree coding using inter-prediction, the prediction tree construction unit 207 may access points from a previously coded frame from the geometry reconstruction unit 216. The dashed lines from the geometry reconstruction unit 216 indicate the data path when inter-prediction is performed. The arithmetic coding unit 214 may entropy code syntax elements representing the coded prediction tree.

[0055]

[0082] Instead of performing predictive tree-based coding, geometry encoding unit 250 may perform octree-based coding. Octree analysis unit 210 may generate an octree based on the voxelized transform coordinates. Surface approximation analysis unit 212 may analyze the points to determine a surface representation of the set of points. Arithmetic coding unit 214 may entropy code syntax elements representing surface information determined by the octree and / or surface approximation analysis unit 212. Geometry encoding unit 250 may output these syntax elements in geometry bitstream 203. Geometry bitstream 203 may also include other syntax elements, including syntax elements that are not arithmetically coded.

[0056]

[0083] Octree-based coding can be performed as either an intra-prediction technique or an inter-prediction technique. To perform octree coding using inter-prediction, the octree analysis unit 210 and the surface approximation analysis unit 212 may access points from a previously coded frame from the geometry reconstruction unit 216. The dashed line from the geometry reconstruction unit 216 indicates the data path when inter-prediction is performed.

[0057]

[0084] The geometry reconstruction unit 216 may reconstruct transformation coordinates of points in the point cloud based on the octree, prediction tree, data indicative of the surface determined by the surface approximation analysis unit 212, and / or other information. The number of transformation coordinates reconstructed by the geometry reconstruction unit 216 may differ from the original number of points in the point cloud due to voxelization and surface approximation. In this disclosure, the resulting points may be referred to as reconstructed points.

[0058]

[0085] 6 is a block diagram illustrating an example of the attribute encoding unit 260 of FIG. 2 in more detail. The attribute encoding unit 250 may include a color transformation unit 204, an attribute transfer unit 208, a RAHT unit 218, a predictive encoding unit 219, a LoD generation unit 220, a lifting unit 222, a coefficient quantization unit 224, an arithmetic encoding unit 226, and an attribute reconstruction unit 228. The attribute encoding unit 260 may encode attributes of points of a point cloud to generate an attribute bitstream 205 including an encoded representation of the set of attributes. The attributes may include information about points in a point cloud, such as a color associated with the point in the point cloud.

[0059]

[0086] The color transformation unit 204 may apply a transformation to convert the color information of the attributes into a different domain. For example, the color transformation unit 204 may convert the color information from an RGB color space to a YCbCr color space. The attribute transfer unit 208 may transfer attributes of the original points of the point cloud to the reconstructed points of the point cloud. The attribute transfer unit 208 may use the original positions of the points as well as the generated positions from the attribute encoding unit 250 (e.g., from the geometry reconstruction unit 216) to perform the transfer.

[0060]

[0087] The RAHT unit 218 may apply RAHT coding to the attributes of the reconstructed points. In some examples, under RAHT, attributes of a 2x2x2 block of point locations are taken and transformed along one direction to obtain four low-frequency nodes (L) and four high-frequency nodes (H). The four low-frequency nodes (L) are then transformed in a second direction to obtain two low-frequency nodes (LL) and two high-frequency nodes (LH). The two low-frequency nodes (LL) are transformed in a third direction to obtain one low-frequency node (LLL) and one high-frequency node (LLH). The low-frequency node LLL corresponds to the DC coefficient, and the high-frequency nodes H, LH, and LLH correspond to the AC coefficients. The transformation in each direction may be a 1-D transformation using two coefficient weights. The low-frequency coefficients may be considered as the coefficients of a 2x2x2 block for the next higher level of the RAHT transformation, and the AC coefficients are coded without modification, and such transformation continues up to the top-level root node. This transformation continues up to the top-level root node. The tree traversal for encoding is a top-to-bottom traversal used to calculate the weights used for the coefficients, and the transformation order is bottom-to-top. The coefficients can then be quantized and coded.

[0061]

[0088] Alternatively or additionally, the LoD generation unit 220 and the lifting unit 222 may apply LoD processing and lifting, respectively, to the attributes of the reconstructed points. LoD generation is used to divide the attributes into different refinement levels. Each refinement level provides refinement to the attributes of the point cloud. The first refinement level provides a coarse approximation and includes a small number of points, while subsequent refinement levels typically include more points, and so on. The refinement levels may be configured using a distance-based metric or may use one or more other classification criteria (e.g., subsampling from a particular rank). Thus, a refinement level may include all reconstructed points. Each level of detail is generated by taking the union of all points up to a particular refinement level; for example, LoD1 is obtained based on refinement level RL1, LoD2 is obtained based on RL1 and RL2, and LoD1 is obtained by the union of RL1, RL2, ..., RLN. In some cases, LoD generation is followed by a prediction scheme (such as predictive transform) in which attributes associated with each point in the LoD are predicted from a weighted average of previous points, and the residual is quantized and entropy coded. Lifting schemes build on predictive transform mechanisms, where update operators are used to update the coefficients and adaptive quantization of the coefficients is performed.

[0062]

[0089] The predictive coding unit 219 may be configured to determine an attribute value of a current point based on attribute values ​​of already-encoded points. For example, the predictive coding unit 219 may be configured to determine a list of prediction candidates and select a candidate from the list as a prediction of the attribute value of the current point. The predictive coding unit 219 may then determine a residual value representing the difference between the predicted attribute value of the current point and the actual attribute value of the current point. This disclosure describes techniques for generating a candidate list for predictive geometry coding. Predictive geometry coding is described in more detail below. The predictive coding unit 219 includes a neighbor replacement unit (NRU) 221, which may be configured to perform techniques of this disclosure, including neighbor replacement techniques, described in more detail below.

[0063]

[0090] The RAHT unit 218, the predictive coding unit 219, and the lifting unit 222 may generate coefficients based on the attributes. The coefficient quantization unit 224 may quantize the coefficients generated by the RAHT unit 218, the predictive geometry coding unit 219, or the lifting unit 222. In the case of the predictive coding unit 219, the coefficient quantization unit 224 may quantize the determined residual values, or quantization may be skipped. The arithmetic coding unit 226 may apply arithmetic coding to syntax elements representing the quantized coefficients. The G-PCC encoder 200 may output these syntax elements in the attribute bitstream 205. The attribute bitstream 205 may also include other syntax elements, including non-arithmetically coded syntax elements.

[0064]

[0091] The attribute coding unit 260, like the geometry coding unit 250, may code the attributes using either intra-prediction or inter-prediction techniques. The above description of the attribute coding unit 260 generally describes intra-prediction techniques. In other examples, the RAHT unit 215, the LoD generation unit 220, and / or the lifting unit 222 may also use attributes from previously coded frames to further code attributes of the current frame. In this regard, the attribute reconstruction unit 228 may be configured to reconstruct coded attributes and store them for potential future use in inter-prediction coding.

[0065]

[0092] Figure 7 is a block diagram illustrating the example geometry decoding unit 350 of Figure 3 in more detail. The geometry decoding unit 350 may be configured to perform a process that is the inverse of the process performed by the geometry encoding unit 250 of Figure 5. The geometry decoding unit 350 receives the geometry bitstream 203 and generates positions of points in a point cloud frame. The geometry decoding unit 350 may include a geometry arithmetic decoding unit 302, an octree synthesis unit 306, a prediction tree synthesis unit 307, a surface approximation synthesis unit 310, a geometry reconstruction unit 312, and an inverse coordinate transformation unit 320.

[0066]

[0093] The geometry decoding unit 350 may receive the geometry bitstream 203. The geometry arithmetic decoding unit 302 may apply arithmetic decoding (e.g., Context-Adaptive Binary Arithmetic Coding (CABAC) or other types of arithmetic decoding) to syntax elements in the geometry bitstream 203.

[0067]

[0094] The octree synthesis unit 306 may synthesize an octree based on syntax elements parsed from the geometry bitstream 203. Starting with the root node of the octree, the occupancy of each of the eight child nodes at each octree level is signaled in the bitstream. When the signaling indicates that a child node at a particular octree level is occupied, the occupancy of this child node's children is signaled. Signaling of the nodes at each octree level occurs before proceeding to subsequent octree levels.

[0068]

[0095] At the final level of the octree, each node corresponds to a voxel location, and when a leaf node is occupied, one or more points may be designated as occupied at the voxel location. In some instances, due to quantization, some branches of the octree may terminate before the final level. In such cases, the leaf node is considered an occupied node with no child nodes. In instances where surface approximations are used in the geometry bitstream 203, the surface approximation synthesis unit 310 may determine a surface model based on syntax elements parsed from the geometry bitstream 203 and based on the octree.

[0069]

[0096] Octree-based coding can be performed as either an intra-prediction technique or an inter-prediction technique. To perform octree coding using inter-prediction, the octree synthesis unit 306 and the surface approximation synthesis unit 310 may access points from previously decoded frames from the geometry reconstruction unit 312. The dashed line from the geometry reconstruction unit 312 indicates the data path when inter-prediction is performed.

[0070]

[0097] The prediction tree synthesis unit 307 may synthesize a prediction tree based on syntax elements parsed from the geometry bitstream 203. The prediction tree synthesis unit 307 may be configured to synthesize a prediction tree using any of the techniques described above, including using intra-prediction techniques or both intra-prediction techniques. To perform predictive tree coding using inter-prediction, the prediction tree synthesis unit 307 may access points from a previously decoded frame from the geometry reconstruction unit 312. The dashed lines from the geometry reconstruction unit 312 indicate the data path when inter-prediction is performed.

[0071]

[0098] The geometry reconstruction unit 312 may perform reconstruction to determine the coordinates of points within the point cloud. For each position in a leaf node of the octree, the geometry reconstruction unit 312 may reconstruct the node position by using the binary representation of the leaf node in the octree. At each respective leaf node, the number of points in the respective leaf node is signaled, which indicates the number of overlapping points at the same voxel position. When geometric quantization is used, the point positions are scaled to determine the reconstructed point position value.

[0072]

[0099] The inverse coordinate transformation unit 320 may apply an inverse transform to the reconstructed coordinates (positions) of points in the point cloud to translate them from the transformed domain back to the original domain. The positions of points in the point cloud may be in the floating-point domain, whereas point positions in the G-PCC codec are coded in the integer domain. An inverse transform may be used to transform the positions back to the original domain.

[0073]

[0100] Figure 8 is a block diagram illustrating the example attribute decoding unit 360 of Figure 3 in further detail. The attribute decoding unit 360 may be configured to perform a process that is the inverse of the process performed by the attribute encoding unit 260 of Figure 6. The attribute decoding unit 360 receives the attribute bitstream 205 and generates attributes for the points of the point cloud frame. The attribute decoding unit 360 may include an attribute arithmetic decoding unit 304, an inverse quantization unit 308, an inverse RAHT unit 314, a prediction decoding unit 315, an LoD generation unit 316, an inverse lifting unit 318, an inverse transform color unit 322, and an attribute reconstruction unit 328.

[0074]

[0101] The attribute arithmetic decoding unit 304 may apply arithmetic decoding to syntax elements in the attribute bitstream 205. The inverse quantization unit 308 may inverse quantize attribute values. The attribute values ​​may be based on syntax elements obtained from the attribute bitstream 205 (e.g., including syntax elements decoded by the attribute arithmetic decoding unit 304).

[0075]

[0102] Depending on how the attribute values ​​are coded, the inverse RAHT unit 314 may perform RAHT coding to determine color values ​​for points of the point cloud based on the dequantized attribute values. RAHT decoding is performed from the top to the bottom of the tree. At each level, component values ​​are derived using the low-frequency and high-frequency coefficients derived from the inverse quantization process. At leaf nodes, the derived values ​​correspond to the attribute values ​​for the coefficients. The weight derivation process for points is similar to the process used in the G-PCC encoder 200. Alternatively, the LoD generation unit 316 and the inverse lifting unit 318 may use a level-of-detail-based technique to determine color values ​​for points of the point cloud. The LoD generation unit 316 decodes each LoD to progressively represent the attributes of the points with greater detail. Using predictive transformation, the LoD generation unit 316 derives a predicted value for a point from a weighted sum of points in the previous LoD or previously reconstructed within the same LoD. The LoD generation unit 316 may add the predicted value to the residual (obtained after inverse quantization) to obtain a reconstructed value of the attribute. When a lifting scheme is used, the LoD generation unit 316 may also include an update operator to update the coefficients used to derive the attribute value. In this case, the LoD generation unit 316 may also apply inverse adaptive quantization.

[0076]

[0103] The predictive decoding unit 315 may be configured to determine an attribute value for a current point based on attribute values ​​of already-encoded points. For example, the predictive decoding unit 315 may be configured to determine a list of prediction candidates (i.e., the same list determined by the predictive geometry encoding unit 219) and select a candidate from the list as a prediction of the attribute value of the current point. The predictive decoding unit 315 may then determine a residual value representing the difference between the predicted attribute value for the current point and the actual attribute value for the current point and add the residual value to the predicted value to determine a final attribute value for the current point. This disclosure describes techniques for generating a candidate list for predictive geometry coding. Predictive geometry coding is described in more detail below. The predictive decoding unit 315 includes an NRU 317, which may be configured to perform techniques of this disclosure, including neighbor replacement techniques, described in more detail below.

[0077]

[0104] 8, the inverse color transform unit 322 may apply an inverse color transform to the color values. The inverse color transform may be the inverse of the color transform applied by the color transform unit 204 of the encoder 200. For example, the color transform unit 204 may convert the color information from the RGB color space to the YCbCr color space. Accordingly, the inverse color transform unit 322 may convert the color information from the YcbCr color space to the RGB color space.

[0078]

[0105] The attribute reconstruction unit 328 may be configured to store attributes from previously decoded frames. Attribute coding may be performed as either an intra-prediction technique or an inter-prediction technique. To perform attribute decoding using inter-prediction, the inverse RAHT unit 314, the prediction decoding unit 315, and / or the LoD generation unit 316 may access attributes from previously decoded frames from the attribute reconstruction unit 328.

[0079]

[0106] Various units are illustrated in FIGS. 5-8 to aid in understanding the operations performed by the G-PCC encoder 200 and the G-PCC decoder 300. The units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. A fixed-function circuit refers to a circuit that provides a specific function and has predefined operations that can be performed. A programmable circuit refers to a circuit that can be programmed to perform various tasks and provides flexibility in the operations that can be performed. For example, a programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by the software or firmware instructions. While a fixed-function circuit may execute software instructions (e.g., receive parameters or output parameters), the types of operations that the fixed-function circuit performs are generally invariant. In some examples, one or more of the units may be different circuit blocks (fixed function or programmable), and in some examples, one or more of the units may be an integrated circuit.

[0080]

[0107] Predictive geometry coding (see, e.g., G-PCC Codec Description) has been introduced as an alternative to octree geometry coding, in which nodes are arranged in a tree structure (which defines the prediction structure), and various prediction strategies are used to predict the coordinates of each node in the tree with respect to the node's predictor. Figure 9 shows an example of a prediction tree, a directed graph, with arrows pointing in the prediction direction. Nodes with horizontal crosshatched patterns are root vertices and have no predictors; nodes with crosshatched patterns have two children; nodes with diagonal crosshatched patterns have three children; nodes with no crosshatched patterns have one child; and nodes with vertical crosshatched patterns are leaf nodes and have no children. Except for the root node, all nodes have only one parent node.

[0081]

[0108] Figure 9 is a conceptual diagram illustrating an example of a prediction tree. Node 900 is the root vertex and has no predictors. Nodes 902 and 904 have two children. Node 906 has three children. Nodes 908, 910, 912, 914, and 916 are leaf nodes and have no children. The remaining nodes each have one child. Except for the root node 900, all nodes have only one parent node.

[0082]

[0109] For each node, four prediction strategies are specified based on the node's parent (p0), grandparent (p1), and great-grandparent (p2): No Prediction / Zero Prediction (0) Delta forecast (p0) Linear prediction (2 * p0-p1) Parallelogram prediction (2 * p0+p1-p2)

[0083]

[0110] The G-PCC encoder 200 can employ any algorithm to generate the prediction tree, and the algorithm used can be determined based on the application / use case, and several strategies can be used, some of which are described in the G-PCC Codec Description.

[0084]

[0111] For each node, the residual coordinate values ​​are coded in the bitstream starting from the root node in a depth-first manner. For example, the G-PCC encoder 200 can code the residual coordinate values ​​in the bitstream.

[0085]

[0112] Predictive geometry coding is primarily useful for category 3 (LIDAR-acquired) point cloud data, e.g., for low-latency applications.

[0086]

[0113] 10A and 10B are conceptual diagrams illustrating an example of a rotational LIDAR acquisition model. The angular mode for predictive geometry coding is now described. In predictive geometry coding, angular mode can be used, whereby LIDAR sensor characteristics can be utilized in coding the prediction tree more efficiently. Position coordinates are transformed into the (r, φ, i) (radius, azimuth, and laser index) domain 600, and prediction is performed in this domain 600 (e.g., residuals are coded in the r, φ, i domain). Because coding in r, φ, i is not lossless due to rounding errors, a second set of residuals, corresponding to Cartesian coordinates, can be coded. A description of the encoding and decoding strategy used for angular mode for predictive geometry coding is reproduced below from the G-PCC Codec Description:

[0087]

[0114] The technique focuses on the point cloud acquired using a rotational LIDAR model. In this case, the LIDAR 602 has N lasers (e.g., N=16, 32, 64) that rotate around the Z axis according to an azimuth angle φ. Each laser is focused at a different elevation angle θ(i). i=1...N and height

[0088]

number

[0089] Assume that laser i strikes a point M with Cartesian integer coordinates (x, y, z) defined according to the coordinate system shown in Figures 10A-10B.

[0090]

[0115] This technique models the position of M using three parameters (r, φ, i) calculated as follows:

[0091]

number

[0092]

[0116] More precisely, this technique uses a quantized version of (r,φ,i) (

[0093]

number

[0094] ), three integers

[0095]

number

[0096] and i are calculated as follows:

[0097]

number

[0098] where (q r ,o r ) and (q φ ,o φ ) are respectively

[0099]

number

[0100] is the quantization parameter that controls the precision of |t|. sign(t) is a function that returns 1 if t is positive, and (-1) otherwise. |t| is the absolute value of t.

[0101]

[0117] To avoid reconstruction inconsistencies due to the use of floating-point arithmetic,

[0102]

number

[0103] and tan(θ(i)) i=1...N The value of can be pre-calculated and quantized as follows:

[0104]

number

[0105] where:

[0106]

number

[0107] and (q θ ,o θ ) are respectively

[0108]

number

[0109] is the quantization parameter that controls the precision of

[0110]

[0118] The reconstructed Cartesian coordinates are obtained as follows:

[0111]

number

[0112] where app_cos(.) and app_sin(.) are approximations to cos(.) and sin(.). The calculations can be performed using fixed-point representations, lookup tables, and / or linear interpolation.

[0113]

[0119]

number

[0114] Note that x, y, z may differ from (x, y, z) due to various reasons such as quantization, approximation methods, model inaccuracies, model parameter inaccuracies, etc.

[0115]

[0120] (r x ,r y ,r z ) be the reconstruction residual, defined as:

[0116]

number

[0117]

[0121] For this technique, the G-PCC encoder 200 may proceed as follows: 1) Model parameters

[0118]

number

[0119] and the quantization parameter q r ,

[0120]

number

[0121] q θ and q φ Encode the following. 2) The geometry prediction method described in the text of ISO / IEC FDIS 23090-9 Geometry-based Point Cloud Compression, ISO / IEC JTC 1 / SC29 / WG 7 m55637, Teleconference, October 2020, 3) Expression

[0122]

number

[0123] applies to. New predictors can be introduced that exploit the properties of LIDAR. For example, the rotation speed of the LIDAR scanner around the z-axis is typically constant.

[0124]

number

[0125] can be predicted as follows:

[0126]

number

[0127] in this case, (δ φ (k)) k=1...K is the set of potential rates that the G-PCC encoder 200 can use. The index k may be explicitly written into the bitstream or inferred from the context based on a deterministic strategy applied by both the G-PCC encoder 200 and the G-PCC decoder 300, and n(j) is the number of skipped points, which may be explicitly written into the bitstream or inferred from the context based on a deterministic strategy applied by both the G-PCC encoder 200 and the G-PCC decoder 300. n(j) is also referred to herein as the "phi multiplier." Note that the phi multiplier is currently only used with the delta predictor. 4) Reconstruction residual (r x ,r y ,r z ) is encoded at each node.

[0128] The G-PCC decoder 300 may proceed as follows: 1) Model parameters

[0129]

number

[0130] and the quantization parameter q r ,

[0131]

number

[0132] q θ and q φ Decrypt the 2) The geometry prediction scheme described in the text of ISO / IEC FDIS 23090-9 Geometry-based Point Cloud Compression, ISO / IEC JTC 1 / SC29 / WG 7 m55637, Teleconference, Oct. 2020, is used to estimate the nodes associated with the cloud.

[0133]

number

[0134] Decode the parameters. 3) The reconstructed coordinates as described above

[0135]

number

[0136] Calculate. 4) Residual (r x ,r y ,r z ) is decrypted. As explained in more detail below, lossy compression involves the reconstructed residual (r x ,r y ,r z ) can be supported by quantizing 5) Calculate the original coordinates (x,y,z) as follows:

[0137]

number

[0138] Lossy compression is the process of reconstructing residuals (r x ,r y ,r z ) or by deleting points.

[0139] The quantized reconstruction residual can be calculated as follows:

[0140]

number

[0141] In the formula (q x ,o x ), (q y ,o y ) and (q z ,o z ) are respectively

[0142]

number

[0143] is a quantization parameter that controls the precision of the residual. For example, the G-PCC encoder 200 or the G-PCC decoder 300 can calculate a quantized residual.

[0144]

[0125] The G-PCC encoder 200 or the G-PCC decoder 300 can use trellis quantization to further improve the RD (rate-distortion) performance results.

[0145]

[0126] The quantization parameters may be varied at the sequence / frame / slice / block level to achieve region-adaptive quality and / or for rate control purposes.

[0146]

[0127] Inter prediction in G-PCC prediction geometry coding is G-PCC 2 nd This is described in the Edition codec description, ISO / IEC JTC 1 / SC29 / WG 7 MDS21558, Teleconference, April 2022 (hereinafter referred to as "MDS21558").

[0147]

[0128] The G-PCC encoder 200 and the G-PCC decoder 300 may be configured to perform inter prediction for predictive geometry coding as described in MDS21558, A.K. Ramasubramonian, L. Pham Van, G. Van der Auwera, M. Karczewicz, [G-PCC] EE13.2 report on inter prediction, Test 2, ISO / IEC JTC1 / SC29 / WG7 m56839, April 2021 (hereinafter "m56839"), and A.K. Ramasubramonian, G. Van der Auwera, L. Pham Van, M. Karczewicz, [G-PCC][EE13.2-related] Additional results for inter prediction for predictive geometry, ISO / IEC JTC1 / SC29 / WG7 m56841, April 2021 (hereinafter "m56841").

[0148] Predictive geometry coding uses a prediction tree structure to predict the position of a point. When angle coding is enabled, x, y, and z coordinates are converted to radius, azimuth, and laser ID, and the residual is signaled in these three coordinates as well as the x, y, and z dimensions. The intra-prediction used for radius, azimuth, and laser ID can be one of four modes, and the predictors are nodes classified as parent, grandparent, and great-grandparent in the prediction tree with respect to the current node. Predictive geometry coding as currently designed in G-PCC Ed.1 is an intra-coding tool that uses only points within the same frame for prediction. Furthermore, using points from previously decoded frames may provide better prediction and therefore better compression performance.

[0149]

[0130] Inter-prediction was originally proposed in MDS21558 and m56839 to predict the radius of a point from a reference frame. For each point in the prediction tree, it is determined whether the point is inter-predicted or intra-predicted (indicated by a flag). If intra-predicted, the intra-prediction mode of predictive geometry coding is used. When inter-prediction is used, the azimuth angle and laser ID are still predicted using intra-prediction, while the radius is predicted from the point in the reference frame that has the same laser ID as the current point and the azimuth angle closest to the current azimuth angle. A further improvement of this process in m56841 also allows inter-prediction of the azimuth angle and laser ID in addition to the radius prediction. When inter-coding is applied, the radius, azimuth angle, and laser ID of the current point are predicted based on points in the reference frame that are near the azimuth angle position of the previously decoded point. Furthermore, separate sets of contexts are used for inter-prediction and intra-prediction.

[0150]

[0131] The process in m56841 is shown in Figure 11. Figure 11 is a conceptual diagram showing an example of inter-predicting a current point (curPoint) 1100 of a current frame from a point (interPredPt) 1102 of a reference frame. Extending inter-prediction to azimuth angle, radius, and laserID may include the following steps. For a given point, select the previous decoded point (prevDecP0) 1104. Select a location point in the reference frame (refFrameP0) 1106 that has the same scaling azimuth and laserID as prevDecP0 1104. Find the first point (interPredPt) 1102 in the reference frame that has an azimuth angle greater than the azimuth angle of refFrameP0 1106. Point interPredPt 1102 is sometimes called the "next" inter-predictor.

[0151] 12 is a flowchart showing an example decoding flow associated with an "inter flag" signaled per point. The inter flag signaled for a point indicates whether inter prediction is applied to the point. The flowcharts of this disclosure are given as examples. Other examples may include more, fewer, or different steps, or the steps may be performed in a different order.

[0152] 12, the G-PCC decoder 300 may determine whether the inter flag of the next point to be decoded (i.e., the current point of the current frame of the point cloud data) indicates that the current point is inter predicted (800). If the inter flag of the current point does not indicate that the current point is inter predicted (the "No" branch of 1200), the G-PCC decoder 300 may identify intra prediction candidates (812). For example, the G-PCC decoder 300 may determine an intra prediction strategy (e.g., no prediction, delta prediction, linear prediction, parallelogram prediction, etc.) to determine a predictor for the current point. A syntax element (pred_mode) signaled in the geometry bitstream 203 may indicate the intra prediction strategy to be used to determine a predictor for the current point.

[0153] On the other hand, if the inter flag for the current point indicates that the current point is inter-predicted ("Yes" branch of 1200), the G-PCC decoder 300 may identify a previous point in decoding order (e.g., the previous point 1108) (1202). The previous point may have coordinates (r, phi, and laserID). The G-PCC decoder 300 may then derive a quantized phi coordinate (i.e., an azimuthal coordinate) of the previous point (804). The quantized phi coordinate may be denoted as Q(phi). The G-PCC decoder 300 may then check a reference frame (e.g., the reference frame 1106) for a point (i.e., an inter-predicted point (e.g., interPredPt 1104)) that has a quantized phi coordinate greater than the quantized phi coordinate of the previous point (1206). The G-PCC decoder 300 may use the inter-predicted point as a predictor for the current point (1208).

[0154]

[0135] Regardless of whether the G-PCC decoder 300 determines the predictor for the current point using intra prediction (e.g., as described with respect to step 1212) or inter prediction (e.g., as described with respect to steps 1202-1208), the G-PCC decoder 300 may add a delta-phi multiplier (1210).

[0155]

[0136] Figure 13 is a conceptual diagram showing an exemplary additional inter predictor point 1300 obtained from an initial point having a larger azimuth angle than the inter predictor point 1314. In the inter prediction process for predicted geometry described above with reference to Figure 11, the radius, azimuth angle, and laserID of the current point (current point 1100) are predicted based on a point (inter prediction point 1104) near a collocated azimuth angle position (reference position 1110) in a reference frame (reference frame 1106) when inter coding is applied. In the example of Figure 13, the G-PCC encoder 200 and the G-PCC decoder 300 can determine the additional inter predictor point 1300 using the following steps. a) For a given point (current point 1300 in current frame 1304), determine the previous point 1302 in the current frame 1304 ("prev dec point" in Figure 13). b) Determine a reference position 1306 in the reference frame 1308 that has the same scaled azimuth angle and laserID as the previous point 1302 determined in step a) ("Reference point with same scaled azimuth angle and laserID" in Figure 13). c) Determine a position in the reference frame 1308 as the first point having a larger azimuth angle (e.g., a scaled azimuth angle) than the reference position 1306 determined in step b), which is used as the inter prediction point (inter pred point 1310 in Figure 13).

[0156] The additional inter prediction point can be obtained by finding the first point ("additional inter prediction point 1312" in Figure 13) that has a larger azimuth angle (e.g., a scaled azimuth angle) than the inter prediction point 1310 determined in step c), as shown in Figure 13. If inter coding is applied, additional signaling may be used to indicate which of the predictors is selected. The additional inter predictor point may also be called the "NextNext" inter predictor.

[0157] In some examples, the G-PCC encoder 200 (e.g., the arithmetic coding unit 214 of the G-PCC encoder 200) and the G-PCC decoder 300 (e.g., the geometry arithmetic decoding unit 302 of the G-PCC decoder 300) can apply a context selection algorithm to code the inter prediction flag. The inter prediction flag values ​​of the five previously coded points can be used to select the context of the inter prediction flag in predictive geometry coding.

[0158]

[0139] The G-PCC encoder 200 and the G-PCC decoder 300 may be configured to perform global motion compensation. When global motion (GM) parameters are available, inter prediction may be applied using reference frames that are motion compensated using the GM parameters, as described in A.K. Ramasubramonian, G. Van der Auwera, L. Pham Van, M. Karczewicz, [G-PCC][New proposal] Results on inter prediction for predictive geometry coding, ISO / IEC JTC1 / SC29 / WG7 m59650, April 2022. The GM parameters may include rotation parameters and / or translation parameters.

[0159]

[0140] Figure 14 shows a flow diagram illustrating a motion compensation process when reference frames are stored in the spherical domain and motion compensation is performed in the Cartesian domain. In the example of Figure 14, the G-PCC encoder 200 and the G-PCC decoder 300 may be configured to convert 1404 a reference frame 1402 in the spherical domain from the spherical domain to the Cartesian domain to generate a reference frame 1406 in the Cartesian domain. The G-PCC encoder 200 and the G-PCC decoder 300 may be configured to perform 1408 motion compensation on the reference frame 1406 to generate a compensated reference frame 1410 in the Cartesian domain. The G-PCC encoder 200 and the G-PCC decoder 300 may be configured to convert 1412 the compensated reference frame 1410 from the Cartesian domain to the spherical domain to generate a compensated reference frame 1414 in the spherical domain.

[0160]

[0141] Typically, global motion compensation is applied in the Cartesian domain, but in some cases, global motion compensation can also be performed in the spherical domain. Depending on which domain the reference frame is stored in and in which domain the reference frame is compensated, one or more Cartesian-to-spherical domain transformations or spherical-to-Cartesian domain transformations can be applied. For example, when the reference frame is stored in the spherical domain and motion compensation is performed in the Cartesian domain, the motion compensation process can include one or more of the steps shown in Figure 14.

[0161] In such cases, the compensated reference frame can be used for inter prediction: Given a position (x, y, z) in the Cartesian coordinate system, the corresponding radius and azimuth angle are calculated (similar to the CartesianToSpherical transform function) as follows (floating-point implementation): int64_t r0=int64_t(std::round(hypot(xyz[0],xyz[1]))); auto phi0=std::round((atan2(xyz[1],xyz[0]) / (2.0 * M_PI)) * scalePhi); where scalePhi is modified for different rate points in the lossy setting. A maximum value of 24 bits is used for the azimuth angle when encoding geometry losslessly. A fixed-point implementation of the azimuth angle is available in the convertXyZToRpl function.

[0162] [Table 1]

[0163] [Table 2]

[0164]

[0143] The G-PCC encoder 200 and the G-PCC decoder 300 may be configured to perform attribute prediction, including default attribute prediction by three neighbors. Attribute prediction may be performed, for example, by the predictive encoding unit 219 and the predictive decoding unit 315. A simplified prediction structure when LOD is equal to 1 is described in "Improved G-PCC lossless and near-lossless coding," ISO / IEC JTC1 / SC29 / WG11 input document m44899, Macau, China, October 2018.

[0165]

[0144] (P i ) i=1...N Let be the set of locations associated with the points of the point cloud, and (M i ) i=1...N , (P i ) i=1...N Let I be the Morton code associated with I. First, the points are sorted in ascending order according to their associated Morton codes. Let I be the array of point indices ordered according to this process. The encoder / decoder compresses / decompresses the points, respectively, according to the order defined by I. At each iteration i, point P i is selected. P up to s (e.g. s=64) previous points i The distance between the i The k (e.g., k=3) nearest neighbors of are selected to be used for prediction in the same manner as in the current version of G-PCC.

[0166]

[0145] The G-PCC encoder 200 and the G-PCC decoder 300 may be configured to perform intra-LOD prediction for attribute predicting transform as described in Reference Structure Modification On Attribute Predicting Transform in TMC13, ISO / IEC JTC1 / SC29 WG11 input document m46107, Marrakech, MA, January 2019 and [G-PCC][New] The modification on intra LOD prediction for attribute predicting transform coding, ISO / IEC JTC1 / SC29 / WG11 input document m54633, Online, July 2020.

[0167]

[0146] Figure 15 shows a reference structure for intra LOD prediction. The EnableReferringSameLoD flag is introduced to control the reference structure for predicting transformations in order to maintain a trade-off between coding efficiency and parallel processing. When the EnableReferringSameLoD flag is set to 1, 3D points at the same LOD can be used for prediction. Arrow 1502 represents an example of intra LOD prediction, where both P6 and P1 are at LOD1, and P6 is predicted from P1. Arrow 1504 represents an example of non-intra LOD prediction, where P6 and P4 are at different LODs. Note that intra LOD prediction is always used when the number of LODs is equal to 1.

[0168] The G-PCC encoder 200 and the G-PCC decoder 300 may be configured to perform equal-distance neighbor search as described in [G-PCC][New proposal]Improved implementation of the Prediction and Lifting schemes., ISO / IEC JTC1 / SC29 / WG11 input document m51010, Geneva, CH, October 2019. The lifting and prediction schemes extensively use nearest neighbor search during the LOD generation and predictor construction stages. Document m51010 describes how equal-distance neighbors are handled.

[0169] FIG. 16 shows an example of processing neighbors in subsequent LODs that have the same distance from the current point. According to document m51010, neighbors in subsequent LODs that have the same distance from the current point are processed according to priority, as shown in FIG. 16. In the example of FIG. 16, a smaller value of the Morton base index corresponds to a smaller value of the Morton code, and a smaller value of the priority index corresponds to a higher priority. Essentially, points are included in the neighbor list based on the concept of distance. When two points have the same distance, a clear method is needed to determine which point to select. For the purpose of the example, assume that two points X1 and X2 are at the same distance. If X1 is searched first, X1 can be inserted into the list. If X2 is searched, X2 can be inserted only if there is space or if there is another entry in the list that has a distance greater than X2. That is, X2 does not replace X1, and therefore, for this point, X1 has priority over X2 because X1 came first. On the other hand, if X2 was first searched before X1, then X2 would be inserted instead of X1. Therefore, the priority index can be thought of as corresponding to the order of search.

[0170]

[0149] Figure 17 shows an example of processing neighbors in the same LOD that have the same distance from the current point. According to document m51010, neighbors in the same LOD that have the same distance from the current point are processed according to the priority explained in Figure 17. In the example of Figure 17, a smaller value of the Morton base index corresponds to a smaller value of the Morton code, and a smaller value of the priority index corresponds to a higher priority.

[0171]

[0150] Neighbors in subsequent LODs have higher priority than neighbors in the same LOD.

[0172]

[0151] The G-PCC encoder 200 and the G-PCC decoder 300 may be configured to perform an optimized nearest neighbor search for lifting / prediction as described in CE13.6 report on attribute LOD construction and neighbor search, ISO / IEC JTC1 / SC29 / WG11 input document m54668, Online, July 2020.

[0173] The approximate k-NN solution indeed provides a good approximation of k-NN, but fails to capture the actual nearest neighbor when significant jumps in terms of Morton order are observed between neighboring points. An example of this scenario is shown in Figure 18 by points P 1802 and Q 1804.

[0174] To improve the nearest neighbor search, the first optimization uses a lookup table to accelerate the k-NN search, more precisely, to determine whether a voxel's neighbors are occupied and use that occupancy to determine the k-NN for the current point. More precisely, N(i,1), N(i,2), ..., N(i,H) (see Figure 19 for an example of voxel neighbors) are calculated using R d Let C be the set of neighbors of P(i) in . For example, the 6 / 18 / 26 connectivity of P(i) connects {P(i)}, and C=[0,...,2 c-1]×[0,...,2 c -1]×[0,...,2 c -1] is the bounding cube of B (i.e., B ⊂ C).

[0175]

[0154] Figure 19 shows an example of voxel neighbors. In the example of Figure 19, voxel neighbor 1902 has 6-connectivity, i.e., there are 6 inter-boundary connections. Voxel neighbor 1904 has 18-connectivity, and voxel neighbor 1906 has 26-connectivity.

[0176] The G-PCC encoder 200 and the G-PCC decoder 300 may be configured to pre-compute a LUT that maps any point X in C to a range ρ(X) = (ρs(X), ρe(X)) that describes the index in B of points that have the same position (or Morton code) as point X. The set of points S(i) to be searched to determine the k-NN of point P(i)∈A is constructed as follows: S(i)←{} For each neighbor N(i,1), use the LUT to check if the neighbor belongs to B. If N(i,1)∈B, add all points in the range ρ(N(i,1)) to S(i). If S(i) has less than k elements If S(i) is empty, then S(i) ← {Q(j),Q(j+1),Q(j-1),...,Q(j+Δ),Q(j-Δ)} If S(i) has at least one element j * Let be the index of the nearest neighbor P(i) in S(i). S(j * )={Q(j * ),Q(j * +1),Q(j * -1),...,Q(j * +Δ),Q(j * -Δ)} to apply additional refinement searches

[0177]

[0156] The LUT and linear search may also be applied in reverse order (eg, first the linear search, then the LUT-based search).

[0178]

[0157] Figure 20 shows an example of space division. Allocating a LUT to hold C can be expensive in terms of memory. To reduce such requirements, a bounding cube C 2002 is allocated to the LUT of size 2 e The matrix P(i) may be divided into smaller subcubes {E(a,b,c)} (see subcubes 2004, 2006, 2008, and 2010). When points are traversed in Morton order, all points within one subcube can be traversed in order before switching to the next subcube. Therefore, only a LUT capable of storing a single subcube is required. The LUT is initialized each time P(i) enters a new subcube. Only points of B within that subcube are added to the LUT. Neighborhood relationships across subcube boundaries are ignored for points on the subcube boundaries.

[0179]

[0158] This approach can exploit the sparsity of the point cloud and LOD structure to reduce the size and improve the efficiency of the k-NN search. More precisely, this approach constrains the distances used in the lifting / prediction scheme as follows: Constrain the sequence of distances {d(1),d(2),...d(L)} as follows:

[0180]

number

[0181] d(l+1)=2×d(l) n0 is a parameter that is calculated by the encoder and explicitly signaled in the bitstream.

[0182] To determine n, the G-PCC encoder 200 and the G-PCC decoder 300 may first select a random subset of the points in the point cloud and, for each point, calculate the distance by which that point is separated from its nearest neighbor. Let δ be the p-th percentile (e.g., the 75th percentile) of these distances. n is chosen as the smallest integer that verifies that

[0183]

number

[0184]

[0160] The G-PCC encoder 200 and the G-PCC decoder 300 may be configured to perform a neighbor search process for attribute LOD prediction as described in W. Zhang, T. Tian, ​​L. Yang, F. Yang, M.-L. Champel, S. Gao, [G-PCC][EE13.49] Report on Attribute LOD Prediction, ISO / IEC JTC1 / SC29 / WG7 m58236, October 2021 and W. Zhang, L. Yang, F. Yang, T. Tian, ​​M. L. Champel, S. Gao, [G-PCC][New] Neighbor Search Improvements for Attribute LOD Prediction, ISO / IEC JTC1 / SC29 / WG07 m57324, July 2021.

[0185] In the G-PCC Ed.1 design, attribute prediction is performed between the point to be coded and its N nearest neighbors. Given the complex distribution of 3D points, selecting the nearest point as the predictor (i.e., using distance as the only criterion) is not always optimal.

[0186] FIG. 21 shows an example of neighborhood point distribution. In the proposed process, the point distribution is taken into account when selecting potential predictors. More specifically, the relative positions of potential predictors of the current point P 2102 are considered along with the distance to P 2102. In the example of FIG. 21, P2 2104 is closer to the current point P 2102 than P3 2106, but P3 2106 may be a better predictor of P 2102. The following steps detail the process. According to the first step, the G-PCC encoder 200 and the G-PCC decoder 300 generate two neighbor lists. List 1 contains the three nearest neighbors obtained using the existing process in G-PCC. List 2 contains N (e.g., 3) points to be omitted when updating List 1. The final predictor list is generated by updating List 1 using the points in List 2 as the eligibility check described in Steps 2 and 3.

[0187]

[0163] Figure 22 shows the generation of List 1 and List 2. For the purpose of explanation, consider eight neighbors N0...N7 that are sequentially (in that order) considered as candidates for the predictor list of the current point P. Chart 2202 shows the distance of each of the eight neighbors from the current point P (e.g., the distance from P to N0 is 3, the distance from P to N1 is 4, etc.). Lists 1 2204 and List 2 2206 show the list generation processes for the first list and the second list. In the initial state, moving from left to right, the first three neighbors N0, N1, and N2 have already been processed and are included in List 1 based on the increasing distance from P. That is, dist(P,N0)<=dist(P,N2)<=dist(P,N1), where dist(P,X) represents the distance between point P and point X. List 2 is initially empty, and k is set equal to 3, where (k - 3) indicates the index at which the next candidate should be added to List 2. The next neighbor to be processed is N3, and dist(P,N3)=dist(P,P2)=4, where Pn indicates the current nth candidate in List 1 when n≦3, or the (n - 3)th candidate in List 2 when n is greater than 3 and less than or equal to 6. When dist(P,N3)=dist(P,P2), if List 2 is not full (List 2 is considered to have a size of 3), N3 is added to List 2. Since List 2 is empty, N3 is added as Pk (k = 3), and k is incremented by 1. The next candidate is N4, and dist(P,N4)=1<dist(P,P0)=3. In this case, the current candidate at P2, which is N1, is added to List 2 as Pk (k = 4), and k is incremented by 1. The remaining candidates in List 1 are pushed to the right, and N4 is added as the new candidate in P0. The next candidate is N5, and dist(P,N5)=2<dist(P,P1)=3. In this case, the current candidate at P2, which is N0, is added to List 2 as Pk (k = 5), k is incremented by 1, and N5 is added as the new candidate in P1. Note that the counter k reaches 6, and since there are only three entries in List 2, k is reset to 3.The next candidate is N6, and dist(P, N6) = 2 < dist(P, P2) = 3. In this case, N0 which is the current candidate within P2 is added to list 2 as Pk (k = 3), k is incremented by only 1, and N6 is added as the new candidate within P2. The last candidate is N7, and dist(N7) = 3. Since dist(N7) > dist(P2), it is not added to any list. At the end of the neighbor list generation process, list 1 has neighbors N4, N5, and N6, and list 2 has neighbors N0, N2, and N1.

[0188]

[0164] According to the second step, the G-PCC encoder 200 and the G-PCC decoder 300 check the distribution of points within list 1. If P1 or P2 is already in the direction exactly opposite to P0 as shown in FIG. 23, the neighbor distribution is considered to be already sufficiently spread, and thus there is no need to check the points within list 2 any further. Otherwise, check the point Pn within list 2, and if dist(Pn, P) ≤ T1 and Pn is in the direction exactly opposite to P0, replace P2 with Pn. Otherwise, if dist(Pn, P) ≤ T2 and Pn is in the direction exactly opposite to P1, replace P2 with Pn. In the current implementation, T1 = w * dist(P2, P), T2 = w * dist(P1, P), and w << 5 = 54.

[0189]

[0165] According to the third step, the G-PCC encoder 200 and the G-PCC decoder 30 perform a point distribution check (e.g., a loose opposition check). If P2 has not been replaced after step 2 and P2 or P1 is in the same direction relative to P0 (P0, P1, and P2 are in the same region), the distribution check is relaxed to some extent. Specifically, the distribution check is based on a predefined loose opposition, as shown in Figure 23. Similar to step 2, point Pn in list 1 is checked. If dist(Pn, P) ≤ T1 and Pn is in the loose opposition relative to P0, Pn is used to replace P2. Otherwise, if dist(Pn, P) ≤ T2 and Pn is in the loose opposition relative to P1, Pn is used to replace P2.

[0190]

[0166] Figure 23 illustrates the concept of opposite directions. A current point P defines the intersection of the x-axis, y-axis, and z-axis, and the x-axis, y-axis, and z-axis form the xy-plane, xz-plane, and yz-plane. Two points on opposite sides of the xy-plane, the xz-plane, and the yz-plane are strictly opposite points. Points on opposite sides of two of the xy-plane, xz-plane, and yz-plane and on the same side of one of the xy-plane, xz-plane, and yz-plane are loosely opposite points. For example, referring to Figure 23, two points are strictly opposite if they are in quadrants 0 and 7, 1 and 6, 2 and 5, or 3 and 4.

[0191]

[0167] The G-PCC encoder 200 and the G-PCC decoder 300 may be configured to signal (eg, transmit and receive, respectively) a flag that enables the new neighbor discovery process.

[0192]

[0168] The modified neighbor discovery process is enabled by a flag signaled in APS. When the flag is 1, the modified neighbor discovery process is enabled, otherwise the default neighbor discovery process in G-PCC Ed.1 is used. The current signaling in G-PCC is as follows (the enabling flag is called predictionWithDistributionEnabled):

[0193] [Table 3]

[0194]

[0169] The code for neighbor replacement is shown below: numend1 and numend2 are indices corresponding to thresholds T1 and T2, where T1 and T2 are derived based on weights w=distCoefficient / 32=54 / 32.

[0195] [Table 4-1]

[0196] [Table 4-2]

[0197] [Table 4-3]

[0198] [Table 4-4]

[0199]

[0170] The problems and solutions of a first aspect (aspect 1) of the present disclosure will be described. Local partitioning units (LPUs) are specified as 3D spatial regions of a point cloud frame. Global motion compensation may or may not be applied to each LPU. In G-PCC, there are two types of LPUs: Type 0: An LPU defined using thresholds. In this type, two thresholds are signaled, specifying a lower threshold T1 and an upper threshold T2. Points with z coordinate values ​​between T1 and T2 are considered one LPU of type 0 (also called ground points), and the remaining points are considered another LPU of type 0 (also called object points). Type 1: Defined by a rectangular parallelepiped with specified width, height, and depth. Points within the rectangular parallelepiped with the specified width, height, and depth are considered to be part of the corresponding LPU.

[0200] For Type 1 LPUs, only width, height, and depth need to be signaled, and for Type 0 LPUs, only two thresholds need to be signaled. However, for Type 1 LPUs, the thresholds are currently also signaled. This also signals unnecessary bits in the bitstream. The current signaling could be as follows:

[0201] [Table 5-1]

[0202] [Table 5-2]

[0203]

[0172] As a proposed solution to the problem described above, the G-PCC encoder 200 and the G-PCC decoder 300 can be configured to determine whether to signal a threshold value based on the LPU type, and to refrain from signaling a threshold value based on a determination that a threshold value does not need to be signaled for a particular LPU type.

[0204]

[0173] In some cases, the decision may also depend on whether the coding mode is predictive geometry or octree geometry.

[0205]

[0174] An exemplary implementation of the solution of aspect 1 is as follows: <add> and< / add> The text between the delimiters indicates that the text is to be added. <del> and< / del> The text between indicates that the text will be deleted.

[0206]

[0175] We propose to apply conditions to the signaling of the thresholds, which depend on the LPU type and whether octree coding is applied or not.

[0207] [Table 6-1]

[0208] [Table 6-2]

[0209] In some examples, the signaling may be modified as follows:

[0210] [Table 7]

[0211]

[0176] The problems and solutions of the second aspect (aspect 2) of the present disclosure will be described. The neighbor search process for attribute LOD prediction described above, the G-PCC encoder 200 and the G-PCC decoder 300, are configured to derive six nearest neighbors. The third neighbor is replaced by one of the fourth, fifth, or sixth neighbors depending on certain conditions. A flag to enable the modified neighbor search process is used for attribute inter prediction (delimiters below) <section 1>and< / section 1> The maximum number of neighbors that can be used for the text between the neighbors is signaled regardless of the maximum number of neighbors that can be used for the text between the neighbors. This results in unnecessary bits being signaled and coding inefficiencies.

[0212] [Table 8]

[0213]

[0177] As a proposed solution to the above problem, the G-PCC encoder 200 and the G-PCC decoder 300 may be configured to determine whether an enable flag for modified neighbor discovery is signaled based on the maximum number of attribute neighbors. When it is determined that the enable flag is not signaled / does not exist, a default neighbor discovery process is applied (effectively inferring that the enable flag is 0).

[0214]

[0178] In one example, this determination may be based on the condition that the maximum number of attribute neighbors is greater than or equal to three.

[0215]

[0179] In another example, this determination may be based on the condition that the maximum number of attribute neighbors is equal to three.

[0216] An example implementation of the solution of aspect 2 is given below: The signaling of the flag enabling neighbor discovery is modified by adding a condition on the number of neighbors (&&aps.num_pred_nearest_neighbors_minus1>=2) as follows:

[0217] [Table 9]

[0218]

[0181] When predictionWithDistributionEnabled is not signaled, the value of predictionWithDistributionEnabled is inferred to be 0.

[0219]

[0182] The problems and solutions of the third aspect (Aspect 3) of the present disclosure will be described. The following examples describe several processes for replacing the third predictor with one of the fourth, fifth, or sixth predictor candidates. However, some of these techniques can also be applied to replacing other predictors (e.g., the zeroth or first predictor). For simplicity, replacing a predictor is sometimes referred to as replacing a direction.

[0220] The first problem (problem 1) of aspect 3 relates to the lack of a strict opposite check between P1 and P2. According to the first problem, the neighbor search process for attribute LOD prediction described above utilizes the derivation of six nearest neighbors. The neighbors are arranged in increasing distance from the current point. The third neighbor in the list is replaced by one of the fourth, fifth, or sixth neighbors depending on certain conditions. The basic idea of ​​the modified search process is to include prediction candidates in the "opposite direction" or "loosely opposite direction" to provide a better distribution of predictors. However, the currently defined rule is not optimal because replacement may still occur even when there is an opposite or loosely opposite direction between the first three candidates. This results in the inclusion of a more distant neighbor (fourth, fifth, or sixth neighbor) in the list instead of a closer neighbor (second neighbor), resulting in suboptimal performance.

[0221]

[0184] A proposed solution to problem 1 is that when P1 and P2 are in the exact opposite direction to each other, the neighbors are considered to be sufficiently diffused and the list is not updated. An implementation of this can be as follows: For step 2 above, the G-PCC encoder 200 and the G-PCC decoder 300 can perform a modified check of the point distribution (e.g., an exact opposite check). The modification for step 2 described is that the delimiter representing the text being added is <add> and< / add> The text between and the delimiter that indicates that the text is to be deleted <del> and< / del> It is explained using the text between

[0222]

[0185] The G-PCC encoder 200 and the G-PCC decoder 300 may be configured to check the distribution of points in list 1. If P1 or P2 is already in the exact opposite direction to P0 (the definition of opposite is shown in Figure 23), the distribution of neighbors is considered to be already sufficiently spread out, and therefore there is no need to check any more points in list 2. <add> Otherwise, if P1 is in the exact opposite direction to P2, there is no need to check any more points in list 2.< / add> Otherwise, check point Pn in list 2, and if dist(Pn,P) ≤ T1 and Pn is in the exact opposite direction to P0, use Pn to replace P2. Otherwise, if dist(Pn,P) ≤ T2 and Pn is in the exact opposite direction to P1, use Pn to replace P2. In the current implementation, T1 = w * dist(P2,P), T2=w * dist(P1,P) and w<<5=54.

[0223] The second problem (Problem 2) of aspect 3 concerns the unnecessary second threshold T2 in the strict opposite check. According to Problem 2, when checking for strict opposite directions, two thresholds are specified. When comparing the strict opposite direction with P0, threshold T1 is used, and when comparing the strict opposite direction with P1, threshold T2 is used. Because the list is first constructed based on distance, dist(P1,P) can be less than or equal to dist(P2,P). Therefore, T2 ≦ T1. When comparing the distance of a point from P, a point that satisfies dist(Pn,P) ≦ T2 always satisfies dist(Pn,P) ≦ T1. Having two thresholds and two comparisons leads to unnecessary complexity. If the goal is to check whether any point is strictly opposite to P0 or P1, it may be easier to check directly using one threshold.

[0224]

[0187] An implementation of the proposed solution to problem 2 could be as follows:

[0225] Step 2: Checking point distribution (strict opposite check) Check the distribution of points in list 1. If P1 or P2 is already in the exact opposite direction to P0 (the definition of opposite is shown in Figure 23), the distribution of neighbors is considered to be already spread out enough, and therefore there is no need to check any more points in list 2. Otherwise, check point Pn in list 2 and determine if dist(Pn, P) ≤ T1 and Pn is closer to P0. <add> or P1< / add> and in the strictly opposite direction, Pn is used to replace P2, <del> Otherwise, if dist(Pn,P) ≤ T2 and Pn is in the exact opposite direction to P1, use Pn to replace P2.< / del> In the current implementation, T1=w * dist(P2,P), <del>T2=w * dist(P1,P),< / del> w<<5=54.

[0226] The third problem (Problem 3) of aspect 3 relates to the lack of a check for loose opposition between P1 and P2. According to the third problem, when the algorithm reaches step 3, it has been established that P0, P1, and P2 are in the same half-3D plane, which then triggers a check for loose opposition directions. As in Problem 1, it is not tested whether P2 and P1 are in loose opposition to each other. If P1 and P2 are in loose opposition directions, replacing P2 with another point farther away from the current point P may not be optimal.

[0227]

[0189] An implementation of the solution to problem 3 can be as follows:

[0228] Step 3: Check the point distribution (loose contra check) After step 2, P2 has not been replaced and P2 <del> or P1< / del> In the case where P is in the same direction as P (P, P, and P are in the same region), the distribution check is relaxed to some extent. Specifically, the distribution check is based on a predefined loose opposite direction, as shown in FIG. 23. Similar to step 2, point P in list 1 is checked. If dist(P, P) ≦ T and P is in the loose opposite direction to P, P is used to replace P. Otherwise, if dist(P, P) ≦ T and P is in the loose opposite direction to P, P is used to replace P.

[0229] P2 has not been replaced after step 2, <del> P2 or< / del> P1 is in the same direction as P0 (P0, P1 and P2 are in the same region), <add> P0 is not loosely opposed to P2< / add> In this case, the distribution check is relaxed to some extent. Specifically, the distribution check is based on a predefined loose opposite direction, as shown in Figure 23. Similar to step 2, point Pn in list 1 is checked. If dist(Pn,P) ≤ T1 and Pn is in the loose opposite direction to P0, then Pn is used to replace P2. Otherwise, if dist(Pn,P) ≤ T2 and Pn is in the loose opposite direction to P1, then Pn is used to replace P2.

[0230]

[0190] In one example, P2 can be replaced only if it is not loosely opposed to P1 or P0.

[0231] Step 3: Check the point distribution (loose contra check) After step 2, P2 has not been replaced and P2 <del> or P1< / del> is in the same direction as P0 (P0, P1 and P2 are in the same region), <add> P1 is not loosely opposed to P2< / add> In this case, the distribution check is relaxed to some extent. Specifically, the distribution check is based on a predefined loose opposite direction, as shown in Figure 23. Similar to step 2, point Pn in list 1 is checked. If dist(Pn,P) ≤ T1 and Pn is in the loose opposite direction to P0, then Pn is used to replace P2. Otherwise, if dist(Pn,P) ≤ T2 and Pn is in the loose opposite direction to P1, then Pn is used to replace P2.

[0232]

[0191] P2 is not replaced after step 2, <del> P2 or< / del> P1 is in the same direction as P0 (P0, P1 and P2 are in the same region), <add> P0 is not loosely opposed to P2< / add> In this case, the distribution check is relaxed to some extent. Specifically, the distribution check is based on a predefined loose opposite direction, as shown in Figure 23. Similar to step 2, point Pn in list 1 is checked. If dist(Pn,P) ≤ T1 and Pn is in the loose opposite direction to P0, then Pn is used to replace P2. Otherwise, if dist(Pn,P) ≤ T2 and Pn is in the loose opposite direction to P1, then Pn is used to replace P2.

[0233]

[0192] The fourth problem (Problem 4) of aspect 3 relates to the lack of checking the equality of the two directions. According to Problem 4, when the algorithm reaches Step 3, it is established that P0, P1, and P2 are in the same half-3D plane, which then triggers a check of the loose opposite direction. Step 3 is triggered if P0 is equal to P2 or if P0 is equal to P1. However, if P1 is equal to P2, no check is performed. If P1 is equal to P2 and P2 is not replaced, it may result in a less "distributed" predictor, which is not optimal for prediction.

[0234]

[0193] The proposed solution is to check for equality in all cases when checking the loose opposite direction predictors. If all three directions are equal (the following delimiters <section 2>and< / section 2> If any of the directions 3..numend1 are loosely opposed to the direction 0 / 1, then direction 2 is substituted. If direction 2 is equal to direction 0 or direction 1, and direction 0 is not equal to direction 1 (the following delimiters <section 3>and< / section 3> the text between If direction 1 is loosely opposed to direction 0, no substitution is made. Otherwise, direction2 is replaced by direction3..numend1 with the smallest index that is not equal to either direction0 or direction1. (Since the three directions lie in the same half-plane, if a direction is not equal to either 0 or 1, then the direction not equal to 0 or 1 is automatically loosely opposite to either 0 or 1). If direction 2 is not equal to either direction 0 or 1, and direction 0 is equal to direction 1 (as in the following delimiter <section 4>the text between and < / section 4> If direction 2 is the loose opposite of direction 0, do not replace. Otherwise, direction 2 is replaced by direction 3..numend1 that has the smallest loose index with respect to direction 0.

[0235]

[0194] In another case, two of the directions 0, 1, and 2 are not equal. Since there are only four directions in the 3D half-plane, at least two of the directions 0, 1, and 2 are loosely opposite.

[0236]

Table 10-1

[0237]

Table 10-2

[0238]

Table 10-3

[0239]

[0195] The following examples represent exemplary solutions to the problems introduced above. In these examples, the delimiter <add> and< / add> the text between indicates that text has been added, and the delimiter <del> and< / del> the text between indicates that text has been deleted. The delimiters <Aspect X, Solution Y> and < / Aspect X, Solution Y> are intended to identify the text corresponding to the aspects and solutions introduced above.

[0240]

[0196] A first example including solutions to Problem 1, Problem 2, and Problem 3 is shown below.

[0241]

Table 11-1

[0242] [Table 11-2]

[0243] [Table 11-3]

[0244] [Table 11-4]

[0245] [Table 11-5]

[0246] [Table 11-6]

[0247] A second example is provided below, including solutions to Problem 1, Problem 2, and Problem 4.

[0248] [Table 12-1]

[0249] [Table 12-2]

[0250] [Table 12-3]

[0251]

[0197] A third example corresponding to another implementation of Example 2 is as follows.

[0252] [Table 13-1]

[0253] [Table 13-2]

[0254] [Table 13-3]

[0255]

[0198] The embodiments in the various aspects of the present disclosure may be used individually or in any combination.

[0256]

[0199] Figure 24 is a flowchart illustrating an example operation of a G-PCC decoder 300 in accordance with one or more techniques of this disclosure. However, the technique of Figure 24 may be performed by other types of G-PCC decoding devices. Furthermore, the technique of Figure 24 may be performed in whole or in part by the decoding function of a G-PCC encoding device, such as the G-PCC encoder 200.

[0257]

[0200] In the example of Figure 24, the G-PCC decoder 300 determines 2402 a first attribute value for the closest point to the current point in the point cloud, where the closest point corresponds to an already decoded point that is less than a threshold distance from the current point.

[0258]

[0201] The G-PCC decoder 300 determines 2404 a second attribute value for the second-closest point in the point cloud to the current point, where the second-closest point corresponds to a second already decoded point that is less than a threshold distance from the current point.

[0259]

[0202] The G-PCC decoder 300 determines 2406 a third attribute value for the third-closest point in the point cloud to the current point, where the third-closest point corresponds to the third already decoded point that is less than a threshold distance from the current point.

[0260]

[0203] The G-PCC decoder 300 determines (2408) a fourth attribute value for a fourth point of the point cloud. The fourth point corresponds to a fourth previously decoded point that is less than a threshold distance from the current point. The fourth point may be the fourth closest point to the current point, or may be, for example, the fifth or sixth closest point.

[0261] The G-PCC decoder 300 determines 2410 a set of predictor candidates for the attribute value of the current point of the point cloud based on a comparison of the location of the second point with the location of the third point. As described in more detail above, the comparison of the location of the second point with the location of the third point may include, for example, determining whether the second point and the third point are strictly opposite, loosely opposite, or not opposite. For example, if the second point and the third point are strictly opposite or loosely opposite, the G-PCC decoder 300 may include the attribute values ​​of the first point, the second point, and the third point in the set of predictor candidates. In one exemplary scenario, the attribute value of the fourth point is included in the set of predictor candidates only if none of the first point, the second point, and the third point are strictly opposite, and none of the first point, the second point, and the third point are loosely opposite.

[0262]

[0205] In some cases, the determination of whether the second point and the third point are strictly opposed, loosely opposed, or not opposed may be performed only in response to a determination that the first point and the second point are not strictly opposed or not loosely opposed, and / or in response to a determination that the first point and the third point are not strictly opposed or not loosely opposed.

[0263] The G-PCC decoder 300 decodes 2412 the attribute values ​​of the current point based on the set of predictor candidates. The G-PCC decoder 300 may then, for example, reconstruct a point cloud based on the decoded attribute values. The reconstructed point cloud may then be used in any of a variety of applications, including those described in Figures 25-28.

[0264] FIG. 25 is a conceptual diagram illustrating an example distance measurement system 2500 that can be used with one or more techniques of the present disclosure. In the example of FIG. 25, the distance measurement system 2500 includes an illuminator 2502 and a sensor 2504. The illuminator 2502 may emit light 2506. In some examples, the illuminator 2502 may emit the light 2506 as one or more laser beams. The light 2506 may be at one or more wavelengths, such as infrared wavelengths or visible light wavelengths. In other examples, the light 2506 is non-coherent laser light. When the light 2506 strikes an object, such as object 2508, the light 2506 forms return light 2510. The return light 2510 may include backscattered light and / or reflected light. The returning light 2510 may pass through a lens 2511 that directs the returning light 2510 to form an image 2512 of the object 2508 on the sensor 2504. The sensor 2504 generates a signal 2518 based on the image 2512. The image 2512 may include a set of points (e.g., as shown by the dots in the image 2512 of FIG. 25).

[0265] In some examples, the illuminator 2502 and sensor 2504 may be mounted on a rotating structure such that the illuminator 2502 and sensor 2504 capture a 360-degree view of the environment. In other examples, the distance measurement system 2500 may include one or more optical components (e.g., mirrors, collimators, diffraction gratings, etc.) that enable the illuminator 2502 and sensor 2504 to detect objects within a certain range (e.g., up to 360 degrees). Although the example of FIG. 25 shows only a single illuminator 2502 and sensor 2504, the distance measurement system 2500 may include multiple sets of illuminators and sensors.

[0266] In some examples, the illuminator 2502 generates a structured light pattern. In such examples, the distance measurement system 2500 may include multiple sensors 2504 on which respective images of the structured light pattern are formed. The distance measurement system 2500 may use the parallax between the images of the structured light pattern to determine the distance to an object 2508 from which the structured light pattern is backscattered. The structured light-based distance measurement system may have a high level of accuracy (e.g., accuracy in the sub-millimeter range) when the object 2508 is relatively close (e.g., 0.2 meters to 2 meters) to the sensor 2504. This high level of accuracy may be useful in facial recognition applications, such as unlocking mobile devices (e.g., cell phones, tablet computers, etc.), and for security applications.

[0267] In some examples, the distance measurement system 2500 is a time-of-flight (ToF)-based system. In some examples where the distance measurement system 2500 is a ToF-based system, the illuminator 2502 generates pulses of light. In other words, the illuminator 2502 may modulate the amplitude of the emitted light 2506. In such examples, the sensor 2504 detects returned light 2510 from the pulses of light 2506 generated by the illuminator 2502. The distance measurement system 2500 can then determine the distance to the object 2508 from which the light 2506 is backscattered based on the delay time between the emission and detection of the light 2506 and the known speed of light in air. In some examples, rather than (or in addition to) modulating the amplitude of the emitted light 2506, the illuminator 2502 may modulate the phase of the emitted light 2506. In such an example, the sensor 2504 may detect the phase of the returning light 2110 from the object 2508 and determine the distance to a point on the object 2508 using the speed of light and based on the time difference between when the illuminator 2502 generated the light 2506 at a particular phase and when the sensor 2504 detected the returning light 2510 at that particular phase.

[0268] In other examples, the point cloud may be generated without the use of the illuminator 2502. For example, in some examples, the sensor 2504 of the distance measurement system 2500 may include two or more optical cameras. In such examples, the distance measurement system 2500 may use the optical cameras to capture a stereo image of an environment including the object 2508. The distance measurement system 2500 (e.g., the point cloud generator 2520) may then calculate the difference between locations in the stereo image. The distance measurement system 2500 may then use this disparity to determine the distance to the location shown in the stereo image. From these distances, the point cloud generator 2520 may generate the point cloud.

[0269] The sensor 2504 may also detect other attributes of the object 2508, such as color and reflectance information. In the example of Figure 25, the point cloud generator 2520 may generate a point cloud based on the signals 2518 generated by the sensor 2504. The distance measurement system 2500 and / or the point cloud generator 2520 may form part of the data source 104 (Figure 1).

[0270] FIG. 26 is a conceptual diagram illustrating an example vehicle-based scenario in which one or more techniques of this disclosure may be used. In the example of FIG. 26, a vehicle 2600 includes a laser package 2602, such as a LIDAR system. Although not shown in the example of FIG. 26, the vehicle 2600 may also include a data source and a G-PCC encoder, such as G-PCC encoder 200 (FIG. 1). In the example of FIG. 26, the laser package 2602 emits a laser beam 2604 that reflects off a pedestrian 2606 or other object on the road. The data source of the vehicle 2600 may generate a point cloud based on a signal generated by the laser package 2602. The G-PCC encoder of the vehicle 2600 may encode the point cloud to generate a bitstream 2608. The bitstream 2608 may include many fewer bits than the uncoded point cloud obtained by the G-PCC encoder. An output interface of vehicle 2600 (e.g., output interface 108 (FIG. 1)) may transmit bitstream 2608 to one or more other devices. Thus, vehicle 2600 may be able to transmit bitstream 2608 to other devices more quickly than unencoded point cloud data. Additionally, bitstream 2608 may require less data storage capacity.

[0271] In the example of FIG. 26, vehicle 2600 may transmit bitstream 2608 to another vehicle 2610. Vehicle 2610 may include a G-PCC decoder, such as G-PCC decoder 300 (FIG. 1). The G-PCC decoder of vehicle 2610 may decode bitstream 2608 and reconstruct a point cloud. Vehicle 2610 may use the reconstructed point cloud for various purposes. For example, vehicle 2610 may determine based on the reconstructed point cloud that pedestrian 2606 is on the road in front of vehicle 2600 and therefore begin slowing down, e.g., even before the driver of vehicle 2610 recognizes that pedestrian 2606 is on the road. Thus, in some examples, vehicle 2610 may perform autonomous navigation operations, generate notifications or alerts, or perform other actions based on the reconstructed point cloud.

[0272] Additionally or alternatively, vehicle 2600 may transmit bitstream 2608 to server system 2612. Server system 2612 may use bitstream 2608 for various purposes. For example, server system 2612 may store bitstream 2608 for later reconstruction of a point cloud. In this example, server system 2612 may use the point cloud along with other data (e.g., vehicle telemetry data generated by vehicle 2600) to train an autonomous driving system. In other examples, server system 2612 may store bitstream 2608 for later reconstruction for forensic crash investigation (e.g., if vehicle 2600 collides with pedestrian 2606) or transmit notifications or instructions for navigation to vehicle 2600 or vehicle 2610.

[0273] FIG. 27 is a conceptual diagram illustrating an example extended reality system in which one or more techniques of this disclosure may be used. Extended reality (XR) is a term used to cover a range of technologies, including augmented reality (AR), mixed reality (MR), and virtual reality (VR). In the example of FIG. 27, a first user 2700 is located at a first location 2702. The user 2700 is wearing an XR headset 2704. Instead of the XR headset 2704, the user 2700 may use a mobile device (e.g., a mobile phone, a tablet computer, etc.). The XR headset 2704 includes a depth-sensing sensor, such as a LIDAR system, that detects the position of points on an object 2706 at the first location 2702. A data source in the XR headset 2704 may use signals generated by the depth-sensing sensor to generate a point cloud representation of the object 2706 at the location 2702. The XR headset 2704 may include a G-PCC encoder (e.g., G-PCC encoder 200 of FIG. 1) configured to encode the point cloud to generate a bitstream 2708.

[0274] The XR headset 2704 may transmit the bitstream 2708 (e.g., over a network such as the internet) to an XR headset 2710 worn by a user 2712 at a second location 2714. The XR headset 2710 may decode the bitstream 2708 and reconstruct the point cloud. The XR headset 2710 may use the point cloud to generate an XR visualization (e.g., an AR, MR, or VR visualization) representing the object 2706 at the location 2702. Thus, in some examples, such as when the XR headset 2710 generates a VR visualization, the user 2712 at the location 2714 may experience a 3D immersive experience of the location 2702. In some examples, the XR headset 2710 may determine the position of the virtual object based on the reconstructed point cloud. For example, the XR headset 2710 may determine, based on the reconstructed point cloud, that an environment (e.g., location 2702) includes a flat surface and determine that a virtual object (e.g., a cartoon character) should be located on the flat surface. The XR headset 2710 may generate an XR visualization of the virtual object in the determined position. For example, the XR headset 2710 may display the cartoon character sitting on the flat surface.

[0275] FIG. 28 is a conceptual diagram illustrating an example mobile device system in which one or more techniques of this disclosure may be used. In the example of FIG. 28, a mobile device 2800, such as a mobile phone or tablet computer, includes a depth-sensing sensor, such as a LIDAR system, that detects the positions of points on an object 2802 in the environment of the mobile device 2800. A data source of the mobile device 2800 may use signals generated by the depth-sensing sensor to generate a point cloud representation of the object 2802. The mobile device 2800 may include a G-PCC encoder (e.g., G-PCC encoder 200 of FIG. 1) configured to encode the point cloud to generate a bitstream 2804. In the example of FIG. 28, the mobile device 2800 may transmit the bitstream to a remote device 2806, such as a server system or another mobile device. The remote device 2806 may decode the bitstream 2804 to reconstruct the point cloud. The remote device 2806 may use the point cloud for various purposes. For example, the remote device 2806 may use the point cloud to generate a map of the environment of the mobile device 2800. For example, the remote device 2806 may generate a map of the interior of a building based on the reconstructed point cloud. In another example, the remote device 2806 may generate imagery (e.g., computer graphics) based on the point cloud. For example, the remote device 2806 may use the points of the point cloud as vertices of a polygon and use the color attributes of the points as a basis for shading the polygon. In some examples, the remote device 2806 may perform facial recognition using the point cloud.

[0276]

[0219] The following numbered clauses illustrate one or more aspects of the devices and techniques described in this disclosure.

[0277]

[0220] Clause 1. A device for processing point cloud data, the device comprising: a memory configured to store the point cloud data; and one or more processors implemented in circuitry, the one or more processors performing the following steps: determining a first attribute value for a first point of the point cloud, where the first point of the point cloud is a previously decoded point that is closest to a current point of the point cloud; determining a second attribute value for a second point of the point cloud, where the second point of the point cloud is a previously decoded point that is second-closest to the current point of the point cloud; determining a third attribute value for a third point of the point cloud, where the third point of the point cloud is a previously decoded point that is third-closest to the current point of the point cloud; and determining a fourth attribute value for a fourth point of the point cloud, where the fourth point of the point cloud is a previously decoded point that is determining a set of predictor candidates for attribute values ​​of the current point of the point cloud, wherein the current point defines an intersection of an x-axis, a y-axis, and a z-axis, the x-axis, the y-axis, and the z-axis forming an xy-plane, an xz-plane, and a yz-plane, and wherein to determine the set of predictor candidates for the current point of the point cloud, the one or more processors are further configured to generate a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and a fourth attribute value based on a comparison of the location of the second point and the location of the third point; and decoding the attribute values ​​of the current point based on the set of predictor candidates.

[0278]

[0221] Clause 2. The device described in Clause 1, wherein the one or more processors are further configured to: determine whether the second point is exactly opposite the third point, wherein the one or more processors are further configured to generate a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point with the location of the third point; determine that the two points are exactly opposite if they are on opposite sides of the xy plane, on opposite sides of the xz plane, and on opposite sides of the yz plane; and in response to determining that the second point is exactly opposite the third point, include the first attribute value, the second attribute value, and the third attribute value in the set of predictor candidates.

[0279]

[0222] Clause 3. The device described in Clause 2, wherein the one or more processors are further configured to: determine whether the second point is strictly opposite to the first point; and, in response to determining that the second point is not strictly opposite to the first point, determine whether the second point is strictly opposite to the third point, to generate a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point with the location of the third point.

[0280]

[0223] Clause 4. The device described in Clause 2, wherein the one or more processors are further configured to: determine whether the third point is an exact opposite of the first point; and, in response to determining that the third point is not an exact opposite of the first point, determine whether the second point is an exact opposite of the third point, in order to generate a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point with the location of the third point.

[0281]

[0224] Clause 5. The device described in Clause 2, wherein the one or more processors are further configured to: determine whether the second point is the exact opposite of the first point; determine whether the third point is the exact opposite of the first point; and, in response to determining that the second point is not the exact opposite of the first point and determining that the third point is not the exact opposite of the first point, in order to generate a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point with the location of the third point.

[0282]

[0225] Clause 6. The device described in Clause 1, wherein the one or more processors are further configured to include the first attribute value, the second attribute value, and the third attribute value in the set of predictor candidates without including the fourth attribute value in the set of predictor candidates in response to determining that the second point is on the opposite side of the xy plane, the opposite side of the xz plane, and the opposite side of the yz plane relative to the third point, to generate a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison between the location of the second point and the location of the third point.

[0283]

[0226] Clause 7. The device described in Clause 1, wherein the one or more processors are further configured to: determine whether the third point and the second point are loosely opposite to generate a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point, wherein the two points are loosely opposite if they are on two opposite sides of the xy plane, the xz plane, and the yz plane and on the same side of one of the xy plane, the xz plane, and the yz plane; and in response to determining that the second point is loosely opposite to the third point, include the first attribute value, the second attribute value, and the third attribute value in the set of predictor candidates.

[0284]

[0227] Clause 8. A device described in any of clauses 1 to 7, wherein the one or more processors are further configured to determine whether the maximum number of neighbors to be used for prediction is at least three, and in response to determining that the maximum number of neighbors to be used for prediction is at least three, receive a syntax element indicating that a fourth attribute value for a fourth point is eligible for inclusion in the set of predictor candidates.

[0285]

[0228] Clause 9. To decode an attribute value of a current point based on a set of predictor candidates, one or more processors determine a candidate from a set of predictor candidates and receive a residual value; 9. The device of any of clauses 1 to 8, further configured to: determine an attribute value for the current point based on the candidate value and the residual value.

[0286]

[0229] Clause 10. A device described in any of clauses 1 to 9, wherein the first attribute value, the second attribute value, the third attribute value, the fourth attribute value, and the attribute value of the current point include color values.

[0287]

[0230] Clause 11. A device described in any of clauses 1 to 10, wherein the one or more processors are further configured to generate a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point, and decode the attribute value of the current point based on the set of predictor candidates as part of the process of encoding the point cloud data.

[0288]

[0231] Clause 12. The device of any of clauses 1 to 12, wherein the one or more processors are further configured to reconstruct a point cloud based on attribute values ​​of the current point. Clause 13. The device of clause 12, wherein the one or more processors are further configured to generate a map of the interior of the building based on the reconstructed point cloud.

[0289]

[0232] Clause 14. The device of clause 12, wherein the one or more processors are further configured to perform autonomous navigation operations based on the reconstructed point cloud.

[0290]

[0233] Clause 15. The device of clause 12, wherein the one or more processors are further configured to generate computer graphics based on the reconstructed point cloud.

[0291]

[0234] Clause 16. A device as described in clause 12, wherein one or more processors are configured to determine the position of a virtual object based on the reconstructed point cloud and generate an extended reality (XR) visualization in which the virtual object is at the determined position.

[0292]

[0235] Clause 17. The device of clause 12, further comprising a display for presenting an image based on the reconstructed point cloud.

[0293]

[0236] Clause 18. A device according to any one of clauses 1 to 17, wherein the device is one of a mobile phone or a tablet computer.

[0294]

[0237] Clause 19. A device according to any one of clauses 1 to 17, wherein the device is a vehicle.

[0295]

[0238] Clause 20. A device according to any of clauses 1 to 17, wherein the device is an extended reality device.

[0296]

[0239] Clause 21. A method for processing point cloud data, comprising: determining a first attribute value for a first point of the point cloud, the first point of the point cloud being the closest previously decoded point to a current point of the point cloud; determining a second attribute value for a second point of the point cloud, the second point of the point cloud being the second closest previously decoded point to the current point of the point cloud; determining a third attribute value for a third point of the point cloud, the third point of the point cloud being the third closest previously decoded point to the current point of the point cloud; and determining a fourth attribute value for a fourth point of the point cloud, the fourth point of the point cloud being the third closest previously decoded point to the current point. determining a set of predictor candidates for attribute values ​​of a current point of the point cloud, wherein the current point defines an intersection of an x-axis, a y-axis, and a z-axis, the x-axis, the y-axis, and the z-axis form an xy-plane, an xz-plane, and a yz-plane, and determining the set of predictor candidates for the current point of the point cloud includes generating a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point; and decoding an attribute value of the current point based on the set of predictor candidates.

[0297]

[0240] Clause 22. The device described in Clause 21, wherein generating a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point with the location of the third point includes: determining whether the second point is strictly opposite to the third point, wherein the two points are strictly opposite if they are on opposite sides of the xy plane, opposite sides of the xz plane, and opposite sides of the yz plane; and in response to determining that the second point is strictly opposite to the third point, including the first attribute value, the second attribute value, and the third attribute value in the set of predictor candidates.

[0298]

[0241] Clause 23. The device described in Clause 22, wherein generating a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point with the location of the third point includes: determining whether the second point is strictly opposite to the first point; and, in response to determining that the second point is not strictly opposite to the first point, determining whether the second point is strictly opposite to the third point.

[0299]

[0242] Clause 24. The device described in Clause 22, wherein generating a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point with the location of the third point includes: determining whether the third point is the exact opposite of the first point; and, in response to determining that the third point is not the exact opposite of the first point, determining whether the second point is the exact opposite of the third point.

[0300]

[0243] Clause 25. The device described in Clause 22, wherein generating a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point with the location of the third point includes: determining whether the second point is the exact opposite of the first point; determining whether the third point is the exact opposite of the first point; and in response to determining that the second point is not the exact opposite of the first point and determining that the third point is not the exact opposite of the first point, determining whether the second point is the exact opposite of the third point.

[0301]

[0244] Clause 26. The device described in Clause 21, wherein generating a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point includes, in response to determining that the second point is on the opposite side of the xy plane, the opposite side of the xz plane, and the opposite side of the yz plane relative to the third point, including the first attribute value, the second attribute value, and the third attribute value in the set of predictor candidates without including the fourth attribute value in the set of predictor candidates.

[0302]

[0245] Clause 27. The device described in Clause 21, wherein generating a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point includes: determining whether the third point and the second point are loosely opposite, wherein the two points are loosely opposite if they are on two opposite sides of the xy plane, the xz plane, and the yz plane and on the same side of one of the xy plane, the xz plane, and the yz plane; and in response to determining that the second point is loosely opposite of the third point, including the first attribute value, the second attribute value, and the third attribute value in the set of predictor candidates.

[0303]

[0246] Clause 28. A device described in any of clauses 21 to 27, further comprising: determining whether the maximum number of neighbors to be used for prediction is at least three; and in response to determining that the maximum number of neighbors to be used for prediction is at least three, receiving a syntax element indicating that a fourth attribute value for a fourth point is eligible for inclusion in the set of predictor candidates.

[0304]

[0247] Clause 29. A device described in any of clauses 21 to 28, wherein decoding an attribute value of a current point based on a set of predictor candidates includes determining a candidate from the set of predictor candidates, receiving a residual value, and determining an attribute value for the current point based on the candidate value and the residual value.

[0305]

[0248] Clause 30. A device described in any of clauses 21 to 29, wherein the first attribute value, the second attribute value, the third attribute value, the fourth attribute value, and the attribute value of the current point include color values.

[0306]

[0249] Clause 31. A device described in any of clauses 21 to 31, further comprising generating a set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison between the location of the second point and the location of the third point, and decoding the attribute value of the current point based on the set of predictor candidates as part of the process of encoding the point cloud data.

[0307]

[0250] Clause 32. A computer-readable storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to perform the method of any of clauses 21 to 31.

[0308]

[0251] Clause 33. A device for processing point cloud data, comprising: means for determining a first attribute value for a first point of the point cloud, the first point of the point cloud being a previously decoded point closest to a current point of the point cloud; means for determining a second attribute value for a second point of the point cloud, the second point of the point cloud being a previously decoded point second closest to the current point of the point cloud; means for determining a third attribute value for a third point of the point cloud, the third point of the point cloud being a previously decoded point third closest to the current point of the point cloud; and means for determining a fourth attribute value for a fourth point of the point cloud, the fourth point of the point cloud being a previously decoded point farther from the current point than the third point; 1. A device comprising: means for determining a set of predictor candidates for attribute values ​​of a current point of a point cloud, the current point defining an intersection of an x-axis, a y-axis, and a z-axis, the x-axis, y-axis, and z-axis forming an xy-plane, an xz-plane, and a yz-plane, to determine the set of predictor candidates for the current point of the point cloud, the means for determining the set of predictor candidates comprising means for generating a set of predictor candidates having a subset of a first attribute value, a second attribute value, a third attribute value, and a fourth attribute value based on a comparison of a location of a second point and a location of a third point; and means for decoding the attribute values ​​of the current point based on the set of predictor candidates.

[0309] It should be recognized that in some examples, some acts or events of any of the techniques described herein may be performed in a different order, added, merged, or omitted entirely (e.g., not all described acts or events may be required to practice the techniques). Moreover, in some examples, acts or events may be performed in parallel rather than sequentially, for example, through multithreading, interrupt processing, or multiple processors.

[0310] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which correspond to tangible media such as data storage media, or communication media, including any medium that facilitates transfer of a computer program from one place to another, for example, according to a communication protocol. As such, computer-readable media may generally correspond to (1) tangible computer-readable storage media that is non-transitory, or (2) a communication medium such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures to implement the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0311]

[0254] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included within the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but instead cover non-transitory tangible storage media. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically using a laser. Combinations of the above should also be included within the scope of computer-readable media.

[0312]

[0255] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the terms "processor" and "processing circuitry," as used herein, may refer to any of the above structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or may be incorporated into a combined codec. Also, the techniques may be implemented entirely in one or more circuits or logic elements.

[0313] The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units have been described in this disclosure to highlight functional aspects of devices configured to implement the disclosed techniques, but they do not necessarily require realization by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit or may be provided by a collection of interoperable hardware units, including one or more processors as described above, in conjunction with suitable software and / or firmware.

[0314]

[0257] Various examples have been described. These and other examples are within the scope of the following claims.< / section> < / section> < / section> < / section>

Claims

1. 1. A device for processing point cloud data, said device comprising: a memory configured to store the point cloud data; one or more processors implemented in a circuit; wherein the one or more processors: determining a first attribute value for a first point of a point cloud, the first point of the point cloud being a previously decoded point that is closest to a current point of the point cloud; determining a second attribute value for a second point of the point cloud, the second point of the point cloud being the second-closest previously decoded point of the point cloud to the current point; determining a third attribute value for a third point of the point cloud, the third point of the point cloud being the third closest previously decoded point of the point cloud to the current point; determining a fourth attribute value for a fourth point of the point cloud, the fourth point of the point cloud being a previously decoded point that is either farther from the current point than the third point or the same distance to the current point as the third point; determining a set of predictor candidates for attribute values ​​of the current point of the point cloud, the current point defining an intersection of an x-axis, a y-axis, and a z-axis, the x-axis, the y-axis, and the z-axis forming an x-y plane, an x-z ​​plane, and a y-z plane, and wherein to determine the set of predictor candidates for the current point of the point cloud, the one or more processors are further configured to generate the set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of a location of the second point and a location of the third point; decoding the attribute value of the current point based on the set of predictor candidates; A device that is configured to:

2. the one or more processors to generate the set of predictor candidates having the subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point; determining whether the second point is exactly opposite the third point, where two points are exactly opposite if they are on opposite sides of the xy plane, on opposite sides of the xz plane, and on opposite sides of the yz plane; in response to determining that the second point is an exact opposite of the third point, including the first attribute value, the second attribute value, and the third attribute value in the set of candidate predictors; The device of claim 1 , further configured to:

3. the one or more processors to generate the set of predictor candidates having the subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point; determining whether the second point is the exact opposite of the first point; determining whether the second point is exactly opposite the third point in response to determining that the second point is not exactly opposite the first point; and The device of claim 2 , further configured to:

4. the one or more processors to generate the set of predictor candidates having the subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point; determining whether the third point is the exact opposite of the first point; determining whether the second point is the exact opposite of the third point in response to determining that the third point is not the exact opposite of the first point; and The device of claim 2 , further configured to:

5. the one or more processors to generate the set of predictor candidates having the subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point; determining whether the second point is the exact opposite of the first point; determining whether the third point is the exact opposite of the first point; determining whether the second point is the exact opposite of the third point in response to determining that the second point is not the exact opposite of the first point and determining that the third point is not the exact opposite of the first point; and The device of claim 2 , further configured to:

6. the one or more processors to generate the set of predictor candidates having the subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point; 2. The device of claim 1, further configured to, in response to determining that the second point is on an opposite side of the x-y plane, an opposite side of the x-z plane, and an opposite side of the y-z plane relative to the third point, include the first attribute value, the second attribute value, and the third attribute value in the set of predictor candidates without including the fourth attribute value in the set of predictor candidates.

7. the one or more processors to generate the set of predictor candidates having the subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on the comparison of the location of the second point and the location of the third point; determining whether the third point and the second point are loosely opposed, where two points are loosely opposed if they are on two opposite sides of the xy plane, the xz plane, and the yz plane, and on the same side of one of the xy plane, the xz plane, and the yz plane; in response to determining that the second point is loosely opposed to the third point, including the first attribute value, the second attribute value, and the third attribute value in the set of candidate predictors; The device of claim 1 , further configured to:

8. the one or more processors: determining whether the maximum number of neighbors used for prediction is at least three; receiving a syntax element indicating that the fourth attribute value for the fourth point is eligible for inclusion in the set of predictor candidates in response to determining that the maximum number of neighbors to be used for prediction is at least three; The device of claim 1 , further configured to:

9. to decode the attribute value of the current point based on the set of predictor candidates, the one or more processors: determining a candidate from the set of predictor candidates; receiving a residual value; determining the attribute value for the current point based on the candidate value and the residual value; The device of claim 1 , further configured to:

10. The device of claim 1 , wherein the first attribute value, the second attribute value, the third attribute value, the fourth attribute value, and the attribute value of the current point comprise color values.

11. 2. The device of claim 1, wherein the one or more processors are further configured to generate the set of predictor candidates having the subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on the comparison of the location of the second point with the location of the third point, and to decode the attribute value of the current point based on the set of predictor candidates as part of a process of encoding the point cloud data.

12. The device of claim 1 , wherein the one or more processors are further configured to reconstruct the point cloud based on the attribute values ​​of the current point.

13. The device of claim 12 , wherein the one or more processors are further configured to generate a map of an interior of a building based on the reconstructed point cloud.

14. The device of claim 12 , wherein the one or more processors are further configured to perform autonomous navigation operations based on the reconstructed point cloud.

15. The device of claim 12 , wherein the one or more processors are further configured to generate computer graphics based on the reconstructed point cloud.

16. the one or more processors: determining a position of a virtual object based on the reconstructed point cloud; generating an extended reality (XR) visualization with the virtual object at the determined location; The device of claim 12 configured to:

17. The device of claim 12 , further comprising a display for presenting an image based on the reconstructed point cloud.

18. The device of claim 1 , wherein the device is one of a mobile phone or a tablet computer.

19. The device of claim 1 , wherein the device is a vehicle.

20. The device of claim 1 , wherein the device is an extended reality device.

21. 1. A method for processing point cloud data, comprising: determining a first attribute value for a first point of a point cloud, the first point of the point cloud being a previously decoded point that is closest to a current point of the point cloud; determining a second attribute value for a second point of the point cloud, the second point of the point cloud being the second-closest previously decoded point of the point cloud to the current point; determining a third attribute value for a third point of the point cloud, the third point of the point cloud being the third closest previously decoded point of the point cloud to the current point; determining a fourth attribute value for a fourth point of the point cloud, the fourth point of the point cloud being a previously decoded point that is farther from the current point than the third point; determining a set of predictor candidates for attribute values ​​of the current point of the point cloud, the current point defining an intersection of an x-axis, a y-axis, and a z-axis, the x-axis, the y-axis, and the z-axis forming an x-y plane, an x-z ​​plane, and a y-z plane, and determining the set of predictor candidates for the current point of the point cloud includes generating the set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point; decoding the attribute value of the current point based on the set of predictor candidates; A method comprising:

22. generating the set of predictor candidates having the subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point; determining whether the second point is exactly opposite the third point, where two points are exactly opposite if they are on opposite sides of the xy plane, on opposite sides of the xz plane, and on opposite sides of the yz plane; in response to determining that the second point is an exact opposite of the third point, including the first attribute value, the second attribute value, and the third attribute value in the set of candidate predictors; 22. The device of claim 21, comprising:

23. generating the set of predictor candidates having the subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point; determining whether the second point is the exact opposite of the first point; and determining whether the second point is exactly opposite the third point in response to determining that the second point is not exactly opposite the first point.

24. generating the set of predictor candidates having the subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point; determining whether the third point is the exact opposite of the first point; determining whether the second point is the exact opposite of the third point in response to determining that the third point is not the exact opposite of the first point; and 23. The device of claim 22, comprising:

25. generating the set of predictor candidates having the subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point; determining whether the second point is the exact opposite of the first point; determining whether the third point is the exact opposite of the first point; determining whether the second point is the exact opposite of the third point in response to determining that the second point is not the exact opposite of the first point and determining that the third point is not the exact opposite of the first point; and 23. The device of claim 22, comprising:

26. generating the set of predictor candidates having the subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of the location of the second point and the location of the third point; 22. The device of claim 21, responsive to determining that the second point is on an opposite side of the x-y plane, an opposite side of the x-z plane, and an opposite side of the y-z plane relative to the third point, including the first attribute value, the second attribute value, and the third attribute value in the set of predictor candidates without including the fourth attribute value in the set of predictor candidates.

27. generating the set of predictor candidates having the subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on the comparison of the location of the second point and the location of the third point; determining whether the third point and the second point are loosely opposed, where the two points are loosely opposed if they are on two opposite sides of the xy plane, the xz plane, and the yz plane, and on the same side of one of the xy plane, the xz plane, and the yz plane; in response to determining that the second point is a loose opposite of the third point, including the first attribute value, the second attribute value, and the third attribute value in the set of candidate predictors; 22. The device of claim 21, comprising:

28. determining whether the maximum number of neighbors used for prediction is at least three; receiving a syntax element indicating that the fourth attribute value for the fourth point is eligible for inclusion in the set of predictor candidates in response to determining that the maximum number of neighbors to use for prediction is at least three; 22. The device of claim 21 further comprising:

29. decoding the attribute value of the current point based on the set of predictor candidates; determining a candidate from the set of predictor candidates; receiving a residual value; determining the attribute value for the current point based on the candidate value and the residual value; 22. The device of claim 21, comprising:

30. 22. The device of claim 21, wherein the first attribute value, the second attribute value, the third attribute value, the fourth attribute value, and the attribute value of the current point comprise color values.

31. 22. The device of claim 21, further comprising: generating the set of predictor candidates having the subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on the comparison of the location of the second point and the location of the third point; and decoding the attribute value of the current point based on the set of predictor candidates as part of a process of encoding the point cloud data.

32. 1. A computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to: determining a first attribute value for a first point of a point cloud, the first point of the point cloud being a previously decoded point that is closest to a current point of the point cloud; determining a second attribute value for a second point of the point cloud, the second point of the point cloud being the second-closest previously decoded point of the point cloud to the current point; determining a third attribute value for a third point of the point cloud, the third point of the point cloud being the third closest previously decoded point of the point cloud to the current point; determining a fourth attribute value for a fourth point of the point cloud, the fourth point of the point cloud being a previously decoded point that is either farther from the current point than the third point or the same distance to the current point as the third point; determining a set of predictor candidates for attribute values ​​of the current point of the point cloud, the current point defining an intersection of an x-axis, a y-axis, and a z-axis, the x-axis, the y-axis, and the z-axis forming an x-y plane, an x-z ​​plane, and a y-z plane, and wherein to determine the set of predictor candidates for the current point of the point cloud, the one or more processors are further configured to generate the set of predictor candidates having a subset of the first attribute value, the second attribute value, the third attribute value, and the fourth attribute value based on a comparison of a location of the second point and a location of the third point; decoding the attribute value of the current point based on the set of predictor candidates; A computer-readable storage medium that causes the

Images