Inter-predictive coding for geometry point cloud compression

Abstract

A device for encoding or decoding point cloud data may be configured to, in response to a decision to predict a current point in a current frame using an inter prediction mode, identify a reference frame, derive one or more motion compensated reference points from the reference frame, selectively resample points in the reference frame based on the one or more motion compensated reference points to generate a resampled reference frame, and inter predict the current point in the current frame based on the resampled reference frame.

Description

[Technical field]

[0001]

[0001] This application claims priority to U.S. Patent Application No. 18 / 303,390, filed April 19, 2023, U.S. Provisional Patent Application No. 63 / 363,306, filed April 20, 2022, and U.S. Provisional Patent Application No. 62 / 363,552, filed April 25, 2022, the entire contents of each of which are incorporated herein by reference. U.S. Patent Application No. 18 / 303,390, filed April 19, 2023, claims the benefit of U.S. Provisional Patent Application No. 63 / 363,306, filed April 20, 2022, and U.S. Provisional Patent Application No. 62 / 363,552, filed April 25, 2022.

[0002]

[0002] This disclosure relates to point cloud encoding and decoding. [Background technology]

[0003]

[0003] A point cloud is a collection of points in a three-dimensional space. The points may correspond to points on an object in the three-dimensional space. Thus, a point cloud may be used to represent the physical content of a three-dimensional space. A point cloud may have utility in a wide variety of situations. For example, a point cloud may be used to represent the position of an object on a road in the context of an autonomous vehicle. In another example, a point cloud may be used to position virtual objects in an augmented reality (AR) or mixed reality (MR) application in the context of representing the physical content of an environment. Point cloud compression is the process of encoding and decoding a point cloud. Encoding a point cloud may reduce the amount of data required for storage and transmission of the point cloud. Summary of the Invention

[0004]

[0004] In general, this disclosure describes techniques for coding nodes of a point cloud using inter-prediction, such as for the currently developed Geometry Point Cloud Compression (G-PCC) standard. However, the exemplary techniques are not limited to the G-PCC standard. In some examples of G-PCC, the coordinates of the position of a node (also called a point) of a point cloud may be converted to the (r, φ, i) domain, where the position of the node is represented by three parameters: radius r, azimuth angle φ, and laser index i (e.g., laser identifier). When using an angle mode for predictive geometry coding in G-PCC, the G-PCC coder may perform prediction in the (r, φ, i) domain. For example, to code a particular node of a particular frame of a point cloud, the G-PCC coder may determine predicted values ​​for one or more of the radius r, azimuth angle φ, and laser index i of the particular node based on another node of the particular frame, and add the predicted values ​​to the residual data (e.g., the residual radius r, the residual azimuth angle φ, or the residual laser index i) to determine the reconstructed radius r, azimuth angle φ, or laser index i of the particular node. Since coding the residual data may account for a large portion of the coding overhead, the coding efficiency (e.g., the amount of bits used to code a point) may be affected by how close the predicted values ​​(e.g., the predicted radius r, azimuth angle φ, or laser index i) of the particular node are to the reconstructed values ​​(e.g., the reconstructed radius r, azimuth angle φ, or laser index i) of the particular node. Thus, generating more accurate prediction parameters for the nodes may improve overall coding performance.

[0005]

[0005] According to one or more techniques of the present disclosure, a G-PCC coder may predict a current point of a current frame of a point cloud using inter prediction. For example, to predict a current point using inter prediction, the G-PCC coder may identify a reference point in a frame different from the current frame and predict one or more parameters of the current point (e.g., radius r, azimuth angle φ, and laser index i) based on one or more parameters of the reference point. In a coding scenario in which the parameters of the reference point are closer to the parameters of the current point than the parameters of a parent or other available point in the current frame, predicting the current point using inter prediction may reduce the size of the residual data. In this manner, the inter prediction technique of the present disclosure may enable the G-PCC coder to improve coding efficiency.

[0006]

[0006] According to one example of the present disclosure, a device for processing (e.g., encoding or decoding) point cloud data comprises a memory configured to store the point cloud data and one or more processors implemented in a circuit, where the one or more processors are configured to, in response to a decision to predict a current point in a current frame using an inter prediction mode, identify a reference frame, derive one or more motion compensated reference points from the reference frame, selectively resample points in the reference frame based on the one or more motion compensated reference points to generate a resampled reference frame, and inter predict the current point in the current frame based on the resampled reference frame.

[0007]

[0007] According to one example of the present disclosure, a method for processing (e.g., encoding or decoding) a point cloud includes, in response to a decision to predict a current point in a current frame using an inter prediction mode, identifying a reference frame, deriving one or more motion compensated reference points from the reference frame, selectively resampling points in the reference frame based on the one or more motion compensated reference points to generate a resampled reference frame, and inter predicting the current point in the current frame based on the resampled reference frame.

[0008]

[0008] A computer-readable storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to, in response to a decision to predict a current point in a current frame using an inter prediction mode, identify a reference frame, derive one or more motion compensated reference points from the reference frame, selectively resample points in the reference frame based on the one or more motion compensated reference points to generate a resampled reference frame, and inter predict a current point in the current frame based on the resampled reference frame.

[0009]

[0009] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will become apparent from the description, drawings, and claims. [Brief description of the drawings]

[0010] [Figure 1]

[0010] FIG. 1 is a block diagram illustrating an example encoding and decoding system in which techniques of this disclosure may be implemented. [Diagram 2]

[0011] FIG. 1 is a block diagram illustrating an example geometry point cloud compression (G-PCC) encoder, in accordance with one or more aspects of the present disclosure. [Diagram 3]

[0012] FIG. 2 is a block diagram illustrating an example G-PCC decoder in accordance with one or more aspects of the present disclosure. [Figure 4]

[0013] FIG. 1 is a conceptual diagram illustrating octree partitioning for geometry coding in accordance with one or more aspects of the present disclosure. [Figure 5A]

[0014] FIG. 1 is a conceptual diagram of a rotating light detection and ranging LIDAR acquisition model in accordance with one or more aspects of the present disclosure. [Figure 5B] FIG. 1 is a conceptual diagram of a rotating light detection and ranging LIDAR acquisition model in accordance with one or more aspects of the present disclosure. [Figure 6]

[0015] FIG. 2 is a conceptual diagram of a prediction tree for predictive geometry coding, in accordance with one or more aspects of the present disclosure. [Figure 7]

[0016] FIG. 2 is a conceptual diagram illustrating an example inter-prediction process for predicting points of a point cloud, in accordance with one or more aspects of the present disclosure. [Figure 8]

[0017] 11 is a flow diagram illustrating an example decoding flow associated with a syntax element indicating whether a node is coded in inter-prediction mode or intra-prediction mode. FIG. [Figure 9]

[0018] FIG. 13 is a conceptual diagram illustrating an additional inter predictor point obtained from an initial point having a larger azimuth angle than the inter predictor point. [Figure 10]

[0019] 11 is a flowchart illustrating an example technique of how resampled points may be used for inter prediction, in accordance with one or more techniques of this disclosure. [Figure 11]

[0020] 11 is a flowchart illustrating an example technique for how points in a globally compensated reference frame may be resampled using points from a zero-compensated reference frame, in accordance with one or more techniques of this disclosure. [Figure 12A]

[0021] FIG. 13 is a conceptual diagram illustrating how a first set of points in a globally compensated reference frame may be selected for points in a zero-compensated reference frame, in accordance with one or more aspects of the present disclosure. [Figure 12B] FIG. 13 is a conceptual diagram illustrating how a first set of points in a globally compensated reference frame may be selected for points in a zero-compensated reference frame, in accordance with one or more aspects of the present disclosure. [Figure 12C] FIG. 13 is a conceptual diagram illustrating how a first set of points in a globally compensated reference frame may be selected for points in a zero-compensated reference frame, in accordance with one or more aspects of the present disclosure. [Figure 12D] FIG. 13 is a conceptual diagram illustrating how a first set of points in a globally compensated reference frame may be selected for points in a zero-compensated reference frame, in accordance with one or more aspects of the present disclosure. [Figure 13]

[0022] 1 is a flowchart illustrating a process in accordance with the techniques of this disclosure. [Figure 14]

[0023] FIG. 1 is a conceptual diagram illustrating an example distance measurement system that may be used with one or more techniques of the present disclosure. [Figure 15]

[0024] FIG. 1 is a conceptual diagram illustrating an example vehicle-based scenario in which one or more techniques of the present disclosure may be used. [Figure 16]

[0025] FIG. 1 is a conceptual diagram illustrating an example extended reality system in which one or more techniques of this disclosure may be used. [Figure 17]

[0026] FIG. 1 is a conceptual diagram illustrating an example mobile device system in which one or more techniques of the present disclosure may be used. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011]

[0027] When using the angular mode for predictive geometry coding in G-PCC, the G-PCC coder may perform prediction in the angular domain, also referred to as the (r,φ,i) domain. For example, to code a particular node of a particular frame of a point cloud, the G-PCC coder may determine predictions for one or more of the radius r, azimuth angle φ, or laser index i of the particular node based on another node of the particular frame, and add the predicted radius r, azimuth angle φ, and laser index i of the node to the residual data (e.g., residual radius r, residual azimuth angle φ, or residual laser index i) to determine the reconstructed radius r, azimuth angle φ, or laser index i of the particular node. Because coding the residual data can account for significant coding overhead, the coding efficiency (e.g., the amount of bits used to code a point) may be affected by how close the predicted radius r, azimuth angle φ, and laser index i of the particular node are to the reconstructed radius r, azimuth angle φ, and laser index i of the particular node. Therefore, generating more accurate prediction parameters for the nodes may improve the overall coding performance.

[0012]

[0028] According to one or more techniques of the present disclosure, a G-PCC coder may predict a current point of a current frame of a point cloud using inter prediction. For example, to predict a current point using inter prediction, the G-PCC coder may identify a reference point in a frame different from the current frame and predict one or more parameters of the current point (e.g., radius r, azimuth angle φ, and laser index i) based on one or more parameters of the reference point. In a coding scenario, if the parameters of the reference point are closer to the parameters of the current point than the parent or other available points in the current frame, predicting the current point using inter prediction may reduce the size of the residual data compared to intra prediction. In this manner, the inter prediction techniques of the present disclosure may enable the G-PCC coder to improve coding efficiency.

[0013]

[0029] As described in more detail below, inter prediction uses points in a reference frame to predict points in a current frame. In some examples, a zero-motion compensated reference frame, also called a zero-compensated reference frame, may be used, and in some cases, a global motion compensated reference frame may be used for prediction. In a zero-compensated frame, points in a reference frame are used without applying motion compensation. Thus, a zero-motion compensated reference frame may also be simply referred to as a reference frame in this disclosure. A global motion compensated reference frame may be obtained from a (zero-compensated) reference frame by performing motion compensation based on a set of motion parameters. The motion parameters may be derived, for example, using a motion estimation technique, or may be obtained from an external source, such as using GPS parameters, or from the motion of a car capturing a point cloud. When motion compensation is applied to points in a reference frame, the location of the points is modified, which may result in a decrease in coding efficiency.

[0014]

[0030] The reference frame includes points, each of which is associated with a reconstructed position. This reconstructed position is the position of the point relative to a first position in the reference frame. For example, if the points are captured by a light detection and ranging (LIDAR) sensor mounted on the vehicle, the first position may indicate a position associated with the vehicle, such as the position of a particular point in the vehicle / sensor. The position of the point in the reference frame may be relative to the first position. Compared to the reference frame, the vehicle may have moved, and thus a second position may be associated with the vehicle in the current frame such that the position of the point in the current frame is relative to the second position. If there is motion between the reference frame and the current frame, the first and second positions are different. In this case, when motion compensation is applied, the positions of one or more points in the reference frame (which were relative to the first position) may be modified such that the positions of these points are relative to the second position. For example, assume that the vehicle is at location (1,1,1) in the reference frame and moves to location (3,4,5) in the second frame. The first position is (1,1,1) and the second position is (3,4,5). The motion parameters in this example are translations only, and the motion vector is (2,3,4) (obtained by subtracting 1,1,1 from 3,4,5 respectively). Assume that point X in the reference frame has position (a,b,c) relative to the first position. After motion compensation, the position is corrected to (a-2,b-3,c-4), which can be considered as the position of point X relative to the second position.

[0015]

[0031] The above examples include only translations and have been described in the Cartesian domain. However, the motion parameters may include rotations. For example, assume that there are three consecutive points P1, P2, P3 captured by the laser of the LIDAR sensor in frame F1. Since the sampling rate of the LIDAR sensor laser is typically constant, the azimuth angle difference between the consecutive points of the laser is the same (e.g., delAzim). Therefore, the azimuth angle difference between P1 and P2 is the same as the azimuth angle difference between P2 and P3 (excluding noise), both of which are equal to delAzim. Consider another frame F2, where F1 is the reference frame of F2. When motion compensation is applied to the points in F1, the positions of points P1, P2, and P3 are modified. Due to the operation of motion compensation (rotation, and possibly translation), the difference in azimuth angle value between the compensated point of P1 and the compensated point of P2 may not be equal to delAzim. Similarly, the difference in azimuth angle value between the compensated point of P2 and the compensated point of P3 may not be equal to delAzim. When these compensated points are used for inter-prediction of points in the current frame F2, if successive points of the laser have an azimuth angle difference of delAzim, this will result in a suboptimal predictor, since the compensated points of P1, P2, and P3 may not correspond to any points in the current frame F2. This results in a decrease in coding efficiency.

[0016]

[0032] This disclosure describes a technique for correcting the compensated points of P1, P2, and P3 so that they are back on a regular sampling grid. This can be done by looking at the azimuth values ​​of the points in frame F1 (i.e., the zero-compensated reference frame) and deriving new position values ​​corresponding to these azimuth values ​​from the compensated positions of the points in frame F1. This results in a reference frame with points closer to the sampling grid of the sensor, and therefore a predictor closer to the points in the current frame. Note that due to the presence of noise, the point positions captured by the LIDAR sensor are typically not on a regular sampling grid. However, the sampling pattern and frequency are most often the same, or nearly the same, across frames.

[0017]

[0033] According to the techniques of this disclosure, in response to determining that a current point in a current frame is coded using an inter-prediction mode, a G-PCC coder may derive one or more motion-compensated reference points from a reference frame and selectively resample points in the reference frame based on the one or more motion-compensated reference points to generate a resampled reference frame. By inter-predicting the current point in the current frame based on the resampled reference frame, the G-PCC coder may perform more accurate inter-prediction, which reduces bits required to signal residual data, thus improving compression without increasing distortion.

[0018]

[0034] 1 is a block diagram illustrating an example encoding and decoding system 100 that may implement the techniques of this disclosure. The techniques of this disclosure generally relate to coding (encoding and / or decoding) point cloud data, i.e., supporting point cloud compression. In general, point cloud data includes any data for processing a point cloud. Coding may be useful for compressing and / or decompressing the point cloud data.

[0019]

[0035] As shown in Fig. 1, the system 100 includes a source device 102 and a destination device 116. The source device 102 provides encoded point cloud data that is decoded by the destination device 116. Specifically, in the example of Fig. 1, the source device 102 provides the point cloud data to the destination device 116 via a computer-readable medium 110. The source device 102 and the destination device 116 may include any of a wide range of devices, including desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, telephone handsets such as smartphones, televisions, cameras, display devices, digital media players, video gaming consoles, video streaming devices, terrestrial or marine vehicles, spacecraft, aircraft, robots, LIDAR devices, satellites, and the like. In some cases, the source device 102 and the destination device 116 may be capable of wireless communication.

[0020]

[0036] In the example of FIG. 1, the source device 102 includes a data source 104, a memory 106, a G-PCC encoder 200, and an output interface 108. The destination device 116 includes an input interface 122, a G-PCC decoder 300, a memory 120, and a data consumer 118. According to this disclosure, the G-PCC encoder 200 of the source device 102 and the G-PCC decoder 300 of the destination device 116 may be configured to apply techniques of this disclosure related to predictive geometry coding. Thus, the source device 102 represents an example of an encoding device, and the destination device 116 represents an example of a decoding device. In other examples, the source device 102 and the destination device 116 may include other components or arrangements. For example, the source device 102 may receive data (e.g., point cloud data) from an internal or external source. Similarly, the destination device 116 may interface with an external data consumer rather than including the data consumer within the same device.

[0021]

[0037] The system 100 as shown in FIG. 1 is merely an example. In general, other digital encoding and / or decoding devices may perform the techniques of this disclosure related to predictive geometry coding. The source device 102 and the destination device 116 are merely examples of devices in which the source device 102 generates coded data for transmission to the destination device 116. This disclosure refers to devices that perform coding (encoding and / or decoding) of data as "coding" devices. Thus, the G-PCC encoder 200 and the G-PCC decoder 300 represent examples of coding devices, specifically, encoders and decoders, respectively. In some examples, the source device 102 and the destination device 116 may operate substantially symmetrically such that each of the source device 102 and the destination device 116 includes encoding and decoding components. Thus, the system 100 may support one-way or two-way transmission between the source device 102 and the destination device 116, for example, streaming, playback, broadcast, telephony, navigation, and other applications.

[0022]

[0038] In general, the data source 104 represents a source of data (i.e., raw, unencoded point cloud data) and may provide a sequential series of "frames" of data to the G-PCC encoder 200, which encodes the data of the frames. The data source 104 of the source device 102 may include point cloud capture devices, such as any of a variety of cameras or sensors, e.g., 3D scanners or LIDAR devices, one or more video cameras, archives containing previously captured data, and / or data feed interfaces for receiving data from data content providers. Alternatively or in addition, the point cloud data may be computer-generated from scanner data, camera data, sensor data, or other data. For example, the data source 104 may generate computer graphics-based data as source data, or may generate a combination of live data, archive data, and computer-generated data. In each case, the G-PCC encoder 200 encodes the captured data, pre-captured data, or computer-generated data. The G-PCC encoder 200 may reorder the frames from a receiving order (sometimes referred to as a "display order") to a coding order for coding. The G-PCC encoder 200 may generate one or more bitstreams including the encoded data. The source device 102 may then output the encoded data onto a computer-readable medium 110 via an output interface 108 for receipt and / or retrieval by, for example, an input interface 122 of a destination device 116.

[0023]

[0039] The memory 106 of the source device 102 and the memory 120 of the destination device 116 may represent general purpose memories. In some examples, the memory 106 and the memory 120 may store raw data, e.g., raw data from the data source 104 and raw decoded data from the G-PCC decoder 300. Additionally or alternatively, the memory 106 and the memory 120 may store software instructions executable by, e.g., the G-PCC encoder 200 and the G-PCC decoder 300, respectively. Although the memory 106 and the memory 120 are shown in this example separately from the G-PCC encoder 200 and the G-PCC decoder 300, it should be understood that the G-PCC encoder 200 and the G-PCC decoder 300 may also include internal memories for functionally similar or equivalent purposes. Additionally, the memory 106 and the memory 120 may store encoded data, e.g., output from the G-PCC encoder 200 and input to the G-PCC decoder 300. In some examples, portions of memory 106 and memory 120 may be allocated as one or more buffers, e.g., for storing raw, decoded, and / or encoded data. For example, memory 106 and memory 120 may store data representing a point cloud.

[0024]

[0040] The computer-readable medium 110 may represent any type of medium or device capable of transporting encoded data from the source device 102 to the destination device 116. In one example, the computer-readable medium 110 represents a communication medium that allows the source device 102 to transmit encoded data directly to the destination device 116 in real time, for example, via a radio frequency network or a computer-based network. The output interface 108 may modulate and the input interface 122 may demodulate a received transmission signal including the encoded data according to a communication standard such as a wireless communication protocol. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful to facilitate communication from the source device 102 to the destination device 116.

[0025]

[0041] In some examples, source device 102 may output the encoded data from output interface 108 to storage device 112. Similarly, destination device 116 may access the encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, a Blu-ray disc, a DVD, a CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded data.

[0026]

[0042] In some examples, the source device 102 may output the encoded data to a file server 114 or another intermediate storage device that may store the encoded data generated by the source device 102. The destination device 116 may access the stored data from the file server 114 via streaming or download. The file server 114 may be any type of server device capable of storing the encoded data and transmitting the encoded data to the destination device 116. The file server 114 may represent a web server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. The destination device 116 may access the encoded data from the file server 114 through any standard data connection, including an Internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both suitable for accessing the encoded data stored in the file server 114. The file server 114 and the input interface 122 may be configured to operate according to a streaming transmission protocol, a download transmission protocol, or a combination thereof.

[0027]

[0043] The output interface 108 and the input interface 122 may represent a wireless transmitter / receiver, a modem, a wired network component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where the output interface 108 and the input interface 122 comprise wireless components, the output interface 108 and the input interface 122 may be configured to transfer data, such as encoded data, according to a cellular communication standard, such as 4G, 4G-LTE (Long-Term Evolution), LTE Advanced, 5G, etc. In some examples where the output interface 108 comprises a wireless transmitter, the output interface 108 and the input interface 122 may be configured to transfer data, such as encoded data, according to other wireless standards, such as the IEEE 802.11 specification, the IEEE 802.15 specification (e.g., ZigBee™), the Bluetooth™ standard, etc. In some examples, the source device 102 and / or the destination device 116 may include respective system-on-a-chip (SoC) devices. For example, the source device 102 may include a SoC device for implementing functionality attributable to the G-PCC encoder 200 and / or the output interface 108, and the destination device 116 may include a SoC device for implementing functionality attributable to the G-PCC decoder 300 and / or the input interface 122.

[0028]

[0044] The techniques of this disclosure may be applied to encoding and decoding in support of any of a variety of applications, such as communication between autonomous vehicles, communication between scanners, cameras, sensors and processing devices such as local or remote servers, geographic mapping, or other applications.

[0029]

[0045] The input interface 122 of the destination device 116 receives the encoded bitstream from the computer-readable medium 110 (e.g., a communication medium, a storage device 112, a file server 114, etc.). The encoded bitstream may include signaling information defined by the G-PCC encoder 200, such as syntax elements having values ​​that describe characteristics and / or processing of the coded units (e.g., slices, pictures, groups of pictures, sequences, etc.), that are also used by the G-PCC decoder 300. The data consumer 118 uses the decoded data. For example, the data consumer 118 may use the decoded data to determine the location of a physical object. In some examples, the data consumer 118 may include a display for displaying an image based on the point cloud.

[0030]

[0046] The G-PCC encoder 200 and the G-PCC decoder 300 may each be implemented as any of a variety of suitable encoder and / or decoder circuit configurations, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the techniques are implemented partially in software, a device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to implement the techniques of this disclosure. Each of the G-PCC encoder 200 and the G-PCC decoder 300 may be included in one or more encoders or decoders, any of which may be integrated as part of a composite encoder / decoder (codec) in the respective device. A device including G-PCC encoder 200 and / or G-PCC decoder 300 may comprise one or more integrated circuits, microprocessors, and / or other types of devices.

[0031]

[0047] The G-PCC encoder 200 and the G-PCC decoder 300 may operate according to a coding standard, such as the video point cloud compression (V-PCC) standard or the coordinate-based point cloud compression (G-PCC) standard. This disclosure may generally refer to coding (e.g., encoding and decoding) of pictures, which includes the process of encoding or decoding data. An encoded bitstream generally includes a series of values ​​of syntax elements that represent coding decisions (e.g., coding modes).

[0032]

[0048] This disclosure may generally refer to "signaling" some information, such as a syntax element. The term "signaling" may generally refer to communication of values ​​of syntax elements and / or other data used to decode encoded data. That is, the G-PCC encoder 200 may signal values ​​of syntax elements in a bitstream. In general, signaling refers to generating values ​​within the bitstream. As mentioned above, the source device 102 may transfer the bitstream to the destination device 116 in substantially real-time or non-real-time, which may occur, for example, when storing syntax elements in the storage device 112 for later retrieval by the destination device 116.

[0033]

[0049] ISO / IEC MPEG (JTC1 / SC29 / WG11) is investigating the potential need for a standard for a point cloud coding technique with compression capabilities significantly beyond those of current methods, with the goal of producing a standard. The group is working together on this quest in a collaborative effort called the 3D Graphics Team (3DG) to evaluate compression technology designs proposed by those experts in this field.

[0034]

[0050] Point cloud compression activities are categorized into two different approaches. The first approach is "Video Point Cloud Compression" (V-PCC), which segments a 3D object and projects the segments into multiple 2D planes (represented as "patches" in a 2D frame), which are further coded by a legacy 2D video codec, such as the High Efficiency Video Coding (HEVC) (ITU-T H.265) codec. The second approach is "Coordinate-Based Point Cloud Compression" (G-PCC), which directly compresses the 3D coordinates, i.e., the location of a set of points in 3D space, and the associated attribute values ​​(for each point associated with the 3D coordinates). G-PCC addresses the compression of point clouds in both category 1 (static point clouds) and category 3 (dynamically acquired point clouds). A draft G-PCC standard is available at G-PCC DIS, ISO / IEC JTC1 / SC29 / WG11 w19328, Brussels, Belgium, January 2020, and the codec description is available at G-PCC Codec Description v8, ISO / IEC JTC1 / SC29 / WG11 w19525, Brussels, Belgium, January 2020.

[0035]

[0051] A point cloud includes a set of points in 3D space and may have attributes associated with the points. The attributes may be color information such as R, G, B, or Y, Cb, Cr, or reflectance information, or other attributes. Point clouds may be captured by various cameras or sensors such as LIDAR sensors and 3D scanners, or may be computer generated. Point cloud data is used in a variety of applications including, but not limited to, architecture (modeling), graphics (3D models for visualization and animation), and the automotive industry (LIDAR sensors used to aid in navigation).

[0036]

[0052] The 3D space occupied by the point cloud data may be enclosed by a virtual bounding box. The positions of points in the bounding box may be represented with a certain precision, and thus the positions of one or more points may be quantized based on that precision. At the smallest level, the bounding box is divided into voxels, which are the smallest units of space represented by a unit cube. A voxel in a bounding box may be associated with zero, one, or more than one point. The bounding box may be divided into multiple cubic / rectangular regions, sometimes called tiles. Each tile may be coded into one or more slices. The partitioning of the bounding box into slices and tiles may be based on the number of points in each section, or other considerations (e.g., a particular region may be coded as a tile). The slice regions may be further partitioned using partitioning decisions similar to those in video codecs.

[0037]

[0053] Figure 2 provides an overview of a G-PCC encoder 200. Figure 3 provides an overview of a G-PCC decoder 300. The illustrated modules are logical and do not necessarily correspond one-to-one to the implementation code in the reference implementation of the G-PCC codec, i.e., the TMC13 test model software considered by ISO / IEC MPEG (JTC1 / SC29 / WG11).

[0038]

[0054] In both the G-PCC encoder 200 and the G-PCC decoder 300, the location of the point cloud is coded first. The attribute coding depends on the decoded geometry. In Figures 2 and 3, modules 212, 218, 310, and 314 are options typically used for category 1 data. Modules 220, 222, 316, and 318 are options typically used for category 3 data. All other modules are common between category 1 and category 3.

[0039]

[0055] For category 3 data, the compressed geometry is typically represented as an octree from the root to the leaf level of the individual voxels. For category 1 data, the compressed geometry is typically represented by a pruned octree (i.e., an octree from the root to the leaf level of blocks larger than a voxel) plus a model that approximates the surface in each leaf of the pruned octree. In this way, both category 1 and category 3 data share the octree coding mechanism, while category 1 data may additionally approximate the voxels in each leaf with a surface model. The surface model used is a triangulation with 1 to 10 triangles per block, resulting in a triangle soup. Thus, category 1 geometry codecs are known as Trisoup geometry codecs, and category 3 geometry codecs are known as Octree geometry codecs.

[0040]

[0056] FIG. 4 is a conceptual diagram illustrating an example octree partitioning for geometry coding according to the techniques of this disclosure. In the example shown in FIG. 4, the octree 400 may be partitioned into a series of nodes. For example, each node may be a cubic node. At each node of the octree, when the occupancy is not inferred by the G-PCC decoder 300 for one or more of the node's child nodes, which may include up to eight nodes, the G-PCC encoder 200 may signal the occupancy of the node by the points of the point cloud to the G-PCC decoder 300. A number of neighboring nodes are specified, including (a) nodes that share a face with the current octree node, (b) nodes that share a face, edge, or vertex with the current octree node, etc. Within each neighboring node, the occupancy of the node and / or its child nodes may be used to predict the occupancy of the current node or its children. For sparsely populated points at a particular node of the octree, the codec also supports a direct coding mode, in which the 3D location of the point is directly encoded. A flag may be signaled to indicate that direct mode is signaled. In direct mode, the position of a point in the point cloud may be directly coded without any compression. At the lowest level, the number of the point associated with an octree node / leaf node may also be coded.

[0041]

[0057] When geometry is coded, attributes corresponding to the geometry points are coded. When there are multiple attribute points corresponding to one reconstructed / decoded geometry point, attribute values ​​representing the reconstructed points can be derived.

[0042]

[0058] In G-PCC, there are three attribute coding processes: Region Adaptive Hierarchical Transform (RAHT) coding, Interpolation-based Hierarchical Nearest Neighbor Prediction (Predictive Transform), and Interpolation-based Hierarchical Nearest Neighbor Prediction with an Update / Lifting Step (Lifting Transform). RAHT and the Lifting Transform are typically used for category 1 data, and the Predictive Transform is typically used for category 3 data. However, either process may be used for any data, and the attribute coding process used to code the point cloud is specified in the bitstream, just as in the case of the geometry codec in G-PCC.

[0043]

[0059] The coding of attributes can be done in levels of detail (LOD), where each level of detail can be used to obtain a finer representation of the point cloud attributes, which can be specified based on a distance metric from neighboring nodes or based on a sampling distance.

[0044]

[0060] In the G-PCC encoder 200, the residual obtained as the output of the coding method for the attribute is quantized (e.g., by one of the arithmetic coding units 214 and / or 226). The quantized residual may be coded using context-adaptive arithmetic coding.

[0045]

[0061] In the example of FIG. 2, the G-PCC encoder 200 may include a coordinate transformation unit 202, a color transformation unit 204, a voxelization unit 206, an attribute transfer unit 208, an octree analysis unit 210, a surface approximation analysis unit 212, an arithmetic coding unit 214, a geometry reconstruction unit (GRU) 216, a RAHT unit 218, an LOD generation unit 220, a lifting unit 222, a coefficient quantization unit 224, and an arithmetic coding unit 226.

[0046]

[0062] As shown in the example of Figure 2, the G-PCC encoder 200 may receive a set of locations and a set of attributes. The locations may include coordinates of points in the point cloud. The attributes may include information about the points in the point cloud, such as a color associated with the points in the point cloud.

[0047]

[0063] The coordinate transformation unit 202 may apply a transformation to the coordinates of the points to convert the coordinates from an initial domain to a transformation domain. In this disclosure, the transformed coordinates may be referred to as transformed coordinates. The color transformation unit 204 may apply a transformation to convert color information of the attributes to a different domain. For example, the color transformation unit 204 may convert color information from an RGB color space to a YCbCr color space.

[0048]

[0064] Further, in the example of FIG. 2, the voxelization unit 206 may voxelize the transformed coordinates. The voxelization of the transformed coordinates may include quantization and removing some points of the point cloud. In other words, multiple points of the point cloud may be contained within a single "voxel" and then treated as one point in some agar. Further, the octree analysis unit 210 may generate an octree based on the voxelized transformed coordinates. Additionally, in the example of FIG. 2, the surface approximation analysis unit 212 may analyze the points to determine a surface representation of the set of points. The arithmetic coding unit 214 may entropy code syntax elements representing surface information determined by the octree and / or surface approximation analysis unit 212. The G-PCC encoder 200 may output these syntax elements in a geometry bitstream.

[0049]

[0065] The geometric reconstruction unit 216 may reconstruct transformation coordinates of points in the point cloud based on the octree, the data indicative of the surface determined by the surface approximation analysis unit 212, and / or other information. The number of transformation coordinates reconstructed by the geometric reconstruction unit 216 may differ from the original number of points in the point cloud due to voxelization and surface approximation. In this disclosure, the resulting points may be referred to as reconstructed points. The attribute transfer unit 208 may transfer attributes of the original points of the point cloud to the reconstructed points of the point cloud. As shown in FIG. 2, the attribute transfer unit 208 may transfer attributes to one or both of the RAHT unit 218 and the LOD generation unit 220.

[0050]

[0066] Further, the RAHT unit 218 may apply RAHT coding to the attributes of the reconstructed points. Alternatively or in addition, the LOD generation unit 220 and the lifting unit 222 may apply LOD processing and lifting, respectively, to the attributes of the reconstructed points. The RAHT unit 218 and the lifting unit 222 may generate coefficients based on the attributes. The coefficient quantization unit 224 may quantize the coefficients generated by the RAHT unit 218 or the lifting unit 222. The arithmetic coding unit 226 may apply arithmetic coding to syntax elements representing the quantized coefficients. The G-PCC encoder 200 may output these syntax elements in an attribute bitstream.

[0051]

[0067] In the example of FIG. 3, the G-PCC decoder 300 may include a geometry arithmetic decoding unit 302, an attribute arithmetic decoding unit 304, an octree synthesis unit 306, an inverse quantization unit 308, a surface approximation synthesis unit 310, a geometric reconstruction unit 312, a RAHT unit 314, a LoD generation unit 316, an inverse lifting unit 318, an inverse coordinate transformation unit 320, and an inverse color transformation unit 322.

[0052]

[0068] The G-PCC decoder 300 may obtain a geometry bitstream and an attribute bitstream. A geometry arithmetic decoding unit 302 of the decoder 300 may apply arithmetic decoding (e.g., context-adaptive binary arithmetic coding (CABAC) or other types of arithmetic decoding) to syntax elements in the geometry bitstream. Similarly, an attribute arithmetic decoding unit 304 may apply arithmetic decoding to syntax elements in the attribute bitstream.

[0053]

[0069] The octree synthesis unit 306 may synthesize an octree based on syntax elements parsed from the geometry bitstream. In cases where surface approximation is used in the geometry bitstream, the surface approximation synthesis unit 310 may determine a surface model based on the syntax elements parsed from the geometry bitstream and based on the octree.

[0054]

[0070] Further, the geometric reconstruction unit 312 may perform a reconstruction to determine coordinates of points in the point cloud. The inverse coordinate transformation unit 320 may apply an inverse transformation to the reconstructed coordinates to reverse the reconstructed coordinates (locations) of the points in the point cloud from the transformed domain back to the original domain.

[0055]

[0071] In addition, in the example of Figure 3, the inverse quantization unit 308 may inverse quantize the attribute values. The attribute values ​​may be based on syntax elements obtained from the attribute bitstream (e.g., including syntax elements decoded by the attribute arithmetic decoding unit 304). As shown in Figure 3, the inverse quantization unit 308 may forward the attribute values ​​to one or both of the RAHT unit 314 and the LOD generation unit 316.

[0056]

[0072] Depending on how the attribute values ​​are encoded, the RAHT unit 314 may perform RAHT coding to determine color values ​​for the points of the point cloud based on the dequantized attribute values. Alternatively, the LOD generation unit 316 and the inverse lifting unit 318 may use level-of-detail based techniques to determine color values ​​for the points of the point cloud.

[0057]

[0073] 3, the inverse color transformation unit 322 may apply an inverse color transformation to the color values. The inverse color transformation may be the inverse of the color transformation applied by the color transformation unit 204 of the G-PCC encoder 200. For example, the color transformation unit 204 may convert the color information from the RGB color space to the YCbCr color space. In response, the inverse color transformation unit 322 may convert the color information from the YCbCr color space to the RGB color space.

[0058]

[0074] To aid in understanding the operations performed by the G-PCC encoder 200 and the G-PCC decoder 300, various units in FIGS. 2 and 3 are illustrated. The units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. A fixed-function circuit refers to a circuit that provides a specific function and is predefined in the operations that may be performed. A programmable circuit refers to a circuit that may be programmed to perform various tasks and provides flexible functionality in the operations that may be performed. For example, a programmable circuit may execute software or firmware that causes the programmable circuit to operate in a manner defined by the software or firmware instructions. Although a fixed-function circuit may execute software instructions (e.g., receive parameters or output parameters), the type of operations that the fixed-function circuit performs is generally invariant. In some examples, one or more of the units may be different circuit blocks (fixed function or programmable), and in some examples, one or more of the units may be an integrated circuit.

[0059]

[0075] Predictive geometry coding is introduced as an alternative to octree geometry coding, where nodes are arranged in a tree structure (defining the predictive structure) and various prediction strategies are used to predict the coordinates of each node in the tree associated with its predictor. Figure 6 shows a predictive tree 600, an example of a directed graph, conceptually with arrows pointing in the prediction direction. Nodes shaded with horizontal lines are root vertices and have no predictors, nodes shaded with grid lines have two children, nodes shaded with diagonal lines have three children, nodes without shading have one child, and nodes shaded with vertical lines are leaf nodes and have no children. Each node has only one parent node.

[0060]

[0076] Four prediction strategies can be specified for each node based on its parent (p0), grandfather (p1) and great-grandfather (p2). The prediction strategies are no prediction (no prediction), delta prediction (p0), linear prediction (2 * p0-p1), and parallelogram prediction (p0+p1-p2).

[0061]

[0077] The encoder (e.g., G-PCC encoder 200) can employ any algorithm to generate the predictive tree, and the algorithm used can be determined based on the application / use case, and several strategies can be used. For each node, the encoder may encode the residual coordinate value in a bitstream starting from the root node in a depth-first manner. Predictive geometry coding can be particularly useful for category 3 (LIDAR-acquired) point cloud data, e.g., for low latency applications.

[0062]

[0078] In predictive geometry coding, an angular mode can be used, in which case the characteristics of the LIDAR sensor can be exploited in coding the prediction tree more efficiently. The position coordinates are converted to (r,φ,i) (radius, azimuth and laser index) and the prediction is performed in this domain (residuals are coded in the r,φ,i domain). Due to errors in the rounding process, coding in r,φ,i is not lossless, so a second set of residuals can be coded, corresponding to Cartesian coordinates. The description of the encoding and decoding strategy used for the angular mode for predictive geometry coding is reproduced below. The description is based on Figs. 5A and 5B, which are conceptual diagrams of a rotating LIDAR acquisition model.

[0063]

[0079] The techniques of this disclosure may be applicable at least to point clouds acquired using a rotating Lidar model, where the Lidar 502 has N lasers (e.g., N=16, 32, 64) that rotate around the Z axis according to an azimuth angle φ (see FIGS. 5A and 5B). Each laser is rotated at a different elevation angle θ(i). i=1...N and height σ(i) i=1...N Laser i strikes a point M with Cartesian integer coordinates (x, y, z) defined according to coordinate system 500 illustrated in FIG.

[0064]

[0080] The position of M can be modeled with three parameters (r, φ, i) and calculated as follows:

[0065]

number

[0066]

[0081] The coding process is

[0067]

number

[0068] One may use a quantized version of (r,φ,i), denoted by

[0069]

number

[0070] and i may be calculated as follows:

[0071]

number

[0072] however, (q r ,o r ) and (q φ ,o φ ) are, respectively,

[0073]

number

[0074] is the quantization parameter that controls the precision of sign(t) is a function that returns 1 if t is positive, and (-1) otherwise. |t| is the absolute value of t.

[0075]

[0082] To avoid reconstruction inconsistencies due to the use of floating-point arithmetic, σ(i) i=1...N and tan(θ(i)) i=1...N The values ​​of may be pre-computed and quantized as follows:

[0076]

number

[0077] however, (q σ ,o σ ) and (q θ ,o θ ) are, respectively,

[0078]

number

[0079] is the quantization parameter that controls the precision of

[0080] The reconstructed Cartesian coordinates are obtained as follows:

[0081]

number

[0082] In this case, app_cos(.) and app_sin(.) are approximations of cos(.) and sin(.). The calculations can use fixed point representations, lookup tables, and linear interpolation.

[0083] In some examples,

[0084]

number

[0085] can differ from (x,y,z) for a variety of reasons, including: quantization Approximation Model inaccuracies Inaccuracies in model parameters The coordinate system may differ from (x,y,z) for various reasons, including:

[0086] In some examples, the reconstruction residual (r x ,r y ,r z ) can be defined as follows:

[0087]

number

[0088]

[0085] In this process, the G-PCC encoder 200 may proceed as follows. Model parameters

[0089]

number

[0090] and the quantization parameter q r , q σ , q θ , and q φ Encode the following. Represents the geometry prediction scheme described in G-PCC DIS.

[0091]

number

[0092] Apply to New predictors can be introduced that exploit the properties of the lidar. For example, the rotational speed of the lidar scanner around the z-axis is usually constant. Therefore, the G-PCC decoder can use the current

[0093]

number

[0094] can be predicted as follows:

[0095]

number

[0096] in this case, (δ φ (k) k=1...K is the set of possible rates from which the encoder can choose. The index k may be explicitly written into the bitstream or may be inferred from the context based on a deterministic strategy applied by both the encoder and the decoder. n(j) is the number of skipped points and can be either explicitly written into the bitstream or inferred from the context based on a deterministic strategy applied by both the encoder and the decoder. At each node, the reconstruction residual (r x ,r y ,r z ) is encoded.

[0097] The G-PCC decoder 300 may proceed as follows: Model parameters

[0098]

number

[0099] and the quantization parameter q r , q σ , q θ , and q φ Decrypt the According to the geometry prediction method described in the G-PCC Draft International Standard (DIS),

[0100]

number

[0101] Decode the parameters. As mentioned above, the reconstructed coordinates

[0102]

number

[0103] Calculate. Residual (r x ,r y ,r z ) is decrypted. As discussed in the next section, the reconstruction residual (r x ,r y,r z ), it is possible to support lossy compression. Calculate the original coordinates (x,y,z) as follows:

[0104]

number

[0105]

[0087] Lossy compression is a method for compressing the reconstruction residual (r x ,r y ,r z This can be achieved by applying quantization to , or by deleting points.

[0106]

[0088] The quantized reconstruction residual is calculated as follows:

[0107]

number

[0108] In this case, (q x ,o x ), (q y ,o y ), and (q z ,o z ) are, respectively,

[0109]

number

[0110] is the quantization parameter that controls the precision of

[0111] In some examples, the G-PCC encoder 200 and / or the G-PCC decoder 300 may use trellis quantization to further improve the RD (rate-distortion) performance results. The quantization parameters may vary at the sequence / frame / slice / block level to achieve region adaptive quality and for rate control purposes.

[0112]

[0090] A G-PCC coder (e.g., the G-PCC encoder 200 and / or the G-PCC decoder 300) may use inter prediction to perform point cloud compression. By using inter prediction, the G-PCC coder may use point redundancy across frames to provide additional bitrate savings. The embodiments of the various aspects of the present disclosure may be used individually or in any combination.

[0113]

[0091] Although this description is primarily directed to a polar coordinate system, the process disclosed in this application may also be applied to other coordinate systems, such as a Cartesian coordinate system, a spherical coordinate system, or any custom coordinate system that may be used to represent / code point cloud positions and attributes.

[0114]

[0092] The G-PCC coder may determine whether to code a point using inter prediction or intra prediction. For example, the G-PCC encoder may perform an analysis to determine whether it would be beneficial (e.g., in terms of bit rate or other conditions) to code a particular point using inter prediction or intra prediction. In some examples, the G-PCC coder may perform this determination in one of the following ways: In some examples, the G-PCC encoder 200 may use a first set of conditions during encoding, and the G-PCC decoder 300 may use the first set of conditions during decoding. In some examples, the G-PCC encoder 200 may use the first set of conditions during encoding and signal a syntax element indicating an inter prediction mode, and the G-PCC decoder 300 may determine the mode based on the signaled syntax element.

[0115]

[0093] The first set of conditions may be derived using one or more characteristics of the point, such as Cartesian position, angular coordinates (radius, azimuth, elevation), prediction mode of neighboring nodes, etc. Some examples of conditions included in the first set of conditions are as follows: In the first exemplary condition, the point belongs to a frame that is not an intra-coded frame or does not correspond to a random access point. In the second exemplary condition, the azimuth angle of the current point is different from the azimuth angle of a point decoded before the current frame, or the delta azimuth angle between the current point and the previous point is non-zero. In some examples, an approximation of the delta azimuth angle, or a quantized version of the delta azimuth angle, may be used to determine the inter-prediction mode of the current node. In the third exemplary condition, the point belongs to a slice that is a certain type (i.e., P slice or B slice indicating inter-prediction may be applied). In the fourth exemplary condition, a flag is signaled to indicate whether inter-prediction is enabled for a certain frame. This flag may be signaled in a syntax element associated with the frame (e.g., slice / slice header, frame header, etc.) or in a parameter set that the frame references. For example, inter_prediction_enabled_flag may be signaled in a Geometry Parameter Set (GPS) to specify whether inter prediction is enabled for a point cloud frame / slice that references the GPS. When inter_prediction_enabled_flag indicates that inter prediction is not enabled, inter prediction indications for points in the point cloud may not be signaled.

[0116]

[0094] In some examples, the inter prediction mode for a point may be determined based on an azimuth angle residual, or a phi multiplier, associated with the point.

[0117]

[0095] In some examples, inter prediction (or one or more inter prediction modes) may be enabled only when the angular mode is enabled (i.e., inter prediction may be disabled if the angular mode is not enabled).

[0118]

[0096] The G-PCC coder may select a reference frame. In some examples, the G-PCC coder may use a previously decoded frame (or possibly a preceding frame in the decoding order) as a reference frame. In other examples, a frame number indication (using an LSB value or a delta frame number value) may be used to specify the reference frame. More generally, two or more frames may be designated as reference frames, and a point may be coded using inter prediction from any of the reference frames (an indication of the reference frame associated with the point may be signaled or derived).

[0119] In another example, prediction may be performed from two or more frames (e.g., bi-prediction). Thus, the G-PCC coder may predict a point of a current frame based on a reference point in a first reference frame and a reference point in a second reference frame.

[0120]

[0098] The G-PCC coder may utilize multiple inter-prediction modes. When a point is inter-predicted, there may be one or more ways to predict the point from a reference frame. A different mode value may be used to specify each type of prediction (e.g., the G-PCC coder may signal a syntax element indicating the inter-prediction mode used for the current point). As one exemplary inter-prediction mode, the point may be predicted from a zero-motion candidate from a reference frame (e.g., a reference point in a reference frame may be a zero-motion candidate). As another exemplary inter-prediction mode, the point may be predicted from a global motion candidate from a reference frame. As another exemplary inter-prediction mode, the point may be predicted from a candidate point from a reference frame, and other parameters (e.g., a motion vector, etc.) may be used to specify the candidate point.

[0121]

[0099] Some details about exemplary different types of inter prediction are provided below. The G-PCC coder may utilize a partial inter prediction mode. In one example, inter prediction may be applied to predict only a subset of the characteristics of a point in a point cloud. One or more of the characteristics for which intra prediction is not applied may be coded using intra prediction or by other techniques. In one example, when a point is indicated to be inter predicted, the radius of the point may be predicted from a reference point (or reference frame) when an angle mode is enabled, and the azimuth angle and laser ID (or elevation angle) may be derived based on intra prediction or other techniques. In another example, when a point is indicated to be inter predicted, both the radius and azimuth angle of the point may be predicted from a reference point (or reference frame) when an angle mode is enabled, and the G-PCC coder may derive the laser ID (or elevation angle) based on intra prediction or other techniques.

[0122]

[0100] The G-PCC coder may perform inter-prediction mode signaling. In some examples, the inter-prediction mode associated with a point may be signaled in the bitstream. This may be signaled as a flag or as a mode value. In one exemplary inter-prediction mode signaling technique, an inter-pred-flag equal to 0 may indicate that the point is predicted using intra prediction, and an inter-pred-flag equal to 1 may indicate that the point is predicted using inter prediction. In some examples, based on the indication that the point is predicted using inter prediction, a mode value may be signaled to specify a process for performing inter prediction of the point. In another exemplary inter-prediction mode signaling technique, when a point is predicted using inter prediction, a mode value may be signaled, where a mode value of 0 specifies that zero motion candidates are used for prediction, and a mode value of 1 specifies that global motion candidates are used for prediction.

[0123]

[0101] In this disclosure, inter prediction mode may be used interchangeably with an indication of whether a point is coded using inter prediction, and using the particular inter prediction technique used.

[0124]

[0102] Some conditions may affect how the G-PCC coder signals the inter-prediction mode. In some cases, the inter-prediction mode may be signaled only when a second set of conditions applies. The second set of conditions may be derived using one or more characteristics of the point, such as the Cartesian position, the angular coordinates (radius, azimuth, elevation), the prediction mode of a neighboring node, etc. Exemplary conditions included in the second set of conditions may include that the point belongs to a frame that is not an intra-coded frame, or does not correspond to a random access point. Exemplary conditions included in the second set of conditions may include that the azimuth angle of the current point is different from the azimuth angle of a previously decoded point, or that the delta azimuth angle between the current point and the previous point is non-zero. In some examples, an approximation of the delta azimuth angle, or a quantized version of the delta azimuth angle, may be used to determine the inter-prediction mode of the current node. Exemplary conditions included in the second set of conditions may include that the delta azimuth angle between the current point and the previous point is greater than a threshold value. The value of the threshold may be fixed or signaled in the bitstream, or may be derived based on syntax elements signaled in the bitstream.

[0125]

[0103] More generally, the second set of conditions may include one or more criteria, including but not limited to a delta azimuth value, a delta radius value, a delta laserID value, and a delta elevation value. This delta value of the characteristic may be indicated by a corresponding residual value signaled in the bitstream for the point. The second set of conditions may also include delta coordinate values ​​in the Cartesian domain (delta x, delta y, delta z).

[0126] When an inter-prediction mode is not signaled for a particular point, the inter-prediction mode may be inferred to be a fixed value for the point. When an inter-prediction mode is not signaled for a point, it is assumed that the value may be 0 (or a value that specifies intra-prediction).

[0127]

[0105] Next, signaling optimization is described. In some cases, the inter prediction mode may not be signaled for all points, and instead, the inter prediction mode may be derived from a syntax element signaled in the bitstream. The inter prediction mode may be signaled for a prediction tree, and the inter prediction mode may be applied to all points in the prediction tree. The inter prediction mode may be signaled in a geometry slice, and the inter prediction mode may be applied to all points in the slice. The inter prediction mode may be signaled for a prediction block (specified as a fixed number of points, a signaled number of points, etc.), and the inter prediction mode may be applied to all points in the prediction block.

[0128] In some examples, the inter prediction mode may be signaled only for some nodes specified by the tree depth. For example, the inter prediction mode may be signaled only for the root node of the prediction tree.

[0129]

[0107] In one example, when inter prediction is applied for some characteristics of a point (e.g., radius) and intra prediction is applied for other characteristics of a point (e.g., azimuth angle, laserID), only a subset of intra prediction modes may be enabled (e.g., only mode 0 of intra prediction may be enabled for azimuth angle, laserID). In some examples, if only one intra prediction mode is enabled, the mode value is not signaled and is inferred at the decoder. In some examples, if a subset (two or more) of intra prediction modes is enabled, a more optimized coding of the mode syntax element may be applied (e.g., fewer bits may be sufficient to code an index to the subset of modes).

[0130]

[0108] For each point and corresponding to each prediction mode, the G-PCC coder may derive a predicted value of one or more characteristics of the current point based on the position of one or more points in a reference frame. An exemplary prediction process is as follows: 1a. When a zero motion candidate is selected, a prediction candidate may be selected as the point with the closest azimuth and laser ID that corresponds to a particular value of azimuth and laser ID (e.g., the reconstructed azimuth value and laserID of the current point), and the radius value of the prediction candidate is selected as the predicted value of the radius of the current node. 1b. When a zero motion candidate is selected, the prediction candidate may be selected as the point with the closest azimuth and laser ID corresponding to a particular value of azimuth and laser ID (e.g., the reconstructed azimuth value and laserID of a previously decoded and reconstructed point), and the radius, azimuth and laserID values ​​of the prediction candidate are selected as the predicted values ​​of radius, azimuth and laserID of the current node. 2a. When a global motion candidate is selected, a prediction candidate may be selected as the point with the closest azimuth angle and laser ID corresponding to a particular value of azimuth angle and laser ID after application of global motion compensation to the prediction candidate, and the radius value of the prediction candidate is selected as the predicted value of the radius of the current node. 2b. When a global motion candidate is selected, the prediction candidate after application of global motion compensation to the prediction candidate may be selected as the point with the closest azimuth angle and laser ID corresponding to a particular value of azimuth angle and laser ID (e.g., the reconstructed azimuth angle value and laserID of a previously decoded and reconstructed point), and the radius, azimuth angle and laserID values ​​of the prediction candidate are selected as the predicted values ​​of the radius, azimuth angle and laserID of the current node. 3a. When a generic candidate is selected, the predicted candidate may be selected as the point with the closest azimuth and laser ID that corresponds to a particular value of azimuth and laser ID after application of a motion vector (associated with the current point) to the predicted candidate, and the radius value of the predicted candidate is selected as the predicted value of the radius of the current node. 3b. When a generic candidate is selected, the prediction candidate after application of a motion vector to the prediction candidate may be selected as the point with the closest azimuth and laser ID corresponding to a particular value of azimuth and laser ID (e.g., the reconstructed azimuth value and laserID of a previously decoded and reconstructed point), and the radius, azimuth and laserID values ​​of the prediction candidate are selected as the predicted values ​​of radius, azimuth and laserID of the current node.

[0131] In some examples, the motion vector may be used to derive the radius, azimuth, and laser ID (or elevation) of the current point, or a point that is used to derive a prediction of these values. In some examples, the motion vector may be used to derive a prediction of the x, y, z coordinates, or a prediction of these values.

[0132]

[0110] In some examples, two or more points in the reference frame may be used to derive a prediction value. The number of points may be signaled in the slice header or in the GPS. In some examples, an index to two or more points in the reference frame may be signaled to specify a particular point to be used for inter prediction. When a zero motion candidate is selected, the two prediction candidates are selected as the two points with the two closest azimuth angles and laser IDs corresponding to particular values ​​of azimuth angle and laser ID, and the radius values ​​of the two prediction candidates are selected as the prediction value of the radius of the current node. When two or more prediction candidates are selected as candidates for inter prediction (e.g., based on the closeness of laser IDs and / or azimuth angles), the prediction candidates may be selected by signaling an index to a list comprising two or more prediction candidates. In some examples, a weighted average (weights calculated based on the difference in the laser ID values ​​of the current point and the predictor, and / or the difference between the azimuth angle of the current point and the predictor) may also be used as a prediction candidate.

[0133]

[0111] The G-PCC coder may perform reference frame preparation. In some examples, the reference frame may be motion compensated before inter prediction. For example, one or more points of the reference frame may be associated with a motion (e.g., global motion), and a prediction for a subsequent frame may be obtained by compensating the position of the point with the associated motion. This motion may be estimated or obtained from external information (such as GPS information). Thus, the "reference frame" used for inter prediction may be a "compensated reference frame". In other words, the reference frame may be a motion compensated reference frame.

[0134] In some examples, one or more points in the compensated reference frame may be considered as zero motion candidates for the current frame. In some examples, the points in the reference frame for which motion compensation is applied may belong to a particular feature (e.g., object) in the point cloud or may be labeled (e.g., ground, non-ground) by some estimation algorithm. Global motion parameters (e.g., rotation, translation) may be signaled in the bitstream.

[0135] In some examples, in addition to the global motion parameters, additional adjustment values ​​may be signaled for one or more of the following: x, y, z, radius, azimuth, laserID. For example, when a predictor frame is prepared by applying motion compensation (e.g., using the motion parameters), one or more additional adjustment values ​​of x, y, z may be applied in the Cartesian domain, and one or more additional adjustment values ​​of radius, azimuth, laserID may be applied in the spherical domain.

[0136]

[0114] In some examples, adjustment values ​​may be specified for the entire sequence (representing a "global" adjustment), and these values ​​may also be incorporated into the global motion parameters.

[0137]

[0115] In some examples, adjustment values ​​may be specified for groups of frames (representing a "group" adjustment), and these values ​​may also be incorporated into the global motion parameters.

[0138]

[0116] In some examples, adjustment values ​​may be specified for a frame (representing a "frame" adjustment), and these values ​​may also be incorporated into the global motion parameters.

[0139]

[0117] In some examples, adjustment values ​​may be specified for different regions of the point cloud (representing "local" adjustments). These values ​​may also be incorporated into the global motion parameters or may be signaled as modifications to the global motion applicable to the frame.

[0140]

[0118] One or more of the global, frame, and local adjustments may be signaled, and when more than one of these parameters are present, the corresponding adjustments may be applied sequentially or together (e.g., global, frame, and local x-adjustments xg, xf, xl may be applied as xg+xf+xl, respectively, to points belonging to a region). In other examples, all adjustments at one level (e.g., global) may be applied before applying adjustments at a different level (e.g., frame).

[0141]

[0119] The motion vector associated with the inter prediction mode of a point may be signaled in the bitstream. In some cases, multiple points may share the same motion vector, which may be signaled in the slice or in a parameter set (e.g., GPS, or a parameter set used specifically for motion parameters). In some cases, the motion vector associated with a particular point may be predicted from spatially or temporally inter predicted neighboring points, and only the difference between the actual motion vector and the predicted motion vector may be signaled.

[0142]

[0120] The G-PCC coder may perform context selection. For example, the G-PCC coder may determine the context to be used for the predicted geometry parameters / syntax elements based on one or more syntax elements signaled in the bitstream, or their derived variables. The use of additional contexts may improve the compression efficiency of the codec.

[0143] In some examples, the G-PCC coder may utilize the first parameter associated with the point to determine a context for other syntax elements associated with the point. For example, the context of one or more syntax elements associated with the phi multiplier (e.g., ptn_phi_mult_abs_gt0_flag, ptn_phi_mult_sign_flag, ptn_phi_mult_abs_gt1_flag, ptn_phi_mult_abs_minus2, ptn_phi_mult_abs_minus9) may be selected based on the inter flag.

[0144] In one example, the inter flag indicates whether the first set of contexts is selected if inter prediction is used, or whether the second set of contexts is selected if intra prediction is used. This can be applied to the coding of the phi multiplier, first and second order residuals in case of prediction geometry with angle mode enabled (spherical position coding: azimuth, radius, elevation, or laserID).

[0145]

[0123] When the context for a syntax element depends on a first set of conditions, one context may be selected when the first set of conditions are true and a first parameter associated with the point takes a first value, and a second context may be selected when the first set of conditions are true and a first parameter associated with the point takes a second value that is different from the first value.

[0146]

[0124] Different choices of the first parameter are described. Each choice may result in a different trade-off between compression efficiency and storage (more context may require more storage associated with the context state). In one example, the first parameter may be an inter flag associated with the point. In another example, the first parameter may be a prediction mode associated with the point (e.g., a prediction mode used in the predicted geometry). In a further example, the first parameter may be a tuple of an inter flag and a prediction mode associated with the point, e.g., (interFlag, predMode). When there are more than one inter prediction candidates (e.g., zero motion candidate, global motion candidate, etc.), let interPredMode represent the various inter prediction candidates (e.g., 0 for intra, 1 for zero motion candidate, 2 for global motion candidate, etc.). The first parameter may be interPredMode.

[0147]

[0125] In some examples, the G-PCC coder may use different conditions / parameters for different types of frames. For example, for frames that are intra-coded (have no reference to other frames in the sequence for prediction), a second parameter may be selected to determine the context as described in this disclosure, and for other frames (e.g., inter-coded frames), a third parameter may be selected to determine the context as described in this disclosure. The second and third parameters may be different (e.g., for intra frames, a prediction mode may be selected as the second parameter, and for inter frames, a tuple of prediction mode and inter flag may be selected as the third parameter). The G-PCC coder may extend similar context selection to one or more syntax elements / components (first order residual, second order residual, etc.).

[0148]

[0126] Figure 7 is a conceptual diagram illustrating an example inter-prediction process for predicting a point of a point cloud according to one or more aspects of the present disclosure. As shown in Figure 7, a current frame 750 may include a number of points 752A-752L (collectively, "points 752"), and a reference frame 754 may include a number of points 756A-756L (collectively, "points 756"). The reference frame 754 may be a frame that is coded and / or reconstructed before the current frame 750 is decoded and / or reconstructed (e.g., the reference frame 754 may precede the current frame 750 in coding order). The G-PCC coder may utilize intra prediction to predict one or more of the points 752 of the current frame 750 based on one or more of the points 756 of the reference frame 754. For example, the G-PCC decoder (or the reconstruction loop of the G-PCC encoder) may predict one or more parameters (e.g., (r, φ, i)) of a current point 752A of the points 752 based on one or more of the points 756.

[0149]

[0127] To perform inter prediction to predict a current point in a current frame, the G-PCC coder may determine a reference point in a reference frame different from the current frame, and predict one or more parameters of the current point based on the reference point. For example, to predict a current point 752A, the G-PCC coder may determine a reference point 756A, and predict one or more parameters of the current point 752A based on one or more parameters of the reference point 756A. The determined reference point may be referred to as an identified reference point.

[0150]

[0128] The G-PCC coder may determine the reference point using any suitable technique. As an example, the G-PCC coder may determine a pivot point in the current frame that precedes the current point in the coding order, and determine the reference point based on one or more parameters of the pivot point. For example, if the coding order is counterclockwise, the G-PCC coder may determine that point 752B is a point before the current point 752A (e.g., a point that comes immediately before the current point in the coding order) (i.e., determine that point 752B is a pivot point), and determine the reference point based on one or more parameters of the pivot point 752B.

[0151]

[0129] To determine a reference point based on one or more parameters of the pivot point, the G-PCC coder may determine a reference pivot point based on the azimuth angle of the pivot point in the reference frame, and may determine a reference point based on the reference pivot point. For example, the G-PCC coder may determine a point in the reference frame 754 that has the same azimuth angle (or the same azimuth angle and the same laser ID) as the pivot point 752B. In the example of FIG. 7, the G-PCC coder may determine that the point 756B is the reference pivot point because the point 756B has the same azimuth angle as the pivot point 752B. Although the reference pivot point in the example of FIG. 7 corresponds to an actual point (e.g., an actual point in the frame 754), the techniques of this disclosure are not necessarily so limited. For example, in some examples, the reference pivot point may be a virtual point that does not correspond to a reconstructed point in the reference frame 754.

[0152]

[0130] In some examples, the G-PCC coder may determine the reference pivot point based on the actual (e.g., unscaled) azimuth angle of the pivot point. In other examples, the G-PCC coder may determine the reference pivot point based on the scaled azimuth angle of the pivot point. For example, the G-PCC coder may determine the scaled azimuth angle by scaling the azimuth angle of the pivot point by a constant value.

[0153]

[0131] To determine a reference point based on a reference pivot point, the G-PCC coder may identify a point in a reference frame that has an azimuth angle greater than that of the reference pivot point as a reference point. For example, the G-PCC coder may determine which of the points 756 have an azimuth angle value greater than that of the reference pivot point, and select a point with a minimum azimuth angle value (from the set of points 756 that have an azimuth angle value greater than that of the reference pivot point). In this example of FIG. 7, the point 756A may be a point in the reference frame 754 that has a minimum azimuth angle greater than that of the reference pivot point 765B. Thus, the G-PCC coder may identify the point 756A as a reference point for performing intra prediction of the current point 752A.

[0154]

[0132] In some examples, the G-PCC coder may determine the reference point based on the actual (e.g., unscaled) azimuth angle of the reference pivot point. In other examples, the G-PCC coder may determine the reference point based on the scaled azimuth angle of the reference pivot point. For example, the G-PCC coder may determine the scaled azimuth angle of the reference pivot point by scaling the azimuth angle of the pivot point by a constant value. Thus, in some examples, the G-PCC coder may determine the reference point by identifying as the reference point a point (e.g., point 756A) having the smallest scaled azimuth angle greater than the scaled azimuth angle of the reference pivot point. In some examples, the G-PCC coder may utilize the point with the second smallest azimuth angle greater than the scaled azimuth angle. For example, the G-PCC coder may determine the reference point by identifying as the reference point the point having the second smallest scaled azimuth angle greater than the scaled azimuth angle of the reference pivot point (e.g., point 756L).

[0155]

[0133] The G-PCC coder may predict parameters of the current point 752A based on parameters of the reference point 756A. For example, the G-PCC coder may signal residual data representing a difference between parameters of the current point 752A and the reference point 756A. The G-PCC decoder may add the residual data to the parameters of the reference point 756A to reconstruct the parameters of the current point 752A.

[0156]

[0134] Although described above as using a single reference point in a single reference frame, the techniques of this disclosure are not so limited. As an example, multiple reference points in a single reference frame may be used together to predict a current point. For example, a G-PCC coder may determine multiple reference points in a reference frame and based on a reference pivot point. The G-PCC coder may predict one or more parameters of a current point in a current frame based on multiple reference points. As another example, reference points from multiple reference frames may be used to predict a current point.

[0157] As explained above, the G-PCC coder may perform azimuth prediction. Let (r, phi, laserID) be the three coordinates of the pivot point in a spherical coordinate system (called radius, azimuth, and laserID). The techniques disclosed herein may also be applied to other coordinate systems.

[0158]

[0136] In some examples, the G-PCC coder may code points in the current point cloud frame in an ordered manner, as follows:

[0159]

[0137] First, for a current point in a current frame, the G-PCC coder may select a pivot point in the current frame that precedes the first point in decoding order. In some examples, the pivot point is a preceding point in the current frame in decoding order. In some examples, the pivot point is a second preceding point in the current frame in decoding order. More generally, two or more preceding points may be selected as the pivot point of the current point. In some examples, the pivot point may be a hypothetical point derived based on a previously decoded point in the current frame and an azimuth displacement that is a multiple of the azimuth quantization scale value (predetermined or derived from a signaled syntax element).

[0160]

[0138] Second, the G-PCC coder may select a point in the reference frame, the reference pivot point, associated with the pivot point. The reference pivot point may be selected as a point in the reference frame that has the same azimuth as the pivot point and laser ID. In some examples, points with other laser ID values ​​may also be candidates for the reference pivot point (e.g., the reference pivot point may be selected as a point in the reference frame that has the same azimuth as the pivot point and a laser ID within the range of [LaserID-M, LaserID+M], where LaserID is the laser ID of the pivot point, and M is a fixed value (e.g., 1) or is selected based on the distance of the pivot point from the origin, or is derived as a function of LaserID (e.g., for smaller values ​​of LaserID, M may be smaller, and for larger values ​​of LaserID, M may be larger)). In some examples, a distance metric may be defined using an azimuth angle and a laser ID, and a reference pivot point is selected as a point having a minimum distance from the azimuth angle and laser ID of the pivot point using the distance metric. A normalized azimuth angle value may be obtained by scaling the azimuth angle by a first constant value. A normalized laser ID may be obtained by scaling the laser ID by a second constant value. A distance metric may be obtained by calculating a norm (e.g., L2 norm, L1 norm) on the normalized laser ID and azimuth angle values ​​of the pivot point and the reference point. More generally, a reference pivot point may be selected as a point in a reference frame that is in the neighborhood of a point with the same azimuth angle and laser ID as the pivot point. In some examples, a reference pivot point may be a virtual point in a reference frame derived from a pivot point that has the same azimuth angle and laser ID as the pivot point.

[0161]

[0139] Third, the G-PCC coder may select a reference point in a reference frame associated with the reference pivot point. The reference point may be selected as a point in a reference frame that has a minimum azimuth angle greater than that of the reference pivot point and the same laser ID as the reference pivot point. The reference point may be selected as a point in a reference frame that has a second minimum azimuth angle greater than that of the reference pivot point and the same laser ID as the reference pivot point. In some examples, when a reference point is not available, inter prediction may be disabled for the current point. In some examples, a reference point may be selected as the reference pivot point.

[0162]

[0140] Fourth, the G-PCC coder may calculate a first residual between the reference point and the reference pivot point.

[0163]

[0141] Fifth, the G-PCC coder may use the first residual to derive a first prediction for the current value. The prediction may be derived by adding a component of the first residual to a respective component of the pivot point (e.g., a radial prediction may be obtained by adding a radial component of the first residual to a radial component of the pivot point (similarly for the azimuth angle)). In some examples, the first prediction may be set equal to the reference point.

[0164]

[0142] Sixth, the G-PCC coder may code a second residual between the first prediction and the position of the current point.

[0165]

[0143] Seventh, configuration of residuals: One or more residuals disclosed in the present disclosure may include one or more of the following: The residuals may include a radial residual between the reference pivot point and the reference point. The residuals may comprise an azimuth residual between the reference pivot point and the reference point.

[0166]

[0144] Eighth, the G-PCC coder may derive the current point based on the first prediction and the second residual. In some examples, the G-PCC coder may derive the current point from the second residual (e.g., not based on the first prediction).

[0167]

[0145] The G-PCC coder may apply one or more techniques described above to the quantized azimuth angle value. The scale value used for quantization may be derived from a signaled value or may be predetermined. The quantized azimuth angle value and laserID may be used to search for an inter-predicted point in the reference. For example, the azimuth angle of a previously decoded and reconstructed point may be quantized, and the inter-predicted point with the quantized azimuth angle and laserID closest to the quantized azimuth angle and laserID of the previous point may be selected as the predictor for the azimuth angle, radius, and laserID of the current point, either as the quantized / dequantized predictor or dequantized.

[0168]

[0146] The G-PCC coder may apply one or more techniques described above to the quantized laserID value. The scale value used for quantization may be derived from a signaled value or may be predetermined. The azimuth angle and the quantized laserID value may be used to search for an inter-predicted point in the reference. For example, the laserID of a previously decoded and reconstructed point may be quantized, and the inter-predicted point with the azimuth angle and quantized laserID closest to the azimuth angle and quantized laserID of the previous point may be selected as the predictor for the azimuth angle, radius, and laserID of the current point, either as the quantized / dequantized predictor or dequantized.

[0169]

[0147] The G-PCC coder can apply one or more techniques described above to the quantized azimuth angle and the quantized laserID value. The scale value used for quantization can be derived from a signaled value or can be predetermined. The quantized azimuth angle and the quantized laserID value can be used to search for an inter-predicted point in the reference. For example, the azimuth angle and the laserID of a previously decoded and reconstructed point can be quantized, and the inter-predicted point with the quantized azimuth angle and the quantized laserID that is closest to the quantized azimuth angle and the quantized laserID of the previous point can be selected as the predictor for the azimuth angle, radius, and laserID of the current point, either as the quantized / dequantized predictor or as the dequantized predictor.

[0170] In some examples, a reference frame may refer to a set of (radius, azimuth, laserID) tuples derived from the reference frame. For example, for each point in the reference frame, if there is no other point in the set with the same azimuth and laserID, the G-PCC coder may add the radius, azimuth, and laserID to the set. In some cases, a quantized value of the azimuth may be added. In some cases, even if there is another tuple in the set with the same phi and laserID, e.g., (r1, phi, laserID), (r, phi, laserID) may be added if the value of r is less than the value of r1 (in this case, the existing tuple (r1, phi, laserId) may be replaced by the new (r, phi, laserID)). In some cases, the points in the reference frame may be in the x, y, z domain. The points may be stored as is for reference or by transforming to the spherical domain. In some cases, a motion-compensated position may be added to the reference frame. The compensation may be based on signaled motion vectors (eg, global motion vectors with rotation and / or translation) associated with the current frame and the reference frame.

[0171]

[0149] The G-PCC coder may perform context selection for inter-prediction modes. As described above, the G-PCC coder may signal an inter-prediction flag and an inter-prediction mode. The context used by the G-PCC coder to code the inter-prediction flag or the inter-prediction mode may be selected as follows, and the G-PCC coder may set b1, b2, b3, ... bN to be the inter-prediction flag values ​​for N previously decoded points (b1 is the previous node, b2 is the node immediately before b1, etc.). The G-PCC coder may select a context index as a number generated by using b1, b2, ... bN as follows, i.e., ctxIdx=b1+b2<<1+b3<<2+...bN<<(N-1). In another example, the G-PCC coder may select a context index as b1+b2+...+bN. In some examples, the values ​​of b1, b2, ... bN may also be used to select the context of other syntax elements, such as inter prediction modes, motion vectors, etc. In some examples, N may be fixed as 5.

[0172]

[0150] The G-PCC coder may perform improved radius residual coding. The radius, azimuth, and laserID associated with each point may be coded in the predictive geometry coding. The residual of the radius component of each point may be coded with an "equal_to_zero" flag, a sign bit, a number of bits for the residual, and / or the residual. The number of bits may be an indicator of the relative value of the radius residual. When the points captured by the LIDAR are close to each other in the point cloud (e.g., points of an object or building captured by one of the lasers), the radius may not vary significantly across the points. Since the object (most likely solid) is continuous, consecutive LIDAR scan points may be returned to the sensor, where the azimuth angle difference between consecutive points is close to the sampling frequency of the azimuth angle. The difference in azimuth angle or azimuth angle residual for adjacent points (of the same laser) on the object is close to 0.

[0173]

[0151] The context of the number of bits for the radius residual is selected based on the azimuth residual of the current point. For example, based on the absolute value of the azimuth residual, the context can be switched as follows, where N is a fixed value (signaled or predetermined). Table 1 below provides an example of a context index for the number of bits used in the number of bits for the radius residual N.

[0174] [Table 1]

[0175]

[0152] It should be noted that other processes for selecting a context index based on the radius residual may also be used.

[0176]

[0153] The following specific examples illustrate implementations of some of the techniques of this disclosure.

[0177] Example A For each point in the full non-intracoded point cloud frame (random access point), a flag may be used to specify whether the point is coded using inter prediction. When inter prediction is applied, a "zero motion vector" candidate is chosen to predict the radius of the predicted point from the point in the reference frame that has the same laser ID value as the current point and has the closest quantized azimuth value from the current point. The changes to the syntax structure are provided below ( <add> ...< / add> The tag indicates the addition <delete> ...< / delete> The tag indicates the deletion.

[0178] [Table 2-1]

[0179] [Table 2-2]

[0180] [Table 2-3]

[0181] <add> A ptn_inter_flag[nodeIdx] equal to 0 specifies that the radial residual of the current node is coded using intra prediction. A ptn_inter_flag[nodeIdx] equal to 1 specifies that the radial residual of the current node is coded using inter prediction. When a value for ptn_inter_flag[nodeIdx] is not present, a value for ptn_inter_flag[nodeIdx] equal to 0 is inferred.

[0182] < / add>

[0155] The value of random access InterEnableFlag is set to 0 when the current point cloud frame is a random access point or an all-intra frame, and is set to 1 otherwise.

[0183]

[0156] In some examples, when ptn_inter_flag is equal to 1, the syntax element ptn_pred_mode[ ] is not signaled and is inferred to be a default value (eg, equal to 0, corresponding to predicted geometry mode 0).

[0184]

[0157] The modifications to the location prediction process are as follows: Location Prediction Process The inputs to this process are: A variable predMode that indicates the prediction mode of the current node, Arrays aPos0, aPos1, and aPos2 with values ​​aPosX[k], k = 0..2, and X = 0..2, each containing the position associated with an ancestor node of the Xth generation in a depth-first tree traversal order; A variable curDepth, which indicates the distance in nodes between the current node and the root node of the current prediction tree; <add> This is a variable interFlag that indicates whether the radius of the current node is coded using inter prediction. < / add> The output from this process is an array predPos with values ​​predPos[k], k=0..2, indicating the predicted point position associated with the nodeIdx-th tree node. When predMode is equal to 0, the predicted point locations are calculated as follows: for(k=0;k<3;k++) predPos[k]=0 if(geometry_angular_enabled_flag){ predPos[0]=ptn_radius_min_value if(curDepth>0) { predPos[1] = aPos0[1] predPos[2]=aPos0[2] } } When predMode is equal to 1, the predicted point location is the location associated with the first ancestor. for(k=0;k<3;k++) predPos[k] = aPos0[k] When predMode is equal to 2, the predicted point location is a linear combination of the locations associated with the first two ancestors. for(k=0;k<3;k++) predPos[k]=aPos0[k]+aPos0[k]-aPos1[k] Otherwise, predMode is equal to 3 and the predicted point location is a linear combination of the locations associated with all three ancestors. for(k=0;k<3;k++) predPos[k]=aPos0[k]+aPos1[k]-aPos2[k] <add> When interFlag is equal to 1, predPos[0] is derived as follows: Let refFramePos[k], for k=0..2, be the position of the point in the reference frame, and let refFramePos[2]=predPos[2], and let the absolute difference between the value of refFramePos[1] and predPos[1] be the smallest among the points in the reference frame. predPos[0] is set equal to refFramePos[0]. < / add> In one example, the quantized values ​​of refFramePos[1] and predPos[1] are used to measure the absolute difference in values, where the quantization scale may be a fixed number (eg, geom_angular_azimuth_step_minus1+1).

[0185] Example B In embodiment B, the prediction mode may not be signaled when inter prediction is used. For example, when inter prediction is used, the prediction mode is inferred to be equal to 0. Changes to the syntax structure are provided below ( <add> ...< / add> The tag indicates the addition <delete> ...< / delete> The tag indicates the deletion.

[0186] [Table 3]

[0187]

[0159] When ptn_pred_mode[nodeIdx] is not signaled, ptn_pred_mode[nodeIdx] may be inferred to be equal to 0. In some examples, the following inference may be added: When ptn_inter_flag[nodeIdx] is equal to 1, ptn_pred_mode[nodeIdx] is inferred to be equal to 0.

[0188] Example C In embodiment C, the order of the signaling of phi and prediction mode may not be changed. The signaling of prediction mode may be conditioned on the signaling of the inter flag. The signaling of ptn_phi_mult_abs_gt0_flag may also be conditioned on the inter flag. Changes to the syntax structure are provided below ( <add> ...< / add> The tag indicates the addition <delete> ...< / delete> The tag indicates the deletion.

[0189] [Table 4]

[0190]

[0161] When ptn_inter_flag[nodeIdx] is not signaled, ptn_inter_flag[nodeIdx] is inferred to be equal to 0.

[0191]

[0162] When ptn_phi_mult_abs_gt0_flag[nodeIdx] is not signaled, ptn_phi_mult_abs_gt0_flag[nodeIdx] is inferred to be equal to 0. In another example, the following inference is added: When ptn_inter_flag[nodeIdx] is equal to 1, ptn_phi_mult_abs_gt0_flag[nodeIdx] is inferred to be equal to 1.

[0192] Example D

[0163] In embodiment D, the order of the pred mode and phi syntax elements may not be changed. For example, the inter flag may be signaled after the pred mode but before the phi syntax elements.

[0193] [Table 5]

[0194]

[0164] When ptn_pred_mode[nodeIdx] is greater than 0 and InterEnableFlag is equal to 1 (ptn_inter_flag[] is not signaled), ptn_inter_flag[nodeIdx] is inferred to be 0.

[0195] Example E

[0165] Example E is similar to example A with the modification that the signaling of the inter flags also depends on the value of pred_mode. The following whole syntax is added:

[0196] [Table 6]

[0197]

[0166] When InterEnableFlag is equal to 1 and PtnPhiMult[nodeIdx] is equal to 0 or ptn_pred_mode[nodeIdx] is not equal to 0, ptn_inter_flag[nodeIdx] is inferred to be equal to 0.

[0198] Example F

[0167] In this example, the prediction for the radius, predPos[0], is obtained by comparing not only the azimuth angles of points with the same laser ID as the current point, but also points of neighboring laser IDs.

[0199]

[0168] When interFlag is equal to 1, the G-PCC coder may derive predPos[0] as follows: The G-PCC coder may set refFramePos[k], for k=0..2, to be the position of the point in the reference frame, such that refFramePos[2] takes one of the values ​​in {predPos[2]-1, predPos[2], predPos[2]+1}, such that the absolute difference between the value of refFramePos[1] and predPos[1] is the lowest among the points in the reference frame. The G-PCC coder may set predPos[0] equal to refFramePos[0].

[0200]

[0169] In some cases, points from more laser IDs within the reference frame may be considered to obtain the reference point.

[0201] Alternatively, rather than comparing the azimuth angles (ie, refFramePos[1] and predPos[1]), a weighted cost function may be specified as follows: J(refPos)=w1 * f1(refPos[1],predPos[1])+w2 * f2(refPos[2],predPos[2])

[0202]

[0171] The point in the reference frame with position refPos[k], for k=0..2, chosen to predict the radius is the point with the minimum value of J(refPos).

[0203]

[0172] In some cases, the value of f1(x,y) may be the square of the difference between x and y, or more generally, any function indicating the distance between the current point and a reference point in the azimuth (or second) dimension.

[0204]

[0173] In some cases, the value of f2(x,y) may be the square of the difference / absolute difference between x and y, or more generally, any function indicating the distance between the current point and a reference point in the laser ID (or third) dimension.

[0205]

[0174] In other cases, more than one reference candidate may be selected for a point, and an inter-pred candidate index may be signaled to specify which reference candidate is selected for that point.

[0206] For example, for the current point, three points in the reference frame may be chosen as follows: The point with laser ID equal to predPos[2]-1 that has the azimuth value closest to predPos[1] The point with laser ID equal to predPos[2] that has the azimuth value closest to predPos[1] The point with laser ID equal to predPos[2]+1 that has the azimuth value closest to predPos[1]

[0207]

[0176] An index into this set / list may be signaled to specify the point selected for prediction.

[0208]

[0177] More generally, for each current point coded with inter prediction for a radius, The reference frame may be selected based on a pre-determined decision (eg, a previous decoded frame) or based on one or more signaled indications (eg, a frame index / counter value). The set RefCandSuperSet is selected among the points in the reference frame based on a first set of conditions (e.g., based on laser ID range and / or azimuth angle range, one or more processes disclosed herein). A subset, RefCandSubset, is selected from RefCandSuperSet based on a second set of criteria (e.g., closest azimuth angle from the current point for a particular laser ID). If RefCandSubset has two or more entries, then position the entries in the list RefCandList based on a third set of conditions (e.g., position the entries based on incremental laser ID difference from the current point), otherwise (only one candidate in RefCandSubset), one candidate in RefCandSubset is used for prediction. Signals the index to the entry in the RefCandList used for the prediction. Use the prediction entry to predict the radius of the current point.

[0209]

[0178] In some examples, when a point in a reference frame is not present in the RefCandSubset, inter prediction may be disabled, and the inter flag may or may not be signaled in this case.

[0210] Example G According to one or more aspects of the present disclosure, the G-PCC coder may signal in the bitstream parameters for deriving the azimuth quantization / scale. In some examples, the G-PCC coder may inter-predict all three components using previously decoded points in the current point cloud frame.

[0211]

[0180] The azimuth angle scale value (used in deriving the inter-prediction candidates) may be signaled in the GPS as follows:

[0212] [Table 7]

[0213]

[0181] In some examples, inter_azim_scale_log2 may be signaled independently of inter_prediction_enabled_flag.

[0214]

[0182] In another example, the angle mode may also be used to adjust the signaling of inter_azim_scale_log2.

[0215] [Table 8]

[0216]

[0183] Below are example semantics for the above syntax elements: inter_prediction_enabled_flag equal to 1 specifies that inter prediction may be used for GPS referenced point cloud frames. inter_prediction_enabled_flag equal to 0 specifies that inter prediction is not used for GPS referenced point cloud frames. inter_azim_scale_log2 is used to scale azimuth points that may be used in the derivation of inter prediction references. The value of inter_azim_scale_log2 shall be in the range of 0 to numAzimBits, inclusive. (The variable numAzimBits may specify the maximum number of bits used to represent the azimuth angle.)

[0217] In some examples, an inter_azim_scale_log2 syntax element may be an example of a syntax element that specifies a log base 2 value of a scaling factor. The G-PCC coder may determine a scaled azimuth angle (e.g., a scaled azimuth angle of the pivot point) based on the scaling factor.

[0218]

[0185] When a frame is coded, the reconstructed spherical positions of one or more points in the frame may be stored in a reference frame that may be used as an inter-prediction reference by future frames. The reconstructed spherical positions may be stored in a hash table as follows: The G-PCC coder may obtain a spherical coordinate representation for the reconstructed points. The G-PCC coder may obtain the derived azimuth angle and laser index from the reconstructed point and use the tuple of derived azimuth angle and laser index as an index into a hash table.

[0219] For example, the derived quantized azimuth angle, denoted as quantized(val), may be derived by the G-PCC coder as follows: int offset=azimScaleLog2?(1<<(azimScaleLog2-1)):0; quantized(val)=val>=0?(val+offset)>>azimScaleLog2 :-((-val+offset)>>azimScaleLog2);

[0220]

[0186] In some cases, azimScaleLog2 is not signaled and can be inferred to one of the following: Value 0 (i.e. no quantization). The smallest power of two that is less than or equal to the azimuth rate signaled in the bit stream. The smallest power of 2 that is less than or equal to (azimuth rate / 2) signaled in the bit stream. Different sequences may have different azimuth scale values. Each entry in the hash table may be derived from one or more reconstructed points. In one example, the hash table entry for a frame may be the first point in decoding order within the reconstructed frame that has a particular value of derived azimuth angle and laser index. In one example, the hash table entry for a frame may be the point in the reconstructed frame that has the smallest radius among points having a particular value of derived azimuth angle and laser index. In yet another example, the hash table entry for a frame may be a point value that corresponds to the average, mean, median, weighted average, geometric mean, etc., calculated using points having particular values ​​of derived azimuth angle and laser index.

[0221]

[0187] In some examples, a reconstructed point position (r, phi, laserID) in the spherical domain may be entered into the azimuth table as follows: table index = (quantized(Phi), laserID), table entry = (r, phi, laserId).

[0222]

[0188] When coding the current point (in the current frame), the following steps may be performed. The G-PCC coder may set prevNode=(r1, phi1, laserID1) to be the previous node in the current frame in decoding order. In some examples, prevNode may be chosen as the parent node of the current node. In the table associated with the reference frame, the G-PCC coder may check whether there exists an index (quantized(phi), laserID1) where quantized(phi) is greater than quantized(phi1). If no such index exists, the G-PCC coder may not apply inter prediction for the current node. If at least one such index exists, the G-PCC coder may select the index with the smallest quantized(phi) (greater than quantized(phi1)). The G-PCC coder may use the entry associated with the above selected index (r_inter, phi_inter, laserId_inter) as the inter predictor for the current point.

[0223] In some examples, the following code may be used to implement the above technique with a combination of an array of hash tables (instead of a 2D hash table), where computePhiQuantized() corresponds to the quantize() function described previously. In the following, refPointVals is a vector of hash tables that store reference points, refPointVals[currLaserId] is a hash table corresponding to index laserID=currLaserId, and all entries in refPointVals[currLaserId] are stored with increasing values ​​of quantized(phi). Since this table is stored with increasing quantized(phi), the upper_bound function is used to search for the first quantized(phi) laser that is greater than the quantized current azimuth angle quantize(currAzim). In the following example, idx is equal to refPointVals[currLaserId].end(), there is no such table index, and therefore inter prediction is not used. When present, idx->second specifies the inter prediction candidate (r_inter, phi_inter, laserID_inter).

[0224] In some examples, the hash table may not store points with increasing values ​​of quantized(phi), and instead may use a hash function to parse and recover the table entries. In another example, a generic table / data structure may be used to store points from a reference frame. As mentioned above, the following code may be used to implement the above technique: auto quantizedPhi=computePhiQuantized(currAzim); if(refPointVals[currLaserId].size()){ auto idx=refPointVals[currLaserId].upper_bound(quantizedPhi); if(idx==refPointVals[currLaserId].end()) return std::pair<bool,point_t> (false,0); else return std::pair<bool,point_t> (true,idx->second); } return std::pair<bool,point_t> (false,0); If inter prediction is used, the G-PCC coder may signal a flag indicating that inter prediction is used. To code the residual, we use the inter prediction candidates derived above (at the decoder, the prediction candidates are used to add the residual and to derive the reconstructed spherical coordinate position of the current point).

[0225]

[0191] Predictive geometry coding uses a prediction tree structure to predict the location of a point. If angle coding is enabled, the x, y, z coordinates can be converted to radius, azimuth, and laserID, and the residual can be signaled in these three coordinates as well as in the x, y, and z dimensions. The intra prediction used for radius, azimuth, and laserID can be one of four modes, and the predictors are the nodes classified as parent, grandparent, and great-grandparent in the prediction tree with respect to the current node. Predictive geometry coding as currently designed in G-PCC Ed.1 is an intra coding tool because it only uses points within the same frame for prediction. Furthermore, using points from previously decoded frames may provide better prediction and therefore better compression performance.

[0226]

[0192] Inter prediction is proposed to predict the radius of a point from a reference frame. For each point in the prediction tree, the G-PCC coder may decide whether the point is inter predicted or intra predicted (e.g., indicated by a flag). If intra predicted, the intra prediction mode of predictive geometry coding may be used. If inter prediction is used, the azimuth angle and laserID are still predicted using intra prediction, while the radius may be predicted from a point in the reference frame that has the same laserID as the current point and the closest azimuth angle to the current azimuth angle. In some examples, the G-PCC coder may perform inter prediction of the azimuth angle and laserID (e.g., in addition to radius prediction). If inter coding is applied, the G-PCC coder may predict the radius, azimuth angle, and laserID of the current point based on points that are near the azimuth position of the previously decoded point in the reference frame. Furthermore, separate sets of contexts may be used for inter prediction and intra prediction.

[0227]

[0193] As described above, Figure 7 shows an example of inter prediction of a current point (curPoint) from a point (interPredPt) in a reference frame. Extending inter prediction to azimuth angle, radius, and laserID may include the following steps. For a given point, the G-PCC coder can select the previous decoded point (prevDecP0). The G-PCC coder can select a position in the reference frame (refFrameP0) that has the same scaled azimuth angle and laserID as prevDecP0. In the reference point cloud frame, the G-PCC coder can find the first point (interPredPt) that has an azimuth angle greater than that of refFrameP0. interPredPt is sometimes called the "next" inter predictor.

[0228]

[0194] Figure 8 is a flowchart showing an example decoding flow associated with "inter_flag" signaled per point. The inter flag signaled for a point indicates whether inter prediction is applied to the point. The flowchart of this disclosure is given as an example. Other examples may include more, fewer, or different steps, or the steps may be performed in a different order.

[0229] In the example of FIG. 8, the G-PCC decoder 300 may determine whether the inter flag of the next point to be decoded (i.e., the current point of the current frame of the point cloud data) indicates that the current point is inter predicted (800). If the inter flag of the current point does not indicate that the current point is inter predicted (the "no" branch of 800), the G-PCC decoder 300 may identify intra prediction candidates (812). For example, the G-PCC decoder 300 may determine an intra prediction strategy (e.g., no prediction, delta prediction, linear prediction, parallelogram prediction, etc.) to determine a predictor for the current point. A syntax element (pred_mode) signaled in the geometry bitstream 203 may indicate the intra prediction strategy to be used to determine a predictor for the current point.

[0230] On the other hand, if the inter flag for the current point indicates that the current point is inter predicted ("Yes" branch of 800), the G-PCC decoder 300 may identify a previous point in decoding order (e.g., previous point 708) (802). The previous point may have coordinates (r, phi, and laserID). The G-PCC decoder 300 may then derive a quantized phi coordinate (i.e., azimuth coordinate) of the previous point (804). The quantized phi coordinate may be denoted as Q(phi). The G-PCC decoder 300 may then check a reference frame (e.g., reference frame 706) for a point (i.e., an inter predicted point (e.g., interPredPt 704)) that has a quantized phi coordinate greater than the quantized phi coordinate of the previous point (806). The G-PCC decoder 300 may use the inter predicted point as a predictor for the current point (808).

[0231]

[0197] Regardless of whether the G-PCC decoder 300 determines a predictor for the current point using intra prediction (e.g., as described with respect to step 812) or using inter prediction (e.g., as described with respect to steps 802-808), the G-PCC decoder 300 may add a delta-phi multiplier (810).

[0232]

[0198] The LIDAR system can scan and sample content at a specific azimuth frequency. The G-PCC encoder 200 can signal the azimuth of a point using the azimuth residual value (resAz2). The azimuth residual is not the direct difference between the azimuth of the current point and the azimuth of the predictor for the current point (it can be determined using inter-prediction or intra-prediction). Rather, the G-PCC encoder 200 encodes the azimuth residual as a combination of a delta phi multiplier and a residual. For example, azimSpeed can represent the azimuth difference between two consecutive captures of one of the rotating LIDAR sensors (note that there are multiple sensors within each rotating LIDAR system). Thus, if each LIDAR sensor within the rotating LIDAR system captures 1000 points per rotation, azimSpeed is (1<<azimBitDepth) / 1000, which effectively becomes the azimuth difference between adjacent point captures. Here, azimBitDepth is the bit depth used to represent the azimuth value in the G-PCC encoder 200. (1<<azimBitDepth) represents one full rotation (360 degrees). Thus, (1<<azimBitDepth) / 1000 represents the difference in azimuth between two captures. The LIDAR sensors can be designed to sample at regular intervals during rotation, but the difference in azimuth values between adjacent captures may not be azimSpeed or multiple azimSpeeds due to noise and other inaccuracies. The first azimuth residual resAz1, which is the difference between the azimuth of the current point and the predictor, is encoded as a combination of a delta phi multiplier (qphi) and a second azimuth residual (resAz2). qphi can be derived as follows. qphi = deltaPhi >= 0? (deltaPhi + (_geomAngularAzimuthSpeed >> 1)) / _geomAngularAzimuthSpeed : -(-deltaPhi + (_geomAngularAzimuthSpeed >> 1)) / _geomAngularAzimuthSpeed; The value of the coded azimuth residual resAz2 is resAz1 - deltaPhi* The reconstructed azimuth residual (resAz1) is obtained as resAz2+qphi. Both qphi and resAz2 are signaled in the bitstream. The G-PCC decoder 300 can decode resAz2 and qphi. The G-PCC decoder 300 can decode the reconstructed azimuth residual (resAz1) as resAz2+qphi. * It can be obtained as azimSpeed.

[0233]

[0199] Figure 9 is a conceptual diagram showing an example additional inter predictor point 900 obtained from a first point with a larger azimuth than the inter predictor point 914. In the inter prediction method for prediction geometry described above with respect to Figure 7, the radius, azimuth, and laserID of the current point (current point 700) are predicted based on a nearby point (inter prediction point 704) of a collocated azimuth position (reference position 710) in a reference frame (reference frame 706) when inter coding is applied. In the example of Figure 9, the G-PCC encoder 200 and the G-PCC decoder 300 can determine the additional inter predictor point 900 using the following steps. a) For a given point (current point 900 in current frame 904), determine the previous point 902 in the current frame 904 (the "prev dec point" in FIG. 9). b) Determine a reference position 906 in the reference frame 908 having the same scaled azimuth angle and laserID as the previous point 902 determined in step a) ("Reference point with same scaled azimuth angle and laserID" in FIG. 9). c) determine a position in the reference frame 908 as the first point having an azimuth angle (e.g., a scaled azimuth angle) greater than the reference position 906 determined in step b), which is used as the inter prediction point (inter pred point 910 in FIG. 9 ).

[0234] The additional inter prediction point can be obtained by finding the first point ("additional inter prediction point 912" in FIG. 9) that has a larger azimuth angle (e.g., a scaled azimuth angle) than the inter prediction point 910 determined in step c), as shown in FIG. 9. If inter coding is applied, additional signaling may be used to indicate which of the predictors is selected. The additional inter predictor point may also be called the "NextNext" inter predictor.

[0235]

[0201] In some examples, the G-PCC encoder 200 (e.g., the arithmetic encoding unit 214 of the G-PCC encoder 200) and the G-PCC decoder 300 (e.g., the geometry arithmetic decoding unit 302 of the G-PCC decoder 300) can apply an improved context selection algorithm to code the inter prediction flag. The inter prediction flag values ​​of the five previously coded points can be used to select the context of the inter prediction flag in predictive geometry coding.

[0236]

[0202] The above-mentioned techniques may result in one or more drawbacks. Inter prediction uses points in a reference frame to predict points in a given frame. In some examples, a zero-compensated reference frame may be used, and in some cases, a global motion compensated reference frame may be used for prediction. In a zero-motion compensated frame, a point in a reference frame is used without applying motion compensation. A global motion compensated reference frame is obtained from a (zero-compensated) reference frame by motion compensation based on a set of motion parameters. These parameters may be derived, for example, using a motion estimation technique, or may be obtained from external means, for example, using GPS parameters or from the motion of a car that captures a point cloud. When motion compensation is applied to a point in a reference frame, the location of the point is modified, which results in a decrease in coding efficiency.

[0237] According to one or more techniques of this disclosure, a G-PCC coder can perform resampling on a reference frame (e.g., a zero-compensated reference frame, an uncompensated reference frame, etc.). A zero-compensated reference typically refers to compensation with zero motion and zero rotation. In many cases, a zero-compensated reference and an uncompensated reference are equivalent.

[0238]

[0204] The G-PCC coder can selectively perform a resampling operation on points of the reference frame as follows. For each point pt in the (zero-compensated) reference frame, the G-PCC coder can determine whether resampling should be applied for this point. If no resampling is applied, the G-PCC coder may keep the points as they are. Otherwise (if resampling is applied), the G-PCC coder may choose (e.g., select) a first set of points in a globally compensated reference frame (GlobRef) associated with pt. For one or more components of pt, the G-PCC coder may derive a value from the first set of points and use the value derived for pt for the corresponding component. For example, if the point pt is represented as (r, phi, laserId) as components, where r_d is the radius derived from the first set of points, then the point pt is updated and stored as (r_d, phi, laserId).

[0239]

[0205] The individual steps of an exemplary resampling operation relating to point pt are described below.

[0240]

[0206] The G-PCC coder decides whether resampling should be applied to the point. The G-PCC coder may use the following conditions to determine whether resampling should be applied to a point: Whether a point belongs to a road or whether a point belongs to a building / other object in the scene. A "road" point can be determined by one or more thresholds signaled in the bitstream. For example, if two thresholds a1 and a2 are the two z-thresholds signaled that indicate a road point, then for points with z-values ​​in the range [a1, a2], no resampling is applied. For other points, resampling is applied. In some examples, all points may be resampled.

[0241]

[0207] The G-PCC coder selects a first set of points associated with the current point pt in the zero-compensated frame as follows: For a given point pt with components (r, phi, lId), a first set of points in GlobRef may be selected as those points that are close to the colocated point in GlobRef. For example, a point in GlobRef that has a Laser ID value equal to lId and an azimuth angle in the range [phi-delta, phi+delta], where delta is a fixed value based on the Laser ID. In another example, the azimuth scale value may be used to determine a quantized azimuth value for all points (e.g., phi may be quantized to phiQ). The points in the reference frame associated with the quantized azimuth value phiQ may be considered to be part of the bin associated with phiQ. A first set of points may be determined as those points in GlobRef that have a laser ID equal to lId and belong to: bins associated with phiQ, the first non-empty bin associated with a quantized azimuth angle value greater than phiQ; The first non-empty bin associated with a quantized azimuth angle value less than phiQ. The G-PCC coder may trim the first set of points to the one or two points that have an azimuth angle value closest to phi. In one example, if there exists a point P1(r_a, phi_a, lId) that belongs to GlobRef and is associated with bin phiQ, it is as follows. If phi_a = phi, only P1 is retained in the first set. If phi_a > phi, P1 is retained and another point P2(r_b, phi_b, lId) is selected such that phi_b < phi. If phi_a < phi, P1 is retained and another point P3(r_c, phi_c, lId) is selected such that phi_c > phi. If such a point P1 does not exist, two points P2(r_b, phi_b, lId) and P3(r_c, phi_c, lId) are selected from the first set such that phi_b < phi and phi_c > phi. If one or more of points P2 or P3 are not available in GlobRef, the unavailable point(s) may not be included in the first set of points.

[0242]

[0208] The G-PCC coder can derive a radius value from the points within the first set as follows (this can also be applied to other components). If only one point (r_x, phi_x, lId) exists in the first set, the radius of the point pt is updated as r_x. That is, the point pt is updated from (r, phi, lId) to (r_x, phi, lId). If two or more points exist within the first set, the radius of the point pt is obtained as an interpolated value from the radius values of the points within the first set.

[0243]

[0209] The following example of the technique of the present disclosure is provided for reference. In one example, the G-PCC coder can derive a globally compensated reference frame GlobRef and use resampling based on the points in GlobRef to update the points in the zero-compensated reference frame ZeroRef. The G-PCC coder can use the updated (resampled) zero-compensated reference frame for inter prediction.

[0244]

[0210] Figure 10 is a flow chart illustrating an example technique of how a resampled point may be used for inter prediction, according to one or more techniques of this disclosure. In the example of Figure 10, a G-PCC coder (e.g., G-PCC encoder 200 or G-PCC decoder 300) determines that a current point in a current frame should be predicted using an inter prediction mode. In response to determining to predict the current point in the current frame using an inter prediction mode, the G-PCC coder identifies a zero-compensated reference frame, e.g., ZeroRef, associated with the current frame (1002). The G-PCC coder derives a motion-compensated reference frame, e.g., GlobRef, associated with the current frame (1004), and selectively resamples the point in the zero-compensated reference frame based on one or more points in the motion-compensated reference frame to generate a resampled zero-compensated reference frame (1006). The G-PCC coder inter predicts the current point in the current frame based on the resampled zero-compensated reference frame (1008).

[0245]

[0211] The above operation (resampling) can also be implemented as resampling the points in GlobRef using azimuth values ​​from ZeroRef. For each point in ZeroRef, the G-PCC coder can determine whether resampling should be applied to the point. If yes, the G-PCC coder can choose a first set of points from GlobRef, obtain an interpolated radius value r_int from the first set of points, and update the radius value of PT with the interpolated radius value r_int.

[0246]

[0212] Figure 11 is a flow chart illustrating an example technique for how a point in GlobRef can be resampled using a point from ZeroRef, according to one or more techniques of this disclosure. To selectively resample a point in a zero-compensated reference frame, the G-PCC coder selects (1102) a particular point in the zero-compensated reference frame and determines (1104) whether to resample the particular point. In response to determining to resample the particular point, the G-PCC coder determines (1106) a first set of points in the motion-compensated reference frame associated with the particular point and derives (1108) a derived radius value based on the first set of points. The G-PCC coder updates (1110) the radius value of the particular point based on the derived radius value.

[0247] 12A-12D are conceptual diagrams illustrating how a first set of GlobRef points 1202, 1204, and 1206 may be selected for ZeroRef point 1208, according to one or more aspects of the present disclosure. All points shown in FIGS. 12A-12D may have the same laser ID. Solid lines 1210, 1212, 1214, and 1216 may indicate azimuth bin boundaries (all azimuth values ​​within a bin are quantized to the same value). Dotted line 1218 indicates an azimuth value equal to the azimuth of point 1208. Based on the availability of GlobRef points in the same azimuth bin as point 1208 and the difference between the azimuth values ​​of points 1202, 1204, and 1206 (if available) and point 1208, the G-PCC coder can select a first set of points from points 1202, 1204, and 1206.

[0248]

[0214] In the example of Figure 12A, point 1204 is in the same azimuth bin as point 1208, and the azimuth of 1204 is equal to the azimuth of point 1208. Based on this, the G-PCC coder may select a first set of points as point 1204 (e.g., include point 1204 in the first set of points).

[0249]

[0215] In the example of Figure 12B, point 1204 is in the same azimuth bin as PT, and the azimuth of point 1204 is greater than the azimuth of point 1208. Based on this, the G-PCC coder can select a first set of points as points 1202 and 1204 (e.g., include points 1202 and 1204 in the first set of points).

[0250] In the example of Figure 12C, point 1204 is in the same azimuth bin as point 1208, and the azimuth angle of point 1204 is smaller than the azimuth angle of point 1208. Based on this, the G-PCC coder may select a first set of points as 1204 and 1206 (e.g., include points 1204 and 1206 in the first set of points).

[0251]

[0217] In the example of Figure 12D, point 1204 is not in the same azimuth bin as point 1208. Based on this, the G-PCC coder may select a first set of points as points 1202 and 1206 (e.g., include points 1202 and 1206 in the first set of points).

[0252]

[0218] When the first set of points includes only one point (r_x, phi_x, lId), the radius of the point PT may be set equal to r_x. When the first set of points includes two points (r_x, phi_x, lId) and (r_y, phi_y, lId), the radius of the point PT may be set equal to r_xy, which may be derived as follows: delAzim=phi_x-phi_y delRad=r_x-r_y if(delRad==0||delAzim==0) r_xy=r_x else r_xy=r_x+(phi_PT-phi_x) * delRad / delAzim

[0253]

[0219] The above example shows linear interpolation of radial points (e.g., interpolation of points in the first set of points), but higher order interpolators (e.g., cubic, etc.) may be used and the number of points in the first set may be greater than two.

[0254]

[0220] In some instances, the G-PCC coder may not perform interpolation, such as when delRad is greater than a certain threshold (eg, 1024 or 2048).

[0255]

[0221] In some examples, the G-PCC coder may not perform interpolation, such as when delAzim is greater than a particular threshold, or when the quantized value of delAzim is greater than a particular threshold (delAzim quantized with the azimuth scale value is greater than N, where N is a fixed number, e.g., 4).

[0256] In some examples, such as when no interpolation is performed (e.g., due to a threshold), the G-PCC coder may not update the point at ZeroRef. In some examples, such as when no interpolation is performed (e.g., due to a threshold), the G-PCC coder may set the point with respect to the radius of the point having an azimuth closer to phi_PT (e.g., if abs(phi_PT-phi_x) is <(phi_PT-phi_y), then r_xy may be set equal to r_x, otherwise r_xy is set equal to r_y).

[0257]

[0223] Figure 14 is a flowchart illustrating an example technique for performing inter prediction according to one or more techniques of this disclosure. In the example of Figure 10, a G-PCC coder (e.g., the G-PCC encoder 200 or the G-PCC decoder 300) identifies a reference frame (1302) and derives one or more motion-compensated reference points from the reference frame (1304).

[0258]

[0224] The G-PCC coder selectively resamples points in the reference frame based on one or more motion compensated reference points to generate a resampled reference frame (1306). To selectively resample points in the reference frame, the G-PCC coder may be configured to: select a particular point in the reference frame; determine whether to resample the particular point; in response to the decision to resample the particular point, determine a first set of points in the reference frame associated with the particular point, the first set of points including at least one of the one or more motion compensated reference points; derive a derived radius value based on the first set of points; and update the radius value of the particular point based on the derived radius value. To derive the derived radius value based on the first set of points, the G-PCC coder may be configured to selectively interpolate radius values ​​for a plurality of points to determine the derived radius value, and the first set of points may include a plurality of points. To selectively interpolate radius values ​​for the plurality of points, the G-PCC coder may be configured to determine not to interpolate radius values ​​for the first point and the second point in response to one or both of a difference in radius values ​​between a first point and a second point of the plurality of points being greater than a first threshold or a difference in azimuth angle values ​​between the first point and the second point being greater than a second threshold. The G-PCC coder may be configured to not update the radius value of the particular point based on the derived radius value in response to a determination not to interpolate radius values ​​for the plurality of points. The G-PCC coder may be configured to set the radius value of the particular point as the radius value of a single point of the first set of points in response to a determination not to interpolate radius values ​​for the plurality of points. The single point of the first set of points may be, for example, a point having an azimuth angle value closest to the azimuth angle value of the particular point.

[0259]

[0225] The G-PCC coder inter predicts the current point in the current frame based on the resampled reference frame (1308). The G-PCC coder can reconstruct a point cloud that includes the current point.

[0260]

[0226] Figure 14 is a conceptual diagram illustrating an example distance measurement system 1400 that can be used in one or more techniques of the present disclosure. In the example of Figure 14, the distance measurement system 1400 includes an illuminator 1402 and a sensor 1404. The illuminator 1402 may emit light 1406. In some examples, the illuminator 1402 may emit the light 1406 as one or more laser beams. The light 1406 may be one or more wavelengths, such as infrared wavelengths or visible light wavelengths. In other examples, the light 1406 is not a coherent laser light. When the light 1406 hits an object, such as the object 1408, the light 1406 forms a return light 1410. The return light 1410 may include backscattered light and / or reflected light. The returning light 1410 may pass through a lens 1411 that directs the returning light 1410 to cause an image 1412 of the object 1408 to be formed on the sensor 1404. The sensor 1404 generates a signal 1414 based on the image 1412. The image 1412 may comprise a set of points (e.g., as shown by the dots in the image 1412 of FIG. 14).

[0261] In some examples, the illuminator 1402 and the sensor 1404 may be mounted on a rotating structure such that the illuminator 1402 and the sensor 1404 capture a 360A-degree view of the environment. In other examples, the distance measurement system 1400 may include one or more optical components (e.g., mirrors, collimators, diffraction gratings, etc.) that enable the illuminator 1402 and the sensor 1404 to detect the distance of an object within a certain range (e.g., up to 360A degrees). Although the example of FIG. 14 shows only a single illuminator 1402 and sensor 1404, the distance measurement system 1400 may include multiple sets of illuminators and sensors.

[0262]

[0228] In some examples, the illuminator 1402 generates a structured light pattern. In such examples, the distance measurement system 1400 may include a plurality of sensors 1404 on which respective images of the structured light pattern are formed. The distance measurement system 1400 may use the parallax between the images of the structured light pattern to determine the distance to an object 1408 from which the structured light pattern is backscattered. The structured light based distance measurement system may have a high level of accuracy (e.g., accuracy in the sub-millimeter range) when the object 1408 is relatively close (e.g., 0.2 meters to 2 meters) to the sensor 1404. This high level of accuracy may be useful in facial recognition applications, such as unlocking a mobile device (e.g., a mobile phone, a tablet computer, etc.), and for security applications.

[0263]

[0229] In some examples, the distance measurement system 1400 is a time of flight (ToF) based system. In some examples where the distance measurement system 1400 is a ToF based system, the illuminator 1402 generates a pulse of light. In other words, the illuminator 1402 may modulate the amplitude of the emitted light 1406. In such an example, the sensor 1404 detects the return light 1410 from the pulse of light 1406 generated by the illuminator 1402. The distance measurement system 1400 can then determine the distance to the object 1408 from which the light 1406 is backscattered based on the delay time between the emission and detection of the light 1406 and the known speed of light in air. In some examples, rather than (or in addition to) modulating the amplitude of the emitted light 1406, the illuminator 1402 may modulate the phase of the emitted light 1406. In such an example, the sensor 1404 may detect the phase of the returning light 1410 from the object 1408 and determine the distance to a point on the object 1408 using the speed of light and based on the time difference between when the illuminator 1402 generates the light 1406 at a particular phase and when the sensor 1404 detects the returning light 1410 at that particular phase.

[0264]

[0230] In other examples, the point cloud may be generated without using the illuminator 1402. For example, in some examples, the sensor 1404 of the distance measurement system 1400 may include two or more optical cameras. In such examples, the distance measurement system 1400 may use the optical cameras to capture a stereo image of an environment including the object 1408. The distance measurement system 1400 may include a point cloud generator 1416 capable of calculating disparity between locations in the stereo image. The distance measurement system 1400 may then use this disparity to determine distances to locations shown in the stereo image. From these distances, the point cloud generator 1416 may generate a point cloud.

[0265]

[0231] The sensor 1404 may also detect other attributes of the object 1408, such as color and reflectance information. In the example of Figure 14, the point cloud generator 1416 may generate a point cloud based on the signal 1414 generated by the sensor 1404. The distance measurement system 1400 and / or the point cloud generator 1416 may form part of the data source 104 (Figure 1). Thus, the point cloud generated by the distance measurement system 1400 can be encoded and / or decoded according to any of the techniques of this disclosure.

[0266] FIG. 15 is a conceptual diagram illustrating an example vehicle-based scenario in which one or more techniques of the present disclosure may be used. In the example of FIG. 15, a vehicle 1500 includes a distance measurement system 1502. The distance measurement system 1502 may be implemented in the manner discussed with respect to FIG. 15. Although not shown in the example of FIG. 15, the vehicle 1500 may also include a data source, such as the data source 104 (FIG. 1), and a G-PCC encoder, such as the G-PCC encoder 200 (FIG. 1). In the example of FIG. 15, the distance measurement system 1502 emits a laser beam 1504 that reflects off a pedestrian 1506A or other object in a road. The data source of the vehicle 1500 may generate a point cloud based on the signal generated by the distance measurement system 1502. The G-PCC encoder of the vehicle 1500 may encode the point cloud to generate a bit stream 1508, such as a geometry bit stream (FIG. 2) and an attribute bit stream (FIG. 2). The bitstream 1508 may contain many fewer bits than the unencoded point cloud obtained by the G-PCC encoder. In some embodiments, the G-PCC encoder of the vehicle 1500 may encode the bitstream 1508 using radial interpolation as described above. In some embodiments, the G-PCC decoder of the vehicle 1510 may decode the bitstream 1508 using radial interpolation as described above.

[0267]

[0233] An output interface of the vehicle 1500 (e.g., the output interface 108 (FIG. 1)) may transmit the bitstream 1508 to one or more other devices. The bitstream 1508 may include much fewer bits than the unencoded point cloud obtained by the G-PCC encoder. Thus, the vehicle 1500 may be able to transmit the bitstream 1508 to other devices more quickly than the unencoded point cloud data. In addition, the bitstream 1508 may require less data storage capacity.

[0268] In the example of FIG. 15, the vehicle 1500 may transmit a bit stream 1508 to another vehicle 1510. The vehicle 1510 may include a G-PCC decoder, such as the G-PCC decoder 300 (FIG. 1). The G-PCC decoder of the vehicle 1510 may decode the bit stream 1508 to reconstruct a point cloud. The vehicle 1510 may use the reconstructed point cloud for various purposes. For example, the vehicle 1510 may determine based on the reconstructed point cloud that a pedestrian 1506 is on the road in front of the vehicle 1500, and therefore may begin to decelerate, for example, even before the driver of the vehicle 1510 recognizes that the pedestrian 1206 is on the road. Thus, in some embodiments, the vehicle 1510 may perform an autonomous navigation operation based on the reconstructed point cloud.

[0269] Additionally or alternatively, the vehicle 1500 can transmit the bitstream 1508 to the server system 1512. The server system 1512 can use the bitstream 1508 for various purposes. For example, the server system 1512 can store the bitstream 1508 for later reconstruction of a point cloud. In this example, the server system 1512 can use the point cloud along with other data (e.g., vehicle telemetry data generated by the vehicle 1500) to train an autonomous driving system. In other examples, the server system 1512 can store the bitstream 1508 for later reconstruction for forensic crash investigation.

[0270]

[0236] FIG. 16 is a conceptual diagram illustrating an example extended reality system in which one or more techniques of the present disclosure may be used. Extended reality (XR) is a term used to cover a range of technologies including augmented reality (AR), mixed reality (MR), and virtual reality (VR). In the example of FIG. 16, a user 1600 is located at a first location 1602. The user 1600 is wearing an XR headset 1604. Instead of the XR headset 1604, the user 1600 may use a mobile device (e.g., a mobile phone, a tablet computer, etc.). The XR headset 1604 includes a depth detection sensor, such as a distance measurement system, that detects the position of a point on an object 1606 at the location 1602. A data source of the XR headset 1604 may use a signal generated by the depth detection sensor to generate a point cloud representation of the object 1606 at the location 1602. The XR headset 1604 may include a G-PCC encoder (e.g., G-PCC encoder 200 of FIG. 1) configured to encode the point cloud to generate a bitstream 1608. In some embodiments, the G-PCC encoder of the XR headset 1604 may use radial interpolation in encoding the point cloud, as described above.

[0271] The XR headset 1604 may transmit the bitstream 1608 (e.g., over a network such as the Internet) to an XR headset 1610 worn by a user 1612 at a second location 1614. The XR headset 1610 may decode the bitstream 1608 to reconstruct the point cloud. In some examples, the G-PCC decoder of the XR headset 1610 may use radial interpolation in decoding the point cloud, as described above.

[0272]

[0238] The XR headset 1610 may use the point cloud to generate an XR visualization (e.g., an AR, MR, VR visualization) that represents the object 1606 at the location 1602. Thus, in some examples, such as when the XR headset 1610 generates a VR visualization, the user 1612 may have a 3D immersive experience of the location 1602. In some examples, the XR headset 1610 may determine a position of a virtual object based on the reconstructed point cloud. For example, the XR headset 1610 may determine that an environment (e.g., the location 1602) includes a flat surface and determine that a virtual object (e.g., a cartoon character) should be located on the flat surface based on the reconstructed point cloud. The XR headset 1610 may generate an XR visualization with the virtual object in the determined position. For example, the XR headset 1610 may display a cartoon character sitting on the flat surface.

[0273]

[0239] Figure 17 is a conceptual diagram illustrating an example mobile device system in which one or more techniques of the present disclosure may be used. In the example of Figure 17, a mobile device 1700, such as a mobile phone or tablet computer, includes a distance measurement system, such as a LIDAR system, that detects the position of a point on an object 1702 in an environment of the mobile device 1700. A data source of the mobile device 1700 may use a signal generated by a depth detection sensor to generate a point cloud representation of the object 1702. The mobile device 1700 may include a G-PCC encoder (e.g., the G-PCC encoder 200 of Figure 1) configured to encode the point cloud to generate a bitstream 1704.

[0274] In the example of Figure 17, the mobile device 1700 may transmit a bitstream to a remote device 1706, such as a server system or another mobile device. The remote device 1706 may decode the bitstream 1704 to reconstruct the point cloud. In some embodiments, the G-PCC decoder of the remote device 1706 may use radial interpolation in decoding the point cloud, as described above.

[0275]

[0241] The remote device 1706 may use the point cloud for various purposes. For example, the remote device 1706 may use the point cloud to generate a map of the environment of the mobile device 1700. For example, the remote device 1706 may generate a map of the interior of a building based on the reconstructed point cloud. In another example, the remote device 1706 may generate imagery (e.g., computer graphics) based on the point cloud. For example, the remote device 1706 may use the points of the point cloud as vertices of a polygon and use the color attributes of the points as a basis for shading the polygon. In some examples, the remote device 1706 can use the reconstructed point cloud for facial recognition applications or other security applications.

[0276]

[0242] The following numbered clauses may describe one or more aspects of the disclosure.

[0277]

[0243] Clause 1A: A method for processing a point cloud, the method comprising: in response to a decision to predict a current point in a current frame using an inter prediction mode, identifying a zero-compensated reference frame; deriving a motion compensated reference frame; selectively resampling points in the zero-compensated reference frame based on one or more points in the motion compensated reference frame to generate a resampled zero-compensated reference frame; and inter predicting the current point in the current frame based on the resampled zero-compensated reference frame.

[0278]

[0244] Clause 2A: The method of clause 1A, wherein the zero compensated reference frame is ZeroRef and the motion compensated reference frame is GlobRef.

[0279]

[0245] Clause 3A: The method of clause 1A or 2A, wherein selectively resampling a point in the zero compensated reference frame includes selecting a particular point in the zero compensated reference frame, determining whether to resample the particular point, and in response to the decision to resample the particular point, determining a first set of points in the motion compensated reference frame associated with the particular point, deriving a derived radius value based on the first set of points, and updating the radius value of the particular point based on the derived radius value.

[0280]

[0246] Clause 4A: The method of clause 3A, wherein deriving a radius value based on the first set of points includes, when the first set of points includes multiple points, selectively interpolating radius values ​​derived based on the multiple points.

[0281]

[0247] Clause 5A: The method of clause 4A, wherein selectively interpolating the derived radius values ​​includes determining not to interpolate the derived radius values ​​if a difference between radius values ​​of the multiple points is greater than a threshold.

[0282]

[0248] Clause 6A: The method of clause 4A, wherein selectively interpolating the derived radius values ​​includes determining not to interpolate the derived radius values ​​when a difference between azimuth angle values ​​of the multiple points is greater than a threshold.

[0283]

[0249] Clause 7A: The method of any of clauses 4A-6A, further comprising, in response to determining not to interpolate the derived radius values, not updating the radius value of the particular point based on the derived radius values.

[0284]

[0250] Clause 8A: The method of any of clauses 4A to 6A, further comprising, in response to determining not to interpolate the derived radius values, setting the radius value of the particular point as the radius value of a single point of the first set of points.

[0285]

[0251] Clause 9A: The method of clause 8A, wherein a single point in the first set of points has an azimuth angle value that is closest to the azimuth angle value of the particular point.

[0286]

[0252] Clause 10A: A device for processing a point cloud, the device comprising one or more means for performing the methods of any of clauses 1A to 9A.

[0287]

[0253] Clause 11A: The device of clause 10A, wherein the one or more means include one or more processors implemented in circuitry.

[0288]

[0254] Clause 12A: The device of clause 10A or 11A, further comprising a memory for storing data representing the point cloud. Clause 13A: The device of any of clauses 10A to 12A, wherein the device comprises a decoder.

[0289]

[0255] Clause 14A: The device of any one of clauses 10A to 13A, comprising an encoder.

[0290]

[0256] Clause 15A: The device of any one of clauses 10A to 14A, further comprising a device for generating a point cloud.

[0291]

[0257] Clause 16A: The device of any of clauses 10A to 15A, further comprising a display for presenting an image based on the point cloud.

[0292]

[0258] Clause 17A: A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to perform the method of any of clauses 1A to 9A.

[0293]

[0259] Clause 1B: A device for processing point cloud data, comprising: a memory configured to store the point cloud data; and one or more processors implemented in a circuit, wherein the one or more processors are configured to, in response to a decision to predict a current point in a current frame using an inter prediction mode, identify a reference frame, derive one or more motion compensated reference points from the reference frame, selectively resample points in the reference frame based on the one or more motion compensated reference points to generate a resampled reference frame, and inter predict the current point in the current frame based on the resampled reference frame.

[0294]

[0260] Clause 2B: The device described in Clause 1B, wherein to selectively resample a point in the reference frame, the one or more processors are further configured to: select a particular point in the reference frame; decide whether to resample the particular point; in response to a decision to resample the particular point, determine a first set of points in the reference frame associated with the particular point, the first set of points including at least one of the one or more motion compensated reference points; derive a derived radius value based on the first set of points; and update the radius value of the particular point based on the derived radius value.

[0295]

[0261] Clause 3B: The device of Clause 2B, wherein to derive a derived radius value based on the first set of points, the one or more processors are further configured to selectively interpolate radius values ​​for a plurality of points to determine the derived radius value, the first set of points including a plurality of points.

[0296]

[0262] Clause 4B: The device described in Clause 3B, wherein, to selectively interpolate radius values ​​for a plurality of points, the one or more processors are further configured to determine not to interpolate radius values ​​for the first point and the second point in response to one or both of a difference in radius values ​​between a first point and a second point among the plurality of points being greater than a first threshold or a difference in azimuth angle values ​​between the first point and the second point being greater than a second threshold.

[0297]

[0263] Clause 5B: The device described in Clause 4B, wherein the one or more processors are further configured to, in response to a decision not to interpolate radius values ​​for a plurality of points, not update a radius value of a particular point based on the derived radius value.

[0298]

[0264] Clause 6B: The device of clause 4B, wherein the one or more processors are further configured to, in response to a decision not to interpolate radius values ​​of multiple points, set the radius value of the particular point as the radius value of a single point of the first set of points.

[0299]

[0265] Clause 7B: The device of clause 6B, wherein a single point of the first set of points has an azimuth angle value closest to the azimuth angle value of the particular point.

[0300]

[0266] Clause 8B: The device of any of clauses 1B to 7B, wherein the one or more processors are further configured to reconstruct a point cloud that includes the current point to form a reconstructed point cloud.

[0301]

[0267] Clause 9B: The device of clause 8B, wherein the one or more processors are further configured to generate a map of the interior of the building based on the reconstructed point cloud.

[0302]

[0268] Clause 10B: The device of clause 8B or 9B, wherein the one or more processors are further configured to perform autonomous navigation operations based on the reconstructed point cloud.

[0303]

[0269] Clause 11B: The device of any one of clauses 8B to 10B, wherein the one or more processors are further configured to generate computer graphics based on the reconstructed point cloud.

[0304]

[0270] Clause 12B: A device of any of clauses 8B to 11B, wherein the one or more processors are configured to determine a position of a virtual object based on the reconstructed point cloud and generate an extended reality (XR) visualization in which the virtual object is at the determined position.

[0305]

[0271] Clause 13B: The device of any of clauses 8B to 12B, further comprising a display for presenting an image based on the reconstructed point cloud.

[0306]

[0272] Clause 14B: Any of the devices of clauses 1B to 13B, wherein the device includes one or more of a mobile phone, a tablet computer, a vehicle, or an extended reality device.

[0307]

[0273] Clause 15B: A method for processing a point cloud, comprising: in response to a decision to predict a current point in a current frame using an inter prediction mode, identifying a reference frame; deriving one or more motion compensated reference points from the reference frame; selectively resampling points in the reference frame based on the one or more motion compensated reference points to generate a resampled reference frame; and inter predicting the current point in the current frame based on the resampled reference frame.

[0308]

[0274] Clause 16B: The method of clause 15B, wherein selectively resampling a point in the reference frame includes selecting a particular point in the reference frame, determining whether to resample the particular point, and in response to the decision to resample the particular point, determining a first set of points in the reference frame associated with the particular point, the first set of points including at least one of the one or more motion compensated reference points, deriving a derived radius value based on the first set of points, and updating the radius value of the particular point based on the derived radius value.

[0309]

[0275] Clause 17B: The method of clause 16B, wherein deriving a derived radius value based on a first set of points includes selectively interpolating radius values ​​for a plurality of points to determine the derived radius value, the first set of points including a plurality of points.

[0310]

[0276] Clause 18B: The method of clause 17B, wherein selectively interpolating the derived radius values ​​includes determining not to interpolate radius values ​​for the first point and the second point in response to one or both of a difference in radius values ​​between a first point and a second point of the plurality of points being greater than a first threshold or a difference in azimuth angle values ​​between the first point and the second point being greater than a second threshold.

[0311]

[0277] Clause 19B: The method of clause 18B, further comprising, in response to determining not to interpolate radius values ​​for the plurality of points, not updating a radius value for the particular point based on the derived radius value.

[0312]

[0278] Clause 20B: The method of clause 18B, further comprising, in response to a decision not to interpolate radius values ​​of multiple points, setting the radius value of the particular point as the radius value of a single point of the first set of points.

[0313]

[0279] Clause 21B: The method of clause 20B, wherein a single point in the first set of points has an azimuth angle value that is closest to the azimuth angle value of the particular point.

[0314]

[0280] Clause 22B: The method of any one of clauses 15B to 21B, wherein the method for processing a point cloud includes a method for encoding a point cloud.

[0315]

[0281] Clause 23B: The method of any one of clauses 15B to 21B, wherein the method of processing a point cloud includes a method of decoding a point cloud.

[0316]

[0282] Clause 24B: A computer-readable storage medium storing instructions which, when executed by one or more processors, cause the one or more processors to, in response to a decision to predict a current point in a current frame using an inter prediction mode, identify a reference frame, derive one or more motion compensated reference points from the reference frame, selectively resample points in the reference frame based on the one or more motion compensated reference points to generate a resampled reference frame, and inter predict a current point in the current frame based on the resampled reference frame.

[0317]

[0283] Clause 25B: The computer-readable storage medium of clause 24B, wherein the instructions are further configured to: cause one or more processors to select a particular point in the reference frame, determine whether to resample the particular point, and in response to a decision to resample the particular point, determine a first set of points in the reference frame associated with the particular point, the first set of points including at least one of the one or more motion compensated reference points, derive a derived radius value based on the first set of points, and update the radius value of the particular point based on the derived radius value.

[0318]

[0284] Clause 26B: The computer-readable storage medium of clause 25B, wherein the instructions cause one or more processors to selectively interpolate radius values ​​for a plurality of points to determine a derived radius value based on the first set of points, the first set of points including a plurality of points.

[0319]

[0285] Clause 27B: The computer-readable storage medium of clause 26B, wherein the instructions cause one or more processors to determine not to interpolate radius values ​​for a first point and a second point of the plurality of points in response to one or both of a difference in radius values ​​between a first point and a second point of the plurality of points being greater than a first threshold or a difference in azimuth angle values ​​between the first point and the second point being greater than a second threshold.

[0320]

[0286] Clause 28B: The computer-readable storage medium of clause 27B, wherein the instructions further cause the one or more processors to not update the radius value of a particular point based on the derived radius value in response to a determination not to interpolate radius values ​​for the multiple points.

[0321]

[0287] Clause 29B: The instructions further cause the one or more processors to set the radius value of a particular point as the radius value of a single point of the first set of points in response to a determination not to interpolate radius values ​​of multiple points.

[0322]

[0288] Clause 30B: The computer-readable storage medium of clause 29B, wherein a single point of the first set of points has an azimuth angle value closest to the azimuth angle value of the particular point.

[0323] It should be recognized that in some examples, some acts or events of any of the techniques described herein may be performed in a different order, or may be added, merged, or omitted entirely (e.g., not all acts or events described may be required to practice the techniques). Moreover, in some examples, acts or events may be performed in parallel rather than sequentially, for example, through multithreaded processing, interrupt processing, or multiple processors.

[0324]

[0290] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium, such as a data storage medium, or a communication medium including any medium that facilitates transfer of a computer program from one place to another, for example according to a communication protocol. As such, a computer-readable medium may generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures to implement the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0325]

[0291] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included within the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but instead cover non-transitory tangible storage media. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer readable media.

[0326]

[0292] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the terms "processor" and "processing circuitry" as used herein may refer to any of the above structures or any other structures suitable for implementing the techniques described herein. In addition, in some aspects, the functions described herein may be provided in dedicated hardware and / or software modules configured for encoding and decoding, or may be incorporated into a composite codec. The techniques may also be fully implemented in one or more circuits or logic elements.

[0327]

[0293] The techniques of the present disclosure may be implemented in a wide variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chipsets). Various components, modules, or units have been described in the present disclosure to highlight functional aspects of devices configured to implement the disclosed techniques, but they do not necessarily require realization by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit or may be provided by a collection of interoperable hardware units, including one or more processors as described above, in conjunction with suitable software and / or firmware.

[0328]

[0294] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. A device for processing point cloud data, wherein the device is A memory configured to store the aforementioned point cloud data, The circuit comprises one or more processors implemented in the circuit, wherein the one or more processors respond to a decision to predict the current point in the current frame using an interprediction mode. Identify the reference frame, One or more motion-compensated reference points are derived from the aforementioned reference frame. To generate a resampled reference frame, points within the reference frame are selectively resampled based on the one or more motion-compensated reference points. A device configured to interpret the current point in the current frame based on the resampled reference frame.

2. In order to selectively resample the points in the reference frame, one or more processors, Select a specific point within the aforementioned reference frame, Determine whether or not to resample the aforementioned specific point. In response to the decision to resample the aforementioned specific points, Determine a first set of points in the reference frame associated with the specific point, the first set of points including at least one of the one or more motion-compensated reference points. Based on the first set of points, the derived radius value is derived, The device according to claim 1, further configured to update the radius value of a specific point based on the derived radius value.

3. In order to derive the derived radius value based on the first set of points, one or more processors, The device according to claim 2, wherein, in order to determine the derived radius value, the radius values ​​for a plurality of points are selectively interpolated, and the first set of points is further configured to include the plurality of points.

4. In order to selectively interpolate the radius values ​​for the plurality of points, one or more processors, The difference in radius values ​​between the first point and the second point among the aforementioned plurality of points is greater than the first threshold, and / or The difference in azimuth angle values ​​between the first point and the second point is greater than the second threshold. The device according to claim 3, further configured to determine not to interpolate radius values ​​for the first point and the second point in response to the above.

5. The aforementioned one or more processors are The device according to claim 4, further configured not to update the radius value of a particular point based on the derived radius value in response to a decision not to interpolate the radius values ​​for the plurality of points.

6. The aforementioned one or more processors are The device according to claim 4, further configured to set the radius value of a particular point as the radius value of a single point in a first set of points, in response to a decision not to interpolate the radius values ​​for the plurality of points.

7. The device according to claim 6, wherein the single point among the first set of points has the azimuth angle value closest to the azimuth angle value of the particular point.

8. In order to selectively resample the points in the reference frame, the one or more processors: Selecting a point in the reference frame for resampling based on the azimuth angle value of the compensated reference point and the sampling rate of the sensor associated with the point cloud data, Correcting the position value of the selected motion-compensated reference point based on the sampling rate, The device according to claim 1, further configured to perform the following:

9. The device according to claim 1, wherein the one or more processors are further configured to reconstruct a point cloud including the current point to form a reconstructed point cloud.

10. The aforementioned one or more processors are To generate a map of the building's interior based on the reconstructed point cloud, Performing autonomous navigation operations based on the reconstructed point cloud, or To generate computer graphics based on the reconstructed point cloud, The device according to claim 9, further configured to perform the following:

11. The aforementioned one or more processors are Based on the reconstructed point cloud, the position of the virtual object is determined. The system is configured to generate an Extended Reality (XR) visualization in which the virtual object is located at the determined position. The device according to claim 9.

12. The device according to claim 9, further comprising a display for presenting an image based on the reconstructed point cloud.

13. The device according to claim 1, wherein the device includes one or more of a mobile phone, a tablet computer, a vehicle, or an extended reality device.

14. A method for encoding or decoding point cloud data using an interprediction mode, Using the interprediction mode, in response to a decision to predict the current point in the current frame, Identifying the reference frame, Deriving one or more motion-compensated reference points from the aforementioned reference frame, To generate a resampled reference frame, selectively resample points within the reference frame based on the one or more motion-compensated reference points, Interpreting the current point in the current frame based on the resampled reference frame, Methods that include...

15. A computer-readable storage medium for storing instructions, wherein when the instructions are executed by one or more processors, the one or more processors, Using the interprediction mode, in response to a decision to predict the current point in the current frame, Identify the reference frame, One or more motion-compensated reference points are derived from the aforementioned reference frame. To generate a resampled reference frame, points within the reference frame are selectively resampled based on the one or more motion-compensated reference points. A computer-readable storage medium that causes the current point in the current frame to interpret based on the resampled reference frame.

Images