Inter prediction coding and decoding for geometric point cloud compression

Abstract

An apparatus for encoding or decoding point cloud data may be configured to: in response to determining to predict a current point in a current frame using an inter prediction mode: identify a reference frame; deriving one or more motion compensation reference points from the reference frame; selectively resampling points in the reference frame based on the one or more motion compensation reference points to generate a resampled reference frame; and performing inter prediction on the current point in the current frame based on the resampled reference frame.

Description

[0001] This application claims priority to U.S. Patent Application No. 18 / 303,390 filed on April 19, 2023, U.S. Provisional Patent Application No. 63 / 363,306 filed on April 20, 2022, and U.S. Provisional Patent Application No. 62 / 363,552 filed on April 25, 2022, each of which is incorporated herein by reference in its entirety. U.S. Patent Application No. 18 / 303,390 filed on April 19, 2023 claims the benefit of U.S. Provisional Patent Application No. 63 / 363,306 filed on April 20, 2022, and U.S. Provisional Patent Application No. 62 / 363,552 filed on April 25, 2022. Technical Field

[0002] The present disclosure relates to point cloud encoding and decoding. Background Art

[0003] A point cloud is a collection of points in a three-dimensional space. These points can correspond to points on objects within the three-dimensional space. Therefore, a point cloud can be used to represent the physical contents of a three-dimensional space. Point clouds are useful in many situations. For example, a point cloud can be used in the context of an autonomous vehicle to represent the location of objects on a road. In another example, a point cloud can be used in the context of representing the physical contents of an environment in order to locate virtual objects in augmented reality (AR) or mixed reality (MR) applications. Point cloud compression is the process used to encode and decode point clouds. Encoding a point cloud can reduce the amount of data required to store and transmit the point cloud. Summary of the invention

[0004] In general, the present disclosure describes techniques for encoding and decoding nodes of a point cloud using inter-frame prediction, such as for the Geometric Point Cloud Compression (G-PCC) standard currently under development. However, the example techniques are not limited to the G-PCC standard. In some examples of G-PCC, the coordinates of the position of a node (also referred to as a point) of a point cloud can be converted into an (r, φ, i) domain, where the position of the node is represented by three parameters, namely, a radius r, an azimuth φ, and a laser index i (e.g., a laser identifier). When an angular mode is used for predictive geometric encoding and decoding in G-PCC, the G-PCC codec can perform prediction in the (r, φ, i) domain. For example, in order to encode and decode a specific node of a specific frame of a point cloud, the G-PCC codec can determine a predicted value of one or more of the radius r, azimuth φ, and laser index i of the specific node based on another node of the specific frame, and add the predicted value to the residual data (e.g., a residual radius r, a residual azimuth φ, or a residual laser index i) to determine the reconstructed radius r, azimuth φ, or laser index i of the specific node. Since encoding and decoding the residual data may occupy a large part of the codec overhead, the codec efficiency (e.g., the amount of bits used to encode and decode a point) may be affected by how close the predicted value (e.g., predicted radius r, azimuth φ, or laser index i) of a particular node is to the reconstructed value (e.g., reconstructed radius r, azimuth φ, or laser index i) of the particular node. Therefore, generating more accurate prediction parameters for a node can improve the overall codec performance.

[0005] According to one or more techniques of the present disclosure, the G-PCC codec can use inter-frame prediction to predict the current point of the current frame of the point cloud. For example, in order to predict the current point using inter-frame prediction, the G-PCC codec can identify a reference point in a frame different from the current frame, and predict one or more parameters of the current point (e.g., radius r, azimuth φ, and laser index i) based on one or more parameters of the reference point. In a coding scenario where the parameters of the reference point are closer to the parameters of the current point than the parent point or other available points in the current frame, using inter-frame prediction to predict the current point can reduce the size of the residual data. In this way, the inter-frame prediction technology of the present disclosure can enable the G-PCC codec to improve coding efficiency.

[0006] According to an example of the present disclosure, a device for processing (e.g., encoding or decoding) point cloud data includes: a memory configured to store point cloud data; and one or more processors implemented in a circuit and configured to: in response to determining that an inter-frame prediction mode is used to predict a current point in a current frame: identify a reference frame; derive one or more motion compensation reference points from the reference frame; selectively resample a point in the reference frame based on the one or more motion compensation reference points to generate a resampled reference frame; and inter-frame predict the current point in the current frame based on the resampled reference frame.

[0007] According to an example of the present disclosure, a method for processing (e.g., encoding or decoding) a point cloud includes: in response to determining to use an inter-frame prediction mode to predict a current point in a current frame: identifying a reference frame; deriving one or more motion compensation reference points from the reference frame; selectively resampling points in the reference frame based on the one or more point motion compensation reference points to generate a resampled reference frame; and inter-predicting the current point in the current frame based on the resampled reference frame.

[0008] A computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to: in response to determining that an inter-frame prediction mode is used to predict a current point in a current frame: identify a reference frame; derive one or more motion compensated reference points from the reference frame; selectively resample a point in the reference frame based on the one or more motion compensated reference points to generate a resampled reference frame; and inter-predict the current point in the current frame based on the resampled reference frame.

[0009] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a block diagram illustrating an example encoding and decoding system that may perform the techniques of this disclosure.

[0011] Figure 2 is a block diagram illustrating an example Geometric Point Cloud Compression (G-PCC) encoder according to one or more aspects of the present disclosure.

[0012] Figure 3 is a block diagram illustrating an example G-PCC decoder according to one or more aspects of the present disclosure.

[0013] Figure 4 is a conceptual diagram illustrating octree partitioning for geometry coding according to one or more aspects of the present disclosure.

[0014] Figure 5A and Figure 5B is a conceptual diagram of a rotating light detection and ranging (LIDAR) acquisition model according to one or more aspects of the present disclosure.

[0015] Figure 6 is a conceptual diagram of a prediction tree for predictive geometry coding according to one or more aspects of the present disclosure.

[0016] Figure 7 is a conceptual diagram illustrating an example inter-prediction process for predicting points of a point cloud in accordance with one or more aspects of the present disclosure.

[0017] Figure 8 is a flow chart illustrating an example decoding flow associated with a syntax element indicating whether a node is encoded or decoded in an inter-prediction mode or an intra-prediction mode.

[0018] Fig. 9 is a conceptual diagram showing additional inter prediction value points obtained from a first point having an orientation greater than an inter prediction value point.

[0019] Fig.10 is a flow chart illustrating an example technique of how to use resample points for inter prediction in accordance with one or more techniques of this disclosure.

[0020] Fig.11 is a flow chart illustrating an example technique of how to resample points in a global compensated reference frame using points from a zero compensated reference frame in accordance with one or more techniques of this disclosure.

[0021] Figures 12A-12D is a conceptual diagram illustrating how to select a first set of points in a global compensation reference frame for points in a zero compensation reference frame according to one or more aspects of the present disclosure.

[0022] Fig.13 is a flow chart illustrating a process according to the technology of the present disclosure.

[0023] Fig.14 is a conceptual diagram illustrating an example ranging system that may be used with one or more techniques of this disclosure.

[0024] Fig.15 is a conceptual diagram illustrating an example vehicle-based scenario in which one or more techniques of this disclosure may be used.

[0025] Fig.16 is a conceptual diagram illustrating an example extended reality system in which one or more techniques of this disclosure may be used.

[0026] Fig.17is a conceptual diagram illustrating an example mobile device system in which one or more techniques of this disclosure may be employed. DETAILED DESCRIPTION

[0027] When using angular mode for predictive geometry codec in G-PCC, the G-PCC codec can perform prediction in the angular domain (also referred to as the (r, φ, i) domain). For example, to encode and decode a specific node of a specific frame of a point cloud, the G-PCC codec can determine a predicted value of one or more of the radius r, azimuth φ, or laser index i of the specific node based on another node of the specific frame, and add the predicted radius r, azimuth φ, or laser index i of the node to the residual data (e.g., residual radius r, residual azimuth φ, or residual laser index i) to determine the reconstructed radius r, azimuth φ, or laser index i of the specific node. Since encoding and decoding the residual data may result in significant codec overhead, the codec efficiency (e.g., the amount of bits used to encode and decode the point) may be affected by how close the predicted radius r, azimuth φ, and laser index i of the specific node are to the reconstructed radius r, azimuth φ, and laser index i of the specific node. Therefore, generating more accurate prediction parameters for a node can improve overall codec performance.

[0028] According to one or more techniques of the present disclosure, the G-PCC codec can use inter-frame prediction to predict the current point of the current frame of the point cloud. For example, in order to predict the current point using inter-frame prediction, the G-PCC codec can identify a reference point in a frame different from the current frame, and predict one or more parameters of the current point (e.g., radius r, azimuth φ, and laser index i) based on one or more parameters of the reference point. In a coding scenario where the parameters of the reference point are closer to the parameters of the current point than the parent point or other available points in the current frame, using inter-frame prediction to predict the current point can reduce the size of the residual data compared to intra-frame prediction. In this way, the inter-frame prediction technology of the present disclosure can enable the G-PCC codec to improve coding efficiency.

[0029] As will be explained in more detail below, inter-frame prediction uses points in a reference frame to predict points in the current frame. In some examples, a zero motion compensated reference frame, also referred to as a zero compensated reference frame, may be used, and in some cases, a global motion compensated reference frame may be used for prediction. In a zero compensated frame, points in a reference frame are used without applying motion compensation. Therefore, in the present disclosure, a zero motion compensated reference frame may sometimes also be simply referred to as a reference frame. A global motion compensated reference frame may be obtained from a (zero compensated) reference frame by performing motion compensation based on a set of motion parameters. Motion parameters may be derived, for example, using motion estimation techniques, or obtained from an external source (such as using GPS parameters, or from the motion of a car capturing a point cloud). When motion compensation is applied to a point in a reference frame, the position of the point is modified, and this may result in a reduction in encoding and decoding efficiency.

[0030] The reference frame contains points, and each point is associated with a reconstructed position. The reconstructed position is the position of the point relative to a first position in the reference frame. For example, if the point is captured by a light detection and ranging (LIDAR) sensor mounted on a vehicle, the first position may indicate a position associated with the vehicle, such as the position of a particular point in the vehicle / sensor. The position of the point in the reference frame may be relative to the first position. The vehicle may have moved compared to the reference frame, and therefore, the second position may be associated with the vehicle in the current frame so that the position of the point in the current frame is relative to the second position. If there is motion between the reference frame and the current frame, the first position and the second position are different. In this case, if motion compensation is applied, the position of one or more points in the reference frame (relative to the first position) may be modified so that the positions of these points are relative to the second position. For example, assume that the vehicle is at position (1, 1, 1) in the reference frame and moves to position (3, 4, 5) in the second frame. The first position is (1, 1, 1) and the second position is (3, 4, 5). The motion parameters in this example are only translations, and the motion vector is (2, 3, 4) (obtained by subtracting 1, 1, 1 from 3, 4, 5, respectively). Assume that point X in the reference frame has position (a, b, c) relative to the first position. After motion compensation, the position is modified to (a-2, b-3, c-4). This can be thought of as the position of point X relative to the second position.

[0031] The above example only contains translations and is described in the Cartesian domain. However, motion parameters can also include rotations. For example, assume that in frame F1 there are three consecutive points P1, P2, and P3 captured by the laser of a LIDAR sensor. Since the sampling rate of the LIDAR sensor laser is usually constant, the azimuth difference between consecutive points of the laser is the same (e.g., delAzim). Therefore, the azimuth difference between P1 and P2 is the same as the azimuth difference between P2 and P3 (excluding noise), both of which are equal to delAzim. Consider another frame F2, where F1 is the reference frame for F2. When motion compensation is applied to the points in F1, the positions of points P1, P2, and P3 are modified. Due to the operation of motion compensation (rotation, and in some cases translation), the difference in azimuth values ​​between the compensated point of P1 and the compensated point of P2 may not be equal to delAzim. Similarly, the difference in azimuth values ​​between the compensated point of P2 and the compensated point of P3 may not be equal to delAzim. When these compensation points are used for inter-frame prediction of points in the current frame F2, where consecutive points of the laser have an azimuth difference of delAzim, this results in suboptimal prediction values, because the compensation points of P1, P2, and P3 may not correspond to any point in the current frame F2. This results in low codec efficiency.

[0032] This disclosure describes techniques for modifying the compensated points of P1, P2, and P3 so that the points fall back onto a regular sampling grid. This can be performed by looking at the position values ​​of the points in frame F1 (i.e., the zero-compensated reference frame) and deriving new position values ​​corresponding to these position values ​​from the compensated positions of the points in frame F1. This results in the points in the reference frame being closer to the sampling grid of the sensor, and therefore, the predicted values ​​being closer to the points in the current frame. It must be noted that due to the presence of noise, the point positions captured by the LIDAR sensor are typically not on a regular sampling grid; however, in most cases, the sampling pattern and frequency across frames are the same or nearly the same.

[0033] According to the technology of the present disclosure, the G-PCC codec can derive one or more motion compensation reference points from a reference frame in response to determining that a current point in a current frame is encoded and decoded using an inter-frame prediction mode, and selectively resample a point in the reference frame based on the one or more motion compensation reference points to generate a resampled reference frame. By inter-predicting the current point in the current frame based on the resampled reference frame, the G-PCC codec can perform more accurate inter-frame prediction, which reduces the bits required to signal residual data and thus improves compression without increasing distortion.

[0034] Figure 11 is a block diagram illustrating an example encoding and decoding system 100 that can perform the techniques of the present disclosure. The techniques of the present disclosure generally relate to encoding and decoding (encoding and / or decoding) point cloud data, i.e., supporting point cloud compression. In general, point cloud data includes any data used to process point clouds. The codec is effective in compressing and / or decompressing point cloud data.

[0035] like Figure 1 As shown, system 100 includes source device 102 and destination device 116. Source device 102 provides encoded point cloud data to be decoded by destination device 116. In particular, Figure 1 In the example of , source device 102 provides point cloud data to destination device 116 via computer-readable medium 110. Source device 102 and destination device 116 may include any of a variety of devices, including desktop computers, notebook (i.e., laptop) computers, tablet computers, set-top boxes, handheld phones (such as smart phones), televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, land or sea vehicles, spacecraft, aircraft, robots, LIDAR devices, satellites, etc. In some cases, source device 102 and destination device 116 may be equipped for wireless communication.

[0036] exist Figure 1 In the example of , the source device 102 includes a data source 104, a memory 106, a G-PCC encoder 200, and an output interface 108. The destination device 116 includes an input interface 122, a G-PCC decoder 300, a memory 120, and a data consumer 118. According to the present disclosure, the G-PCC encoder 200 of the source device 102 and the G-PCC decoder 300 of the destination device 116 can be configured to apply the technology related to predictive geometry coding and decoding of the present disclosure. Thus, the source device 102 represents an example of an encoding device, and the destination device 116 represents an example of a decoding device. In other examples, the source device 102 and the destination device 116 may include other components or arrangements. For example, the source device 102 can receive data (e.g., point cloud data) from an internal or external source. Similarly, the destination device 116 can interface with an external data consumer instead of including the data consumer in the same device.

[0037] like Figure 1The system 100 shown is only an example. In general, other digital encoding and / or decoding devices can perform the technology related to predictive geometry coding and decoding of the present disclosure. The source device 102 and the destination device 116 are only examples of such devices, wherein the source device 102 generates decoded data for transmission to the destination device 116. The present disclosure refers to a "decoding" device as a device that performs decoding (e.g., encoding and / or decoding) of data. Therefore, the G-PCC encoder 200 and the G-PCC decoder 300 represent examples of decoding devices (specifically, encoders and decoders) respectively. In some examples, the source device 102 and the destination device 116 can operate in a substantially symmetrical manner so that each of the source device 102 and the destination device 116 includes encoding and decoding components. Therefore, the system 100 can support one-way or two-way transmission between the source device 102 and the destination device 116, for example, for streaming, playback, broadcasting, telephone, navigation and other applications.

[0038] In general, the data source 104 represents the source of data (i.e., the original, unencoded point cloud data), and can provide a sequential series of "frames" of data to the G-PCC encoder 200, which encodes the data of the frames. The data source 104 of the source device 102 may include a point cloud capture device, such as any of a variety of cameras or sensors, such as a 3D scanner, a LIDAR device, one or more video cameras, an archive containing previously captured data, and / or a data feed interface for receiving data from a data content provider. Alternatively or additionally, the point cloud data can be generated by a computer from a scanner, a camera, a sensor, or other data. For example, the data source 104 can generate computer graphics-based data as source data, or generate a combination of real-time data, archived data, and computer-generated data. In each case, the G-PCC encoder 200 encodes captured, pre-captured, or computer-generated data. The G-PCC encoder 200 can rearrange frames from the receiving order (sometimes referred to as "display order") to the encoding and decoding order for encoding and decoding. The G-PCC encoder 200 can generate one or more bit streams including encoded data. Source device 102 may then output the encoded data onto computer-readable medium 110 via output interface 108 for receipt and / or retrieval by, for example, input interface 122 of destination device 116 .

[0039] The memory 106 of the source device 102 and the memory 120 of the destination device 116 can represent general memory. In some examples, the memory 106 and the memory 120 can store original data, for example, the original data from the data source 104 and the original decoded data from the G-PCC decoder 300. In addition or alternatively, the memory 106 and the memory 120 can store software instructions that can be executed by, for example, the G-PCC encoder 200 and the G-PCC decoder 300, respectively. Although the memory 106 and the memory 120 are shown separately from the G-PCC encoder 200 and the G-PCC decoder 300 in this example, it should be understood that the G-PCC encoder 200 and the G-PCC decoder 300 can also include internal memories that achieve functionally similar or equivalent purposes. In addition, the memory 106 and the memory 120 can store, for example, coded data output from the G-PCC encoder 200 and input to the G-PCC decoder 300. In some examples, portions of memory 106 and memory 120 may be allocated as one or more buffers, for example, to store raw, decoded, and / or encoded data.For example, memory 106 and memory 120 may store data representing a point cloud.

[0040] The computer-readable medium 110 may represent any type of medium or device capable of delivering the encoded data from the source device 102 to the destination device 116. In one example, the computer-readable medium 110 represents a communication medium that enables the source device 102 to directly send the encoded data to the destination device 116 in real time, for example, via a radio frequency network or a computer-based network. According to a communication standard such as a wireless communication protocol, the output interface 108 may modulate a transmission signal including the encoded data, and the input interface 122 may demodulate the received transmission signal. The communication medium may include any wireless or wired communication medium, for example, a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form a portion of a packet-based network such as a local area network, a wide area network, or a global network such as the Internet. The communication medium may include a router, a switch, a base station, or any other device that may be useful for facilitating communication from the source device 102 to the destination device 116.

[0041] In some examples, source device 102 may output the encoded data from output interface 108 to storage device 112. Similarly, destination device 116 may access the encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded data.

[0042] In some examples, source device 102 may output the encoded data to file server 114 or another intermediate storage device that may store the encoded data generated by source device 102. Destination device 116 may access the stored data from file server 114 via streaming or downloading. File server 114 may be any type of server device capable of storing the encoded data and sending the encoded data to destination device 116. File server 114 may represent a web server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Destination device 116 may access the encoded data from file server 114 via any standard data connection (including an Internet connection). This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of the two that is suitable for accessing the encoded data stored on file server 114. File server 114 and input interface 122 may be configured to operate according to a streaming protocol, a downloading transmission protocol, or a combination thereof.

[0043] Output interface 108 and input interface 122 may represent wireless transmitters / receivers, modems, wired networking components (e.g., Ethernet cards), wireless communication components that operate according to any of the various IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transmit data (e.g., encoded data) according to a cellular communication standard (e.g., 4G, 4G-LTE (Long Term Evolution), Advanced LTE, 5G, etc.). In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to transmit data (e.g., encoded data) according to other wireless standards (e.g., IEEE 802.11 specifications, IEEE 802.15 specifications (e.g., ZigBee 5G, etc.). TM ),Bluetooth TMStandards, etc.) to transmit data (such as encoded data). In some examples, source device 102 and / or destination device 116 may include corresponding system-on-chip (SoC) devices. For example, source device 102 may include a SoC device for performing the functions assigned to G-PCC encoder 200 and / or output interface 108, and destination device 116 may include a SoC device for performing the functions assigned to G-PCC decoder 300 and / or input interface 122.

[0044] The techniques of the present disclosure may be applied to encoding and decoding to support any of a variety of applications, such as communications between autonomous vehicles, communications between scanners, cameras, sensors and processing devices such as local or remote servers, geographic mapping, or other applications.

[0045] The input interface 122 of the destination device 116 receives an encoded bitstream from the computer-readable medium 110 (e.g., a communication medium, a storage device 112, a file server 114, etc.). The encoded bitstream may include signaling information defined by the G-PCC encoder 200 and also used by the G-PCC decoder 300, such as syntax elements having values ​​describing characteristics and / or processing of a coding unit (e.g., a slice, a picture, a group of pictures, a sequence, etc.). The data consumer 118 uses the decoded data. For example, the data consumer 118 may use the decoded data to determine the location of a physical object. In some examples, the data consumer 118 may include a display that presents an image based on a point cloud.

[0046] G-PCC encoder 200 and G-PCC decoder 300 can each be implemented as any of various suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is partially implemented in software, the device can store instructions for the software in a suitable non-transitory computer-readable medium, and use one or more processors to execute the instructions in hardware to perform the technology of the present disclosure. Each of G-PCC encoder 200 and G-PCC decoder 300 can be included in one or more encoders or decoders, and any of the encoders or decoders can be integrated as a part of a combined encoder / decoder (CODEC) in a corresponding device. The device including G-PCC encoder 200 and / or G-PCC decoder 300 can include one or more integrated circuits, microprocessors and / or other types of devices.

[0047] The G-PCC encoder 200 and the G-PCC decoder 300 can operate according to a codec standard, such as a video point cloud compression (V-PCC) standard or a geometric point cloud compression (G-PCC) standard. The present disclosure generally relates to the encoding and decoding (e.g., encoding and decoding) of pictures to include the process of encoding or decoding data. The encoded bitstream generally includes a series of values ​​for syntax elements representing codec decisions (e.g., codec modes).

[0048] The present disclosure may generally relate to "signaling" certain information (such as syntax elements). The term "signaling" may generally refer to the transmission of values ​​for syntax elements and / or other data used to decode the encoded data. That is, the G-PCC encoder 200 may signal values ​​for syntax elements in a bitstream. Generally, signaling refers to generating values ​​in a bitstream. As described above, the source device 102 may transmit the bitstream to the destination device 116 in substantially real time or not in real time (e.g., as may occur when storing syntax elements to the storage device 112 for later retrieval by the destination device 116).

[0049] ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) is investigating the potential need for standardization of point cloud codecs with compression capabilities significantly exceeding current methods and will work towards creating a standard. The group is collaborating on this exploratory activity, known as the 3D Graphics Group (3DG), to evaluate compression technology designs proposed by experts in the field.

[0050] Point cloud compression activities are divided into two different approaches. The first approach is "Video Point Cloud Compression" (V-PCC), which segments 3D objects and projects the segments in multiple 2D planes (represented as "patches" in 2D frames), which are further encoded and decoded by traditional 2D video codecs such as the High Efficiency Video Coding (HEVC) (ITU-T H.265) codec. The second approach is "Geometry-based Point Cloud Compression" (G-PCC), which directly compresses the 3D geometry (i.e., the positions of a collection of points in 3D space) and the associated attribute values ​​(for each point associated with the 3D geometry). G-PCC addresses the compression of both Category 1 point clouds (static point clouds) and Category 3 point clouds (dynamically acquired point clouds). A draft of the G-PCC standard is available in G-PCC DIS, ISO / IEC JTC1 / SC29 / WG11 w19328, January 2020, Brussels, Belgium, and a description of the codec is available in G-PCC Codec Description v8, ISO / IECJTC1 / SC29 / WG11 w19525, January 2020, Brussels, Belgium.

[0051] A point cloud contains a collection of points in 3D space and may have attributes associated with those points. The attributes may be color information such as R, G, B or Y, Cb, Cr, or reflectivity information, or other attributes. Point clouds may be captured by various cameras or sensors (such as LIDAR sensors and 3D scanners), and may also be computer generated. Point cloud data is used in a variety of applications, including but not limited to architecture (modeling), graphics (3D models for visualization and animation), and the automotive industry (LIDAR sensors to aid navigation).

[0052] The 3D space occupied by the point cloud data can be enclosed by a virtual bounding box. The positions of the points in the bounding box can be represented with a certain precision; therefore, the positions of one or more points can be quantized based on this precision. At the smallest level, the bounding box is divided into voxels, which are the smallest spatial units represented by a unit cube. A voxel in a bounding box can be associated with zero, one, or more points. The bounding box can be split into multiple cubic / cuboid regions, which can be referred to as tiles. Each tile can be encoded and decoded into one or more strips. The bounding box can be divided into strips and slices based on the number of points in each segmentation, or based on other considerations (for example, a particular area can be encoded and decoded as a slice). The strip area can be further segmented using splitting decisions similar to those in video codecs.

[0053] Figure 2 An overview of the G-PCC encoder 200 is provided. Figure 3 An overview of the G-PCC decoder 300 is provided. The modules shown are logical and do not necessarily correspond one-to-one with implementation code in the reference implementation of the G-PCC codec, ie, the TMC13 test model software of the ISO / IECMPEG (JTC 1 / SC 29 / WG11) study.

[0054] In both the G-PCC encoder 200 and the G-PCC decoder 300, the point cloud positions are first encoded and decoded. The attribute encoding and decoding depends on the decoded geometry. Figure 2 and Figure 3 , modules 212, 218, 310, and 314 are options commonly used for category 1 data. Modules 220, 222, 316, and 318 are options commonly used for category 3 data. All other modules are common between category 1 and category 3.

[0055] For category 3 data, the compressed geometry is typically represented as an octree, from the root down to the leaf level for each voxel. For category 1 data, the compressed geometry is typically represented by a pruned octree (i.e., an octree from the root down to the leaf level for blocks larger than a voxel) plus a model that approximates the surface within each leaf of the pruned octree. In this way, both category 1 and category 3 data share the octree codec mechanism, while category 1 data can also approximate the voxels within each leaf with a surface model. The surface model used is a triangulation, where each block contains 1-10 triangles, resulting in a triangle soup. Therefore, category 1 geometry codecs are called Trisoup geometry codecs, while category 3 geometry codecs are called Octree geometry codecs.

[0056] Figure 4 is a conceptual diagram illustrating an example octree partitioning for geometric coding according to techniques of the present disclosure. Figure 4 In the example shown, the octree 400 can be divided into a series of nodes. For example, each node can be a cube node. At each node of the octree, when the G-PCC decoder 300 does not infer the occupancy of one or more child nodes (including up to eight nodes) for the node, the G-PCC encoder 200 can send the G-PCC decoder 300 point cloud point to the node occupancy with a signal. Specify multiple neighborhoods, which include (a) nodes that share a face with the current octree node, (b) nodes that share a face, edge or vertex with the current octree node, etc. In each neighborhood, the occupancy of a node and / or its child nodes can be used to predict the occupancy of the current node or its child nodes. For points sparsely filled in some nodes of the octree, the codec also supports a direct encoding and decoding mode, in which the 3D position of the point is directly encoded. A flag can be sent with signaling to indicate that a direct mode is sent with signaling. Using the direct mode, the position of the point in the point cloud can be directly encoded and decoded without any compression. At the lowest level, the number of points associated with an octree node / leaf node may also be encoded and decoded.

[0057] Once the geometry is encoded and decoded, the attributes corresponding to the geometry points are also encoded and decoded. When there are multiple attribute points corresponding to a reconstructed / decoded geometry point, the attribute value representing the reconstructed point can be derived.

[0058] There are three attribute codec passes in G-PCC: Region Adaptive Hierarchical Transform (RAHT) codec, interpolation-based hierarchical nearest neighbor prediction (prediction transform), and interpolation-based hierarchical nearest neighbor prediction (lifting transform) with an update / lifting step. RAHT and lifting transform are typically used for category 1 data, while prediction transform is typically used for category 3 data. However, any of the passes can be used for any data, and, similar to the geometry codec in G-PCC, the attribute codec pass used to encode and decode point clouds is specified in the bitstream.

[0059] The encoding and decoding of attributes can be performed in levels of detail (LOD), where for each level of detail, a finer representation of the point cloud attributes is available. Each level of detail can be specified based on a distance metric to neighboring nodes or based on a sampling distance.

[0060] At the G-PCC encoder 200, the residual obtained as an output of the coding process for the attribute is quantized (eg, by one of the arithmetic coding units 214 and / or 226). The quantized residual may be coded using context adaptive arithmetic coding.

[0061] exist Figure 2 In the example, the G-PCC encoder 200 may include a coordinate transformation unit 202, a color transformation unit 204, a voxelization unit 206, an attribute transfer unit 208, an octree analysis unit 210, a surface approximation analysis unit 212, an arithmetic coding unit 214, a geometric reconstruction unit (GRU) 216, a RAHT unit 218, an LOD generation unit 220, a lifting unit 222, a coefficient quantization unit 224 and an arithmetic coding unit 226.

[0062] like Figure 2 As shown in the example of , the G-PCC encoder 200 can receive a set of positions and a set of attributes. The position can include the coordinates of a point in the point cloud. The attribute can include information about the point in the point cloud, such as a color associated with the point in the point cloud.

[0063] The coordinate transformation unit 202 may apply a transformation to the coordinates of the point to transform the coordinates from the initial domain to the transformed domain. The present disclosure may refer to the transformed coordinates as transformed coordinates. The color transformation unit 204 may apply a transformation to transform the color information of the attribute to a different domain. For example, the color transformation unit 204 may transform the color information from the RGB color space to the YCbCr color space.

[0064] In addition, Figure 2In the example of , the voxelization unit 206 can voxelize the transformed coordinates. Voxelization of the transformed coordinates can include quantizing and removing some points of the point cloud. In other words, multiple points of the point cloud can be subsumed into a single "voxel", which can then be treated as a point in some aspects. In addition, the octree analysis unit 210 can generate an octree based on the voxelized transformed coordinates. Additionally, in Figure 2 In the example of FIG. 2 , the surface approximation analysis unit 212 can analyze the points to potentially determine a surface representation of the set of points. The arithmetic coding unit 214 can entropy encode syntax elements that represent information about the octree and / or the surface determined by the surface approximation analysis unit 212. The G-PCC encoder 200 can output these syntax elements in the geometry bitstream 205.

[0065] The geometric reconstruction unit 216 may reconstruct the transformed coordinates of the points in the point cloud based on the octree, the data representing the surface determined by the surface approximation analysis unit 212, and / or other information. Due to voxelization and surface approximation, the number of transformed coordinates reconstructed by the geometric reconstruction unit 216 may be different from the number of original points of the point cloud. The present disclosure may refer to the resulting points as reconstructed points. The attribute transfer unit 208 may transfer the attributes of the original points of the point cloud to the reconstructed points of the point cloud. Figure 2 As shown, the attribute transfer unit 208 may transfer the attributes to one or both of the RAHT unit 218 and the LOD generation unit 220 .

[0066] In addition, the RAHT unit 218 may apply RAHT coding to the attributes of the reconstruction point. Alternatively or additionally, the LOD generation unit 220 and the lifting unit 222 may apply LOD processing and lifting, respectively, to the attributes of the reconstruction point. The RAHT unit 218 and the lifting unit 222 may generate coefficients based on the attributes. The coefficient quantization unit 224 may quantize the coefficients generated by the RAHT unit 218 or the lifting unit 222. The arithmetic coding unit 226 may apply arithmetic coding to the syntax elements representing the quantized coefficients. The G-PCC encoder 200 may output these syntax elements in the attribute bitstream.

[0067] exist Figure 3 In the example, the G-PCC decoder 300 may include a geometric arithmetic decoding unit 302, an attribute arithmetic decoding unit 304, an octree synthesis unit 306, an inverse quantization unit 308, a surface approximation synthesis unit 310, a geometric reconstruction unit 312, a RAHT unit 314, a LoD generation unit 316, an inverse lifting unit 318, an inverse transform coordinate unit 320 and an inverse transform color unit 322.

[0068] The G-PCC decoder 300 may obtain a geometry bitstream and an attribute bitstream. A geometry arithmetic decoding unit 302 of the decoder 300 may apply arithmetic decoding (e.g., context adaptive binary arithmetic coding and decoding (CABAC) or other types of arithmetic decoding) to syntax elements in the geometry bitstream. Similarly, an attribute arithmetic decoding unit 304 may apply arithmetic decoding to syntax elements in the attribute bitstream.

[0069] The octree synthesis unit 306 may synthesize the octree based on the syntax elements parsed from the geometry bitstream. In the case where surface approximation is used in the geometry bitstream, the surface approximation synthesis unit 310 may determine the surface model based on the syntax elements parsed from the geometry bitstream and based on the octree.

[0070] Furthermore, the geometric reconstruction unit 312 may perform reconstruction to determine the coordinates of the points in the point cloud. The inverse transformed coordinates unit 320 may apply an inverse transformation to the reconstructed coordinates to convert the reconstructed coordinates (positions) of the points in the point cloud from the transformed domain back to the original domain.

[0071] Additionally, in Figure 3 In the example of , the inverse quantization unit 308 may inverse quantize the property value. The property value may be based on a syntax element obtained from the property bitstream (eg, including a syntax element decoded by the property arithmetic decoding unit 304). Figure 3 As shown, the inverse quantization unit 308 may pass the property value to one or both of the RAHT unit 314 and the LOD generation unit 316 .

[0072] Depending on how the attribute values ​​are encoded, the RAHT unit 314 may perform RAHT encoding and decoding to determine color values ​​of points of the point cloud based on the inverse quantized attribute values. Alternatively, the LOD generation unit 316 and the inverse lifting unit 318 may use a level of detail based technique to determine color values ​​of points of the point cloud.

[0073] In addition, Figure 3 In the example of , the inverse color transform unit 322 can apply an inverse color transform to the color values. The inverse color transform can be the inverse of the color transform applied by the color transform unit 204 of the G-PCC encoder 200. For example, the color transform unit 204 can transform the color information from the RGB color space to the YCbCr color space. Correspondingly, the inverse color transform unit 322 can transform the color information from the YCbCr color space to the RGB color space.

[0074] Shows Figure 2 and Figure 3The various units of the G-PCC encoder 200 and the G-PCC decoder 300 are provided to help understand the operations performed by the G-PCC encoder 200 and the G-PCC decoder 300. These units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide specific functions and are pre-set with respect to the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functions with the operations that can be performed. For example, a programmable circuit can execute software or firmware, and the software or firmware causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (for example, to receive parameters or output parameters), but the types of operations performed by fixed-function circuits are generally immutable. In some examples, one or more of the units may be different circuit blocks (fixed-function or programmable), and in some examples, one or more of the units may be integrated circuits.

[0075] Predictive geometry codecs are introduced as an alternative to octree geometry codecs, where nodes are arranged in a tree structure (which defines a prediction structure) and various prediction strategies are used to predict the coordinates of each node in the tree relative to its predicted value. Figure 6 600 is a conceptual diagram showing an example of a prediction tree 600, which is a directed graph in which arrows point in the prediction direction. Horizontally shaded nodes are root vertices and have no predicted values; grid shaded nodes have two child nodes; diagonally shaded nodes have 3 child nodes; non-shaded nodes have one child node, and vertically shaded nodes are leaf nodes, and these nodes have no child nodes. Each node has only one parent node.

[0076] Four prediction strategies can be specified for each node based on its parent node (p0), grandparent node (p1), and great-grandparent node (p2). The prediction strategies include: no prediction, incremental prediction (p0), linear prediction (2*p0-p1), and parallelogram prediction (p0+pl-p2).

[0077] The encoder (e.g., G-PCC encoder 200) can use any algorithm to generate the prediction tree; the algorithm used can be determined based on the application / use case, and several strategies can be used. For each node, the encoder can encode the residual coordinate values ​​in the bitstream starting from the root node in a depth-first manner. Predictive geometry coding can be particularly useful for category 3 (LIDAR acquired) point cloud data (e.g., for low latency applications).

[0078] The angular mode can be used for predictive geometry codec, where the characteristics of the LIDAR sensor can be used to encode and decode the prediction tree more efficiently. The coordinates of the position are converted into (r, φ, i) (radius, azimuth and laser index) and the prediction is performed in this domain (the residuals are encoded and decoded in the r, φ, i domain). Due to errors in rounding, the encoding and decoding in r, φ, i is not lossless, and therefore a second set of residuals corresponding to Cartesian coordinates can be encoded and decoded. The description of the encoding and decoding strategy for the angular mode for predictive geometry codec is reproduced below. The description is based on Figure 5A and Figure 5B , which are conceptual diagrams of the rotating LIDAR acquisition model.

[0079] The technology of the present disclosure can be applied to at least a point cloud acquired using a rotating lidar model. Here, the lidar 502 has N lasers (e.g., N=16, 32, 64) rotating around the Z axis according to an azimuth angle φ (see Figure 5A and Figure 5B ). Each laser can have a different elevation angle θ(i) i＝1...N and height Laser I hit according to Figure 5A The coordinate system 500 described in defines a point M having Cartesian integer coordinates (x, y, z).

[0080] The position of M is modeled with three parameters (r, φ, i), which can be calculated as follows: ● φ=atan2(y,x) ●

[0081] The encoding and decoding process can use the quantized version of (r, φ, i), expressed as Among them, three integers and i can be calculated as follows: ● ● ● in ●(q r , o r ) and (q φ , o φ ) are control and The quantization parameter of the accuracy. ●sign(t) is a function that returns 1 if t is positive, and (-1) otherwise. ●|t| is the absolute value of t.

[0082] To avoid reconstruction mismatches due to the use of floating point operations, andtan(θ(i)) i＝1...N The value of can be precomputed and quantized as follows: in ● and (q θ , o θ ) are control and The quantization parameter of the accuracy. The reconstructed Cartesian coordinates are obtained as follows: ● ● ● where app_cos(.) and app_sin(.) are approximations of cos(.) and sin(.). The calculations can use fixed-point representation, lookup tables, and linear interpolation.

[0083] In some examples, may be different from (x, y, z) for various reasons, including: -Quantification -approximation - Model inaccuracy - Model parameter inaccuracies

[0084] In some examples, the reconstruction residual (r x , r y , r z ) can be defined as follows: - - -

[0085] In this process, the G-PCC encoder 200 may proceed as follows: ●Model parameters and and quantization parameters and q φ Encoding ● Apply the geometric prediction scheme described in G-PCC DIS to represent ○ New prediction values ​​that take advantage of LiDAR characteristics can be introduced. For example, the rotation speed of the LiDAR scanner around the z-axis is usually constant. Therefore, the G-PCC decoder can predict the current in ○(δ φ (k)) k＝1...K is a set of potential speeds that the encoder can choose from. The index k can be written explicitly into the bitstream, or can be inferred from the context based on a deterministic strategy applied by both the encoder and the decoder, and o n(j) is the number of points skipped, which may be written explicitly into the bitstream, or may be inferred from the context based on a deterministic strategy applied by both the encoder and decoder. ●Reconstruct the residual (r) with each node pair x , r y , r z ) for encoding.

[0086] The G-PCC decoder 300 may proceed as follows: ●Model parameters and and quantization parameters and q φ Decoding ● The geometric prediction scheme described in the G-PCC draft international standard (DIS) is used to predict the nodes associated with the Parameters to decode ● Calculate the reconstructed coordinates as described above ● For the residual (r x , r y , r z ) to decode ○ As discussed in the next section, lossy compression can be achieved by quantizing the reconstructed residual (r x , r y , r z ) to support The original coordinates (x, y, z) are calculated as follows: ○ ○ ○

[0087] Lossy compression can be achieved by reconstructing the residual (r x , r y , r z ) is achieved by applying quantization or by discarding points.

[0088] The quantized reconstruction residual is calculated as follows: ● ● ● Among them (q x , o x ), (q y , 0 y ) and (q z , o z ) are control and The quantization parameter of the accuracy.

[0089] In some examples, the G-PCC encoder 200 and / or the G-PCC decoder 300 may use trellis quantization to further improve RD (rate distortion) performance results. The quantization parameter may be changed at the sequence / frame / slice / block level to achieve region-adaptive quality and for rate control purposes.

[0090] The G-PCC codec (e.g., G-PCC encoder 200 and / or G-PCC decoder 300) can use inter-frame prediction to perform point cloud compression. By using inter-frame prediction, the G-PCC codec can use the redundancy of points across frames to provide additional bit rate savings. The examples in various aspects of the present disclosure can be used alone or in any combination.

[0091] Although the discussion is primarily about polar coordinate systems, the processes disclosed in this application may also be applied to other coordinate systems, such as Cartesian coordinate systems, spherical coordinate systems, or any custom coordinate system that can be used to represent / encode point cloud locations and attributes.

[0092] The G-PCC codec can determine whether to use inter-frame prediction or intra-frame prediction to encode and decode a point. For example, the G-PCC codec can perform an analysis to determine whether it is beneficial (for example, in terms of bit rate or other conditions) to encode and decode a specific point using inter-frame or intra-frame prediction. In some examples, the G-PCC codec can perform this determination in one of the following ways. In some examples, the G-PCC coder 200 can use a first set of conditions during encoding, and the G-PCC decoder 300 can use the first set of conditions during decoding. In some examples, the G-PCC coder 200 can use the first set of conditions during encoding, and signal a syntax element indicating an inter-frame prediction mode, and the G-PCC decoder 300 can determine the mode based on the syntax element signaled.

[0093] The first set of conditions can be derived using one or more characteristics of the point, such as Cartesian position, angular coordinates (radius, azimuth, elevation), prediction modes of neighboring nodes, etc. Some example conditions included in the first set of conditions are as follows. In the first example condition, the point belongs to a frame that is not an intra-coded frame, or a frame that does not correspond to a random access point. In the second example condition, the orientation of the current point is different from the orientation of one or more previously decoded points of the current frame, or the incremental orientation between the current point and the previous point is non-zero. In some examples, an approximation of the incremental orientation or a quantized version of the incremental orientation can be used to determine the inter-frame prediction mode of the current node. In the third example condition, the point belongs to a particular type of slice (i.e., a P slice or a B slice indicating inter-frame prediction can be applied). In the fourth example condition, a flag is signaled to indicate whether inter-frame prediction is enabled for a particular frame. The flag can be signaled in a syntax element associated with the frame (e.g., a slice / slice header, a frame header, etc.) or in a parameter set referenced by the frame. For example, inter_prediction_enabled_flag may be signaled in the geometry parameter set (GPS) to specify whether inter prediction is enabled for point cloud frames / strips referencing the GPS. When inter_prediction_enabled_flag indicates that inter prediction is not enabled, inter prediction indication for points in the point cloud may not be signaled.

[0094] In some examples, the inter-prediction mode of a point may be determined based on the residual of the orientation or phi multiplier associated with the point.

[0095] In some examples, inter prediction (or one or more inter prediction modes) may be enabled only when angular mode is enabled (ie, inter prediction may be disabled if angular mode is not enabled).

[0096] The G-PCC codec can select a reference frame. In some examples, the G-PCC codec can use a previously decoded frame (or a previous frame in decoding order in some cases) as a reference frame. In other examples, an indication of a frame number (using an LSB value or an incremental frame number value) can be used to specify a reference frame. More generally, two or more frames can be designated as reference frames, and a point can be encoded and decoded using inter-frame prediction from any of the reference frames (an indication of the reference frame associated with the point can be signaled or derived).

[0097] In another example, prediction may be performed from two or more frames (eg, bidirectional prediction). Thus, the G-PCC codec may predict a point of the current frame based on a reference point in a first reference frame and a reference point in a second reference frame.

[0098] The G-PCC codec can utilize multiple inter-frame prediction modes. When a point is inter-frame predicted, there may be one or more ways to predict the point based on a reference frame. Different mode values ​​can be used to specify each prediction type (for example, the G-PCC codec can signal a syntax element indicating the inter-frame prediction mode for the current point). As an example inter-frame prediction mode, a point can be predicted based on a zero motion candidate from a reference frame (for example, a reference point in a reference frame can be a zero motion candidate). As another example inter-frame prediction mode, a point can be predicted based on a global motion candidate from a reference frame. As another example inter-frame prediction mode, a point can be predicted based on a candidate point from a reference frame, and other parameters (for example, motion vectors, etc.) can be used to specify the candidate point.

[0099] Some details about different types of inter-frame prediction examples are provided below. The G-PCC codec can utilize partial inter-frame prediction modes. In one example, inter-frame prediction can be applied only to a subset of the characteristics of the points in the predicted point cloud. One or more characteristics for which intra-frame prediction is not applicable can be encoded and decoded using intra-frame prediction or by other techniques. In one example, when a point is indicated as inter-frame prediction, the radius of the point can be predicted based on a reference point (or reference frame) when angular mode is enabled, and the azimuth and laser ID (or elevation) can be derived based on intra-frame prediction or other techniques. In another example, if a point is indicated as being inter-frame predicted, both the radius and azimuth of the point are predicted based on a reference point (or reference frame) when angular mode is enabled; the G-PCC codec can derive the laser ID (or elevation) based on intra-frame prediction or other techniques.

[0100] The G-PCC codec can perform signaling of inter-frame prediction modes. In some examples, the inter-frame prediction mode associated with a point can be signaled in the bitstream. This can be signaled as a flag or as a mode value. In one example inter-frame prediction mode signaling technique, an inter-pred-flag equal to 0 can indicate that the point is predicted using intra-frame prediction, and an inter-pred-flag equal to 1 can indicate that the point is predicted using inter-frame prediction. In some examples, based on the indication that the point is predicted using inter-frame prediction, a mode value can be signaled to specify the process of performing inter-frame prediction for the point. In another example inter-frame prediction mode signaling technique, when a point is predicted using inter-frame prediction, a mode value can be signaled, wherein a mode value of 0 specifies that a zero motion candidate is used for prediction, and a mode value of 1 specifies that a global motion candidate is used for prediction.

[0101] In this disclosure, inter prediction mode may be used interchangeably with an indication of whether a point is encoded or decoded with inter prediction, as well as the specific inter prediction technique used.

[0102] Certain conditions may affect the way the G-PCC codec signals the inter-frame prediction mode. In some cases, the inter-frame prediction mode may be signaled only when the second set of conditions applies. The second set of conditions may be derived using one or more characteristics of the point, such as Cartesian position, angular coordinates (radius, azimuth, elevation), prediction modes of neighboring nodes, etc. Example conditions included in the second set of conditions may include that the point belongs to the following frame: the frame is not an intra-frame coded frame, or does not correspond to a random access point. Example conditions included in the second set of conditions may include that the orientation of the current point is different from the orientation of the previously decoded point, or that the incremental orientation between the current point and the previous point is not zero. In some examples, an approximation of the incremental orientation or a quantized version of the incremental orientation may be used to determine the inter-frame prediction mode of the current node. Example conditions included in the second set of conditions may include that the incremental orientation between the current point and the previous point is greater than a threshold; the value of the threshold may be fixed or signaled in the bitstream, or derived based on a syntax element signaled in the bitstream.

[0103] More generally, the second set of conditions may include one or more criteria, including but not limited to an incremental azimuth value, an incremental radius value, an incremental laser ID value, an incremental elevation value. This incremental value of a characteristic may be indicated by a corresponding residual value signaled for a point in the bitstream. The second set of conditions may also include incremental coordinate values ​​in a Cartesian domain (incremental x, incremental y, incremental z).

[0104] When no inter prediction mode is signaled for a particular point, it can be inferred that the inter prediction mode is a fixed value for that point.When no inter prediction mode is signaled for a point, the value can be assumed to be 0 (or a value specifying intra prediction).

[0105] Signaling optimization will now be discussed. In some cases, the inter-frame prediction mode may not be signaled for all points, but may be derived from syntax elements signaled in the bitstream. The inter-frame prediction mode may be signaled for a prediction tree, and the inter-frame prediction mode may be applied to all points in the prediction tree. The inter-frame prediction mode may be signaled in a geometry slice, and the inter-frame prediction mode may be applied to all points in the slice. The inter-frame prediction mode may be signaled for a prediction block (specified as a fixed number of points, or a signaled number of points, etc.), and the inter-frame prediction mode may be applied to all points in the prediction block.

[0106] In some examples, the inter prediction mode may be signaled only to some nodes specified by the tree depth. For example, the inter prediction mode may be signaled only for the root node of the prediction tree.

[0107] In one example, when inter prediction is applied for some characteristics of a point (e.g., radius) and intra prediction is applied for other characteristics of a point (e.g., orientation, laser ID), only a subset of intra prediction modes may be allowed (e.g., only mode 0 of intra prediction may be allowed for orientation, laser ID). In some examples, if only one intra prediction mode is allowed, the mode value is not signaled and inferred at the decoder. In some examples, if a subset (two or more) of intra prediction modes is allowed, a more optimized encoding and decoding of the mode syntax element may be applied (e.g., fewer bits may be sufficient to encode an index to a subset of modes).

[0108] For each point and corresponding to each prediction mode, the G-PCC codec can derive the predicted value of one or more characteristics of the current point based on the position of one or more points in the reference frame. The example prediction process is as follows: 1a. When a zero motion candidate is selected, a prediction candidate may be selected as a point closest to the azimuth and laser ID having specific values ​​corresponding to the azimuth and laser ID (e.g., the reconstructed azimuth value and laser ID of the current point); the value of the radius of the prediction candidate is selected as the predicted value of the radius of the current node. 1b. When a zero motion candidate is selected, a prediction candidate may be selected as a point having the closest azimuth and laser ID corresponding to specific values ​​of azimuth and laser ID (e.g., the reconstructed azimuth value and laser ID of a previously decoded and reconstructed point); the values ​​of the radius, azimuth, and laser ID of the prediction candidate are selected as predicted values ​​of the radius, azimuth, and laser ID of the current node. 2a. When selecting a global motion candidate, after applying global motion compensation to the prediction candidate, the prediction candidate can be selected as the point with the closest orientation and laser ID corresponding to a specific value of orientation and laser ID; the value of the radius of the prediction candidate is selected as the predicted value of the radius of the current node. 2b. When selecting a global motion candidate, after applying global motion compensation to the prediction candidate, the prediction candidate can be selected as the point with the closest orientation and laser ID corresponding to specific values ​​of orientation and laser ID (for example, the reconstructed orientation value and laser ID of a previously decoded and reconstructed point); the values ​​of the radius, orientation and laser ID of the prediction candidate are selected as the predicted values ​​of the radius, orientation and laser ID of the current node. 3a. When a generic candidate is selected, after applying the motion vector (associated with the current point) to the prediction candidate, the prediction candidate can be selected as the point with the closest orientation and laser ID corresponding to the specific values ​​of orientation and laser ID; the value of the radius of the prediction candidate is selected as the predicted value of the radius of the current node. 3b. When a generic candidate is selected, after applying a motion vector to the prediction candidate, the prediction candidate can be selected as the point closest to the azimuth and laser ID corresponding to specific values ​​of the azimuth and laser ID (e.g., the reconstructed azimuth value and laser ID of a previously decoded and reconstructed point); the values ​​of the radius, azimuth and laser ID of the prediction candidate are selected as the predicted values ​​of the radius, azimuth and laser ID of the current node.

[0109] In some examples, the motion vector can be used to derive a point for deriving the radius, azimuth, and laser ID (or elevation) of the current point or a prediction of these values. In some examples, the motion vector can be used to derive a prediction of the x, y, z coordinates or a prediction of these values.

[0110] In some examples, more than one point in the reference frame can be used to derive the prediction value. The number of points can be signaled in the strip header or GPS. In some examples, the index of two or more points in the reference frame can be signaled to specify a specific point for inter-frame prediction. When a zero motion candidate is selected, two prediction candidates are selected as two points with two closest orientations and laser IDs corresponding to specific values ​​of orientation and laser ID; the average of the radius of the two prediction candidates is selected as the prediction value of the radius of the current node. When two or more prediction candidates are selected as candidates for inter-frame prediction (e.g., based on the proximity of laser ID and / or orientation), the prediction candidate can be selected by signaling an index to a list including two or more prediction candidates. In some examples, a weighted average (a weight calculated based on the difference between the laser ID value of the current point and the predicted value and / or the difference between the orientation of the current point and the predicted value) can also be used as a prediction candidate.

[0111] The G-PCC codec can perform preparation of reference frames. In some examples, the reference frame can be motion compensated before inter-frame prediction. For example, one or more points of the reference frame can be associated with a motion (e.g., global motion), and a prediction of a subsequent frame can be obtained by compensating the position of the point with the associated motion. The motion can be estimated or obtained from external information (e.g., GPS information). Therefore, the "reference frame" used for inter-frame prediction can be a "compensated reference frame". In other words, the reference frame can be a motion compensated reference frame.

[0112] In some examples, one or more points in the compensated reference frame can be considered as zero motion candidates for the current frame. In some examples, the point in the reference frame to which motion compensation is applied can belong to a specific feature in the point cloud (e.g., an object), or it can be labeled by some estimation algorithm (e.g., ground, non-ground). Global motion parameters (e.g., rotation, translation) can be signaled in the bitstream.

[0113] In some examples, in addition to the global motion parameters, additional adjustment values ​​may be signaled for one or more of the following: x, y, z, radius, orientation, laser ID. For example, when preparing a predicted frame by applying motion compensation (e.g., using the motion parameters), one or more additional adjustment values ​​of x, y, z may be applied in the Cartesian domain; one or additional adjustment values ​​of radius, orientation, laser ID may be applied in the spherical domain.

[0114] In some examples, adjustment values ​​may be specified for the entire sequence (referred to as "global" adjustments); these values ​​may also be incorporated into the global motion parameters.

[0115] In some examples, adjustment values ​​may be specified for groups of frames (referred to as "group" adjustments); these values ​​may also be incorporated into the global motion parameters.

[0116] In some examples, adjustment values ​​may be specified for frames (represented as "frame" adjustments); these values ​​may also be incorporated into the global motion parameters.

[0117] In some examples, adjustment values ​​may be specified for different regions of the point cloud (representing “local” adjustments); these values ​​may also be incorporated into the global motion parameters, or signaled as a modification to the global motion applied to the frame.

[0118] One or more of the global, frame, and local adjustments may be signaled; when more than one of these parameters is present, the respective adjustments may be applied sequentially or together (e.g., for points belonging to the region, the global, frame, and local x adjustments xg, xf, xl may be applied as xg+xf+xl, respectively). In other examples, all adjustments may be applied at one level (e.g., global) before being applied at a different level (e.g., frame).

[0119] The motion vector associated with the inter prediction mode of a point may be signaled in the bitstream. In some cases, multiple points may share the same motion vector and may be signaled in a slice or parameter set (e.g., GPS, or a parameter set dedicated to motion parameters). In some cases, the motion vector associated with a particular point may be predicted from neighboring points of spatial or temporal inter prediction, and only the difference between the actual motion vector and the predicted motion vector may be signaled.

[0120] The G-PCC codec may perform context selection. For example, the G-PCC codec may determine the context to be used for predicting the geometry parameters / syntax elements based on one or more syntax elements signaled in the bitstream or their derived variables. The use of additional context may improve the compression efficiency of the codec.

[0121] In some examples, the G-PCC codec can utilize a first parameter associated with a point to determine contexts for other syntax elements associated with the point. For example, the context of one or more syntax elements associated with the phi multiplier (e.g., ptn_phi_mult_abs_gt0_flag, ptn_phi_mult_sign_flag, ptn_phi_mult_abs_gt1_flag, ptn_phi_mult_abs_minus2, ptn_phi_mult_abs_minus9) can be selected based on an inter-frame flag.

[0122] In one example, the inter flag indicates whether the first set of contexts is selected in case of inter prediction or the second set of contexts is selected in case of intra prediction. In case prediction geometry with angular mode is enabled (spherical position codec: azimuth, radius, elevation or laser ID), this can be applied to the codecs of phi multipliers, primary residual and secondary residual.

[0123] If the context of a grammatical element depends on a first set of conditions, one context may be selected when the first set of conditions is true and a first parameter associated with the point takes a first value, and a second context may be selected when the first set of conditions is true and the first parameter associated with the point takes a second value different from the first value.

[0124] Different choices for the first parameter are described. Each choice may present a different trade-off between compression efficiency and storage (more context may require more storage associated with the context state). In one example, the first parameter may be an inter-frame flag associated with the point. In another example, the first parameter may be a prediction mode associated with the point (e.g., a prediction mode used in prediction geometry). In another example, the first parameter may be a tuple consisting of an inter-frame flag associated with the point and a prediction mode, such as (interFlag, predMode). When there is more than one inter-frame prediction candidate (e.g., a zero motion candidate, a global motion candidate, etc.), let interPredMode represent the various inter-frame prediction candidates (e.g., 0 for intra-frame prediction, 1 for a zero motion candidate, 2 for a global motion candidate, etc.). The first parameter may be interPredMode.

[0125] In some examples, the G-PCC codec may use different conditions / parameters for different types of frames. For example, for intra-coded frames (predicted without reference to other frames in the sequence), a second parameter may be selected to determine the context described in the present disclosure, while for other frames (e.g., inter-coded frames), a third parameter may be selected to determine the context described in the present disclosure. The second parameter and the third parameter may be different (e.g., for intra-frames, a prediction mode may be selected as a second parameter, and for inter-frames, a tuple of a prediction mode and an inter-frame flag may be selected as a third parameter). The G-PCC codec may extend similar context selection to one or more syntax elements / components (main residual, auxiliary residual, etc.).

[0126] Figure 7 is a conceptual diagram illustrating an example inter-frame prediction process for predicting points of a point cloud according to one or more aspects of the present disclosure. Figure 7 As shown, the current frame 750 may include a plurality of points 752A-752L (collectively referred to as "points 752"), and the reference frame 754 may include a plurality of points 756A-756L (collectively referred to as "points 756"). The reference frame 754 may be a frame that is encoded and / or reconstructed before the current frame 750 is decoded and / or reconstructed (e.g., the reference frame 754 may be located before the current frame 750 in the coding order). The G-PCC codec may utilize intra-frame prediction to predict one or more points 752 of the current frame 750 based on one or more points 756 of the reference frame 754. For example, the G-PCC decoder (or a reconstruction loop of the G-PCC encoder) may predict one or more parameters (e.g., (r, φ, i) of the current point 752A based on the point 752 of the one or more points 756).

[0127] To perform inter-frame prediction to predict a current point in a current frame, the G-PCC codec may determine a reference point in a reference frame different from the current frame, and predict one or more parameters of the current point based on the reference point. For example, to predict the current point 752A, the G-PCC codec may determine a reference point 756A, and predict one or more parameters of the current point 752A based on one or more parameters of the reference point 756A. The determined reference point may be referred to as an identified reference point.

[0128] The G-PCC codec may determine the reference point using any suitable technique. As an example, the G-PCC codec may determine a pivot point in the current frame that is before the current point in the coding order; and determine the reference point based on one or more parameters of the pivot point. For example, where the coding order is counterclockwise, the G-PCC codec may determine that point 752B is a previous point of the current point 752A (e.g., a point immediately before the current point in the coding order) (i.e., determine that point 752B is a pivot point), and determine the reference point based on one or more parameters of pivot point 752B.

[0129] To determine the reference point based on one or more parameters of the pivot point, the G-PCC codec may determine a reference pivot point in the reference frame based on the orientation of the pivot point; and determine the reference point based on the reference pivot point. For example, the G-PCC codec may determine a point in the reference frame 754 that has the same orientation (or the same orientation and the same laser ID) as the pivot point 752B. Figure 7 In the example of , the G-PCC codec can determine that point 756B is the reference pivot point because point 756B has the same orientation as pivot point 752B. Figure 7 The reference pivot point in the example corresponds to an actual point (e.g., an actual point in frame 754), but the technology of the present disclosure need not be so limited. For example, in some examples, the reference pivot point can be a virtual point that does not correspond to a reconstructed point in reference frame 754.

[0130] In some examples, the G-PCC codec can determine the reference pivot point based on the actual (e.g., unscaled) position of the pivot point. In other examples, the G-PCC codec can determine the reference pivot point based on the scaled position of the pivot point. For example, the G-PCC codec can determine the scaled position by scaling the position of the pivot point by a constant value.

[0131] To determine the reference point based on the reference pivot point, the G-PCC codec may identify a point having an orientation greater than the orientation of the reference pivot point as a reference point in the reference frame. For example, the G-PCC codec may determine which points 756 have orientation values ​​greater than the orientation value of the reference pivot point, and select the point with the smallest orientation value (from the set 756 of points having orientation values ​​greater than the orientation value of the reference pivot point). Figure 7 In this example, point 756A may be a point in reference frame 754 having a minimum orientation greater than the orientation of reference pivot point 765B. Thus, the G-PCC codec may identify point 756A as a reference point for performing intra prediction of current point 752A.

[0132] In some examples, the G-PCC codec can determine the reference point based on the actual (e.g., unscaled) position of the reference pivot point. In other examples, the G-PCC codec can determine the reference point based on the scaled position of the reference pivot point. For example, the G-PCC codec can determine the scaled position of the reference pivot point by scaling the position of the pivot point by a constant value. Therefore, in some examples, the G-PCC codec can determine the reference point by identifying as the reference point a point with a minimum scaled position greater than the scaled position of the reference pivot point (e.g., point 756A). In some examples, the G-PCC codec can utilize a point with a second minimum position greater than the scaled position. Therefore, in some examples, the G-PCC codec can determine the reference point by identifying as the reference point a point with a second minimum scaled position greater than the scaled position of the reference pivot point (e.g., point 756L).

[0133] The G-PCC codec may predict the parameters of the current point 752A based on the parameters of the reference point 756A. For example, the G-PCC codec may signal residual data representing the difference between the parameters of the current point 752A and the reference point 756A. The G-PCC decoder may add the residual data to the parameters of the reference point 756A to reconstruct the parameters of the current point 752A.

[0134] Although discussed above is the use of a single reference point in a single reference frame, the technology of the present disclosure is not limited to this. As an example, multiple reference points in a single reference frame can be used together to predict the current point. For example, the G-PCC codec can determine multiple reference points in a reference frame and based on a reference pivot point. The G-PCC codec can predict one or more parameters of the current point in the current frame based on multiple reference points. As another example, reference points from multiple reference frames can be used to predict the current point.

[0135] As discussed above, the G-PCC codec can perform azimuth prediction. Let (r, phi, laser ID) be the three coordinates of the pivot point in a spherical coordinate system (called radius, azimuth, and laser ID). The techniques disclosed herein can also be applied to other coordinate systems.

[0136] In some examples, the G-PCC codec may encode and decode points in the current point cloud frame in the following ordered manner:

[0137] First, for a current point in a current frame, the G-PCC codec may select a pivot point in the current frame that precedes the first point in decoding order. In some examples, the pivot point is the previous point in the current frame in decoding order. In some examples, the pivot point is the second previous point in the current frame in decoding order. More generally, more than one previous point may be selected as the pivot point for the current point. In some examples, the pivot point may be a virtual point derived based on a previously decoded point in the current frame and an azimuth displacement that is a multiple of an azimuth quantizer scale value (predetermined or derived from a signaled syntax element).

[0138] Second, the G-PCC codec can select a point in the reference frame, a reference pivot point associated with the pivot point. The reference pivot point can be selected as a point in the reference frame with the same orientation and laser ID as the pivot point. In some examples, points with other laser ID values ​​can also be candidates for the reference pivot point (e.g., a reference pivot point can be selected as a point in the reference frame with the same orientation as the pivot point and a laser ID in the range of [LaserID-M, LaserID+M], where LaserID is the laser ID of the pivot point and M is a fixed value (e.g., 1) or selected based on the distance of the pivot point from the origin, or derived from the laser ID (e.g., M can be smaller for smaller laser ID values ​​and larger for larger laser ID values)). In some examples, the orientation and laser ID can be used to define a distance metric, and the distance metric can be used to select the reference pivot point as the point with the minimum distance to the orientation and laser ID of the pivot point. The normalized azimuth value may be obtained by scaling the azimuth by a first constant value; the normalized laser ID may be obtained by scaling the laser ID by a second constant value; the distance metric may be obtained by calculating the norm (e.g., L2 norm, L1 norm) of the normalized laser ID and azimuth values ​​of the pivot point and the reference point. More generally, the reference pivot point may be selected as a point in the reference frame that is located in the neighborhood of a point having the same azimuth and laser ID as the pivot point. In some examples, the reference pivot point may be a virtual point in the reference frame that is derived from a pivot point having the same azimuth and laser ID as the pivot point.

[0139] Third, the G-PCC codec may select a reference point in a reference frame that is associated with a reference pivot point. The reference point may be selected as a point in the reference frame that has a minimum bearing greater than the reference pivot point and a laser ID that is the same as the reference pivot point. The reference point may be selected as a point in the reference frame that has a second minimum bearing greater than the reference pivot point and a laser ID that is the same as the reference pivot point. In some examples, when the reference point is not available, inter-frame prediction may be disabled for the current point. In some examples, the reference point may be selected as the reference pivot point.

[0140] Fourth, the G-PCC codec may calculate a first residual between the reference point and the reference pivot point.

[0141] Fifth, the G-PCC codec can use the first residual to derive a first prediction of the current value. The prediction can be derived by adding components of the first residual to the components of the pivot point (e.g., a radius prediction can be obtained by adding a radius component of the first residual to a radius component of the pivot point (similar to that for orientation)). In some examples, the first prediction can be set equal to the reference point.

[0142] Sixth, the G-PCC codec can encode and decode the second residual between the first prediction and the position of the current point.

[0143] Seventh, residual component: One or more residuals disclosed in the present disclosure may include one or more of the following: The residual may include a radius residual between a reference pivot point and a reference point. The residual may include an orientation residual between a reference pivot point and a reference point.

[0144] Eighth, the G-PCC codec may derive the current point based on the first prediction and the second residual. In some examples, the G-PCC codec may derive the current point from the second residual (eg, and not based on the first prediction).

[0145] The G-PCC codec may apply one or more of the above techniques to the quantized position value; the scale value used for quantization may be derived or predetermined based on the signaled value. The quantized position value and laser ID may be used to search for an inter-frame prediction point in a reference. For example, the position of a previously decoded and reconstructed point may be quantized, and the inter-frame prediction point having the quantized position and laser ID closest to the quantized position and laser ID of the previous point may be selected as the prediction value for the position, radius, and laser ID of the current point, as a quantized / dequantized prediction value or an unquantized prediction value.

[0146] The G-PCC codec may apply one or more of the above techniques to the quantized laser ID values; the scale values ​​used for quantization may be derived or predetermined based on the values ​​sent with the signal. The position and quantized laser ID values ​​may be used to search for an inter-frame prediction point in a reference. For example, the laser ID of a previously decoded and reconstructed point may be quantized, and the inter-frame prediction point with the position and quantized laser ID closest to the position and quantized laser ID of the previous point may be selected as the prediction value for the position, radius and laser ID of the current point, as a quantized / dequantized prediction value or an unquantized prediction value.

[0147] The G-PCC codec may apply one or more of the above techniques to the quantized position and quantized laser ID values; the scale values ​​used for quantization may be derived from the signaled values ​​or predetermined. The quantized position and quantized laser ID values ​​may be used to search for inter-frame prediction points in a reference. For example, the position and laser ID of a previously decoded and reconstructed point may be quantized, and the inter-frame prediction point having the quantized position and quantized laser ID closest to the quantized position and quantized laser ID of the previous point may be selected as the prediction value for the position, radius, and laser ID of the current point, as a quantized / dequantized prediction value or an unquantized prediction value.

[0148] In some examples, a reference frame may refer to a set of (radius, orientation, laser ID) tuples derived from a reference frame. For example, for each point in a reference frame, if no other point in the set has the same orientation and laser ID, the G-PCC codec may add the radius, orientation, laser ID to the set. In some cases, a quantized value of the orientation may be added. In some cases, if the value of r is less than the value of r1, a (r, phi, laser ID) may be added even if there is another tuple with the same phi and laser ID in the set (e.g., (r1, phi, laser ID)) (in which case the existing tuple (r1, phi, laser ID) may be replaced by the new (r, phi, laser ID)). In some cases, points in a reference frame may be in the x, y, z domain; these points may be stored as is for reference, or by converting them to the spherical domain. In some cases, motion compensated positions may be added to a reference frame. The compensation may be based on signaled motion vectors (eg, global motion vectors having rotation and / or translation) associated with the current frame and the reference frame.

[0149] The G-PCC codec can perform context selection for inter-frame prediction mode. As discussed above, the G-PCC codec can signal an inter-frame prediction flag and an inter-frame prediction mode. The context for the G-PCC codec to encode and decode an inter-frame prediction flag or an inter-frame prediction mode can be selected as follows: the G-PCC codec can set b1, b2, b3, ... bN to the inter-frame prediction flag values ​​of N previous decoding points (b1 is the previous node, b2 is the previous one of b1, etc.). The G-PCC codec can select a context index as a number generated by using b1, b2, ... bN, as shown below: ctxIdx = b1 + b2 << 1 + b3 << 2 + ... bN << (N-1). In another example, the G-PCC codec can select a context index as b1 + b2 + ... + bN. In some examples, the values ​​of b1, b2, ... bN can also be used to select the context of other syntax elements such as inter-frame prediction mode, motion vector, etc. In some examples, N can be fixed to 5.

[0150] The G-PCC codec can perform improved radius residual coding. The radius, azimuth, and laser ID associated with each point can be coded with predictive geometry coding. The residual of the radius component of each point can be coded with an "equal_to_zero" flag, a sign bit, the number of bits of the residual residual, and / or the residual residual. The number of bits can be an indicator of the relative size of the radius residual. When the LIDAR-captured points are close to each other in the point cloud (e.g., points of an object or building captured by one of the lasers), the radius may not change much between these points. Since the object (most likely a solid) is continuous, consecutive LIDAR scan points can be returned to the sensor with an azimuth difference between consecutive points close to the sampling frequency of the azimuth. The azimuth difference or azimuth residual of adjacent points on the object (of the same laser) is close to zero.

[0151] The context for the number of bits of the radius residual is selected based on the azimuth residual of the current point. For example, based on the size of the azimuth residual, the context can be switched as follows, where N is a fixed value (signaled or predetermined). Table 1 below provides an example of a context index for the number of bits used in the number of bits of the radius residual N. Table 1: Context index of the number of bits used in the number of bits of the radius residual Abs(azimResi) N＝3 N＝4 0 1 1 1 1 1 2 2 2 3 3 3 4 0 4 >＝5 0 0

[0152] Note that other procedures for selecting the context index based on the radius residual may also be used.

[0153] The following specific examples illustrate implementations of several techniques of the present disclosure. Example A

[0154] For each point in the point cloud frame that is not fully intra-coded (random access point), a flag can be used to specify whether the point is coded with inter-frame prediction. When inter-frame prediction is applied, a "zero motion vector" candidate is selected to predict the radius of the predicted point based on the point in the reference frame that has the same laser ID value as the current point and has the closest quantized azimuth value to the current point. The following changes to the syntax structure are provided ( <add> …< / add> The label indicates adding <delete> …< / delete> Tags indicate removal): <add> ptn_inter_flag[nodeIdx] is equal to 0, which specifies that the radius residual of the current node is encoded and decoded using intra prediction. ptn_inter_flag[nodeIdx] is equal to 1, which specifies that the radius residual of the current node is encoded and decoded using inter prediction. When not present, the value of ptn_inter_flag[nodeIdx] is inferred to be equal to 0. < / add>

[0155] When the current point cloud frame is a random access point or a full intra-frame prediction frame, the value of InterEnableFlag is set to 0, otherwise it is set to 1.

[0156] In some examples, when ptn_inter_flag is equal to 1, the syntax element ptn_pred_mode[] is not signaled and is inferred to be a default value (eg, equal to 0 corresponding to prediction geometry mode 0).

[0157] The position prediction process changes as follows: Position prediction process The inputs to this process are: - variable predMode, indicating the prediction mode of the current node, - the arrays aPos0, aPos1 and aPos2 have values ​​aPosX[k], k=0..2 and X=0..2. Each array contains the position associated with the Xth generation ancestor node in the depth-first tree traversal order, - variable curDepth, indicating the node distance between the current node and the root node of the current prediction tree, <add> -Variable interFlag, indicating whether the radius of the current node is encoded and decoded using inter-frame prediction. < / add> The output from this process is an array predPos, with values ​​predPos[k], k=0..2, indicating the predicted point position associated with the nodeIdx-th tree node. When predMode is equal to 0, the predicted point position is calculated as follows, When predMode is equal to 1, the predicted point location is the location associated with the first ancestor. for(k=0;k<3;k++) predPos[k]=aPos0[k] When predMode equals 2, the predicted point location is a linear combination of the locations associated with the first two ancestors for(k=0;k<3;k++) predPos[k]=aPos0[k]+aPos0[k]-aPos1[k] Otherwise, when predMode is equal to 3, the predicted point position is a linear combination of the positions associated with all three ancestors for (k=0; k<3; k++) predPos[k]=aPos0[k]+aPos1[k]-aPos2[k] <add> When interFlag is equal to 1, predPos[0] is derived as follows. Let refFramePos[k] for k=0..2 be the position of the point in the reference frame such that refFramePos[2]=predPos[2], and the absolute difference between the values ​​of refFramePos[1] and predPos[1] is the lowest value among the points in the reference frame. predPos[0] is set equal to refFramePos[0]. < / add> In one example, the quantized values ​​of refFramePos[1] and predPos[1] are used to measure the absolute difference of the values, where the quantization scale can be a fixed number (e.g., geom_angular_azimuth_step_minus1+1) Example B

[0158] In example B, when inter prediction is used, the prediction mode may not be signaled. For example, when inter prediction is used, the prediction mode is inferred to be equal to 0. The following provides changes to the syntax structure ( <add> …< / add> Tags indicate addition, <delete> …< / delete> Tags indicate removal):

[0159] When ptn_pred_mode[nodeIdx] is not signaled, ptn_pred_mode[nodeIdx] may be inferred to be equal to 0. In some examples, the following corollary may be added: when ptn_inter_flag[nodeIdx] is equal to 1, ptn_pred_mode[nodeIdx] is inferred to be equal to 0. Example C

[0160] In example C, the signaling order of phi and prediction mode may not be modified. The signaling of prediction mode may be conditional on the signaling of the inter-frame flag. The signaling of ptn_phi_mult_abs_gt0_flag may also be conditional on the inter-frame flag. The following provides changes to the syntax structure ( <add> …< / add> Tags indicate addition, <delete> …< / delete> Tags indicate removal):

[0161] When ptn_inter_flag[nodeIdx] is not signaled, ptn_inter_flag[nodeIdx] is inferred to be equal to 0.

[0162] When ptn_phi_mult_abs_gt0_flag[nodeIdx] is not signaled, ptn_phi_mult_abs_gt0_flag[nodeIdx] is inferred to be equal to 0. In another example, the following inference is added: when ptn_inter_flag[nodeIdx] is equal to 1, ptn_phi_mult_abs_gt0_flag[nodeIdx] is inferred to be equal to 1. Example D

[0163] In example D, the order of the prediction mode and the phi syntax element may not change. For example, the inter flag may be signaled after the prediction mode but before the phi syntax element.

[0164] When ptn_pred_mode[nodeIdx] is greater than 0 and InterEnableFlag is equal to 1 (ptn_inter_flag[] is not signaled), ptn_inter_flag[nodeIdx] is inferred to be 0. Example E

[0165] Example E is similar to Example A, with the modification that the signaling of the interframe flag also depends on the value of pred_mode. The following full syntax is added.

[0166] When InterEnableFlag is equal to 1 and PtnPhiMult[nodeIdx] is equal to 0 or ptn_pred_mode[nodeIdx] is not equal to 0, ptn_inter_flag[nodeIdx] is inferred to be equal to 0. Example F

[0167] In this example, the prediction predPos[0] of the radius is obtained by comparing not only the positions of points having the same laser ID as the current point, but also the positions of points of adjacent laser IDs.

[0168] When interFlag is equal to 1, the G-PCC codec may derive predPos[0] as follows. The G-PCC codec may set refFramePos[k] for k=0..2 to the position of the point in the reference frame such that refFramePos[2] takes one of the values ​​{predPos[2]-1, predPos[2], predPos[2]+1}, and the absolute difference between the values ​​of refFramePos[1] and predPos[1] is the lowest value among the points in the reference frame. The G-PCC codec may set predPos[0] equal to refFramePos[0].

[0169] In one case, points from more laser IDs in the reference frame may be considered to obtain a reference point.

[0170] In another case, instead of comparing positions (i.e., refFramePos[1] and predPos[1]), a weighted cost function can be specified as follows: J(refPos)=w1*f1(refPos[1],predPos[1])+w2*f2(refPos[2],predPos[2])

[0171] The point with position refPos[k], k=0..2 in the reference frame selected for predicting the radius is the point with the smallest value of J(refPos).

[0172] In one case, the value of f1(x, y) may be the square of the difference between x and y, or more generally, any function that indicates the distance between the current point and a reference point in the azimuth (or second) dimension.

[0173] In one case, the value of f2(x,y) can be the square / absolute difference between x and y, or more generally, any function that indicates the distance between the current point and a reference point in the laser ID (or third) dimension.

[0174] In another case, more than one reference candidate may be selected for a point, and an inter prediction candidate index may be signaled to specify which reference candidate is selected for the point.

[0175] For example, for the current point, three points in the reference frame can be selected as follows: - The point with laser ID equal to predPos[2]-1, which has the closest position value to predPos[1] - The point with laser ID equal to predPos[2], which has the closest position value to predPos[1] - The point with laser ID equal to predPos[2]+1, which has the closest position value to predPos[1]

[0176] An index into this set / list may be signaled to specify the point selected for prediction.

[0177] More generally, for each current point encoded with inter prediction for the radius: - A reference frame is selected based on a predetermined decision (eg, a previously decoded frame) or based on one or more signaled indications (eg, a frame index / counter value). - Based on a first set of conditions (e.g., one or more of the processes disclosed herein based on laser ID range and / or azimuth range), select the set RefCandSuperSet among points in the reference frame - Select a subset RefCandSubset from RefCandSuperSet based on a second set of conditions (e.g., the closest orientation to the current point for a particular laser ID, etc.) - If RefCandSubset has more than one entry, the entries are arranged in the list RefCandList based on a third set of conditions (eg, the entries are arranged based on increasing laser ID differences from the current point); otherwise (there is only one candidate in RefCandSubset), one candidate in RefCandSubset is used for prediction. -Signals the index of the entry in RefCandList used for prediction. - Use the prediction entry to predict the radius of the current point.

[0178] In some examples, when a point in a reference frame is not present in RefCandSubset, inter prediction may be disabled, and the inter flag may or may not be signaled in this case. Example G

[0179] According to one or more aspects of the present disclosure, the G-PCC codec can signal parameters in the bitstream for deriving azimuth quantization / scaling.In some examples, the G-PCC codec can inter-predict all three components using previously decoded points in the current point cloud frame.

[0180] The bearing scale value (used to derive inter prediction candidates) can be signaled in GPS as follows:

[0181] In some examples, inter_azim_scale_log2 may be signaled independently of inter_prediction_enabled_flag.

[0182] In another example, the angular mode may also be used to adjust the signaling of inter_azim_scale_log2.

[0183] Following are example semantics for the above syntax elements. inter_prediction_enabled_flag equal to 1 specifies that inter-frame prediction can be used for point cloud frames referenced to GPS. inter_prediction_enabled_flag equal to 0 specifies that inter-frame prediction is not used for point cloud frames referenced to GPS. inter_azim_scale_log2 is used to scale the azimuth bits that can be used to derive inter prediction references. The value of inter_azim_scale_log2 should be in the inclusive range of 0 to numAzimBits. (The variable numAzimBits specifies the maximum number of bits used to represent the azimuth.)

[0184] In some examples, the inter_azim_scale_log2 syntax element may be an example of a syntax element that specifies a logarithm base 2 value of the scaling factor.The G-PCC codec may determine a scaled position (eg, a scaled position of a pivot point) based on the scaling factor.

[0185] When a frame is encoded and decoded, the reconstructed spherical position of one or more points in the frame can be stored in a reference frame, which can be used as an inter-frame prediction reference by future frames. The reconstructed spherical position can be stored in a hash table as follows: - The G-PCC codec can obtain the spherical coordinate representation of the reconstruction points. - The G-PCC codec can obtain the derived position and laser index from the reconstruction point and use the derived position and laser index The tuple of indices serves as the index into the hash table. For example, the derived quantization direction, denoted as quantization(val), may be derived by the G-PCC codec as follows: int offset=azimScaleLog2? (1<<(azimScaleLog2-1)):0; quantized(val)=val>=0? (val+offset)>>azimScaleLog2:-((-val+offset)>>azimScaleLog2);

[0186] Note that in some cases azimScaleLog2 may not be signaled and is inferred to be one of the following: - Value 0 (ie, no quantization). - is less than or equal to the smallest power of 2 of the azimuth velocity signaled in the bitstream. -The smallest power of 2 that is less than or equal to (azimuth velocity / 2) signaled in the bitstream. Different sequences may have different azimuth scale values. -Each entry in the hash table can be derived from one or more reconstruction points. - In one example, the hash table entry for a frame may be the first point in the decoding order that has a derived position and a particular value for the laser index in the reconstructed frame. - In one example, the hash table entry for a frame may be the point with the smallest radius among the points in the reconstructed frame that have a derived position and a particular value of the laser index. - In another example, the hash table entry for a frame may correspond to a point value that computes an average, mean, median, weighted average, geometric mean, etc. using points with derived orientation and particular values ​​of laser index.

[0187] In some examples, the reconstructed point position in the spherical domain (r, phi, laser ID) can be entered into a position table as follows: table index = (quantized (Phi), laser ID), table entry = (r, Phi, laser ID).

[0188] When encoding and decoding the current point (in the current frame), the following steps can be performed: - The G-PCC codec may set prevNode=(r1, phi1, laserID1) as the previous node in the current frame in decoding order. In some examples, prevNode may be selected as the parent node of the current node. ● In the table associated with the reference frame, the G-PCC codec may check if there is an index (Quantized (phi), Laser ID1) with a Quantized (phi) greater than Quantized (phi1). If there is no such index, the G-PCC codec may not apply inter prediction for the current node. If there is at least one such index, the G-PCC codec may select the index with the smallest Quantized (phi) that is greater than Quantized (phi1). The G-PCC codec may use the entry associated with the above selected index (r_inter, phi_inter, laserId_inter) as the inter prediction value for the current point.

[0189] In some examples, the following code can be used to implement the above technique with a combination of hash table arrays (rather than a 2D hash table), where computePhiQuantized() corresponds to the quantize() function described previously. In the following case, refPointVals is a vector of hash tables storing reference points, refPointVals[currLaserId] is the hash table corresponding to the index laserID=currLaserId, and all entries in refPointVals[currLaserId] are stored as increasing values ​​of quantized (phi). Because the table is stored in increasing quantized (phi), an upper bound function (upper_bound) is used to search for the first quantized (phi) laser greater than the quantized current azimuth quantization (currAzim). In the following example, idx is equal to refPointVals[currLaserId].end(), there is no such table index, so inter prediction is not used. When present, idx->second specifies the inter prediction candidate (r_inter, phi_inter, laserID_inter).

[0190] In some examples, the hash table may not store points in increments of quantized (phi), but rather use a hash function to parse and recover table entries. In another example, a general table / data structure may be used to store points from a reference frame. As described above, the following code may be used to implement the above techniques. auto quantizedPhi=computePhiQuantized(currAzim); if(refPointVals[currLaserId].size()){ auto idx=refPointVals[currLaserId].upper_bound(quantizedPhi); if(idx==refPointVals[currLaserId].end()) return std::pair<bool,point_t> (false,0); else return std::pair<bool,point_t> (true,idx->second); } return std::pair<bool,point_t> (false,0); • If inter prediction is used, the G-PCC codec may signal a flag indicating the use of inter prediction. • Encode and decode the residual using the inter prediction candidates derived above (at the decoder, the prediction candidates are used to add the residual and derive the reconstructed spherical coordinate position of the current point).

[0191] Predictive geometry codec uses a prediction tree structure to predict the location of a point. When angular codec is enabled, the x, y, z coordinates can be converted into radius, azimuth, and laser ID, and the residual can be signaled in these three coordinates as well as in the x, y, z dimensions. Intra-frame prediction for radius, azimuth, and laser ID can be one of four modes, and the predicted value is a node classified as a parent node, grandparent node, and great-grandparent node in the prediction tree relative to the current node. The predictive geometry codec currently designed in G-PCC Ed.1 is an intra-frame codec tool because it uses points in the same frame for prediction. Additionally, using points from previously decoded frames can provide better predictions and therefore better compression performance.

[0192] Inter-frame prediction has been proposed to predict the radius of a point based on a reference frame. For each point in the prediction tree, the G-PCC codec can determine whether the point is inter-frame predicted or intra-frame predicted (for example, indicated by a flag). When intra-frame prediction is used, the intra-frame prediction mode of the predictive geometry codec can be used. When using inter-frame prediction, the position and laser ID are still predicted using intra-frame prediction, and the radius can be predicted based on the point in the reference frame with the same laser ID as the current point and the position closest to the current position. In some examples, the G-PCC codec (for example, in addition to radius prediction) can also perform inter-frame prediction of the position and laser ID. When inter-frame coding is applied, the G-PCC codec can predict the radius, position and laser ID of the current point based on a point close to the position of the previously decoded point in the reference frame. In addition, separate context sets can be used for inter-frame and intra-frame predictions.

[0193] As discussed above, Figure 7 An example of inter-prediction of a current point (curPoint) based on a point in a reference frame (interPredPt) is shown. Extending inter-prediction to azimuth, radius, and laser ID may include the following steps: • For a given point, the G-PCC codec can select the previous decoding point (prevDecP0). - The G-PCC codec may select a location in the reference frame (refFrameP0) that has the same scaled position and laser ID as prevDecP0. ● In the reference point cloud frame, the G-PCC codec can find the first point (interPredPt) whose position is greater than that of refFrameP0. interPredPt can also be called the "next" inter-frame prediction value.

[0194] Figure 8 is a flow chart showing an example decoding process associated with an "inter-frame flag" signaled for each point. The inter-frame flag signaled for a point indicates whether inter-frame prediction is applied to that point. The flow charts of the present disclosure are provided as examples. Other examples may include more, fewer, or different steps, or the steps may be performed in a different order.

[0195] exist Figure 8 In the example of , the G-PCC decoder 300 can determine whether the inter-frame flag of the next point to be decoded (i.e., the current point of the current frame of the point cloud data) indicates that the current point is inter-frame predicted (800). If the inter-frame flag of the current point does not indicate that the current point is inter-frame predicted (the "no" branch of 800), the G-PCC decoder 300 can identify an intra-frame prediction candidate (812). For example, the G-PCC decoder 300 can determine an intra-frame prediction strategy (e.g., no prediction, incremental prediction, linear prediction, parallelogram prediction, etc.) to determine a prediction value for the current point. The syntax element (pred_mode) signaled in the geometry bitstream 203 can indicate the intra-frame prediction strategy used to determine the prediction value of the current point.

[0196] On the other hand, if the inter-frame flag of the current point indicates that the current point is inter-frame predicted (the "yes" branch of 800), the G-PCC decoder 300 can identify the previous point (e.g., the previous point 708) in the decoding order (802). The previous point may have coordinates (r, phi, and laser ID). The G-PCC decoder 300 can then derive the quantized phi coordinates (i.e., azimuth coordinates) of the previous point (804). The quantized phi coordinates can be expressed as Q(phi). The G-PCC decoder 300 can then check the reference frame (e.g., reference frame 706) to find a point (i.e., an inter-frame prediction point (e.g., interPredPt704)) having a quantized phi coordinate greater than the quantized phi coordinate of the previous point (806). The G-PCC decoder 300 can use the inter-frame prediction point as the prediction value of the current point (808).

[0197] Regardless of whether the G-PCC decoder 300 uses intra prediction (e.g., as described with respect to step 812) or inter prediction (e.g., as described with respect to steps 802 - 808) to determine the predicted value for the current point, the G-PCC decoder 300 can add an incremental phi multiplier (810).

[0198] A LIDAR system can scan and sample content at a specific azimuth frequency. The G-PCC encoder 200 can signal the azimuth of a point using an azimuth residual value (resAz2). The azimuth residual is not the direct difference between the azimuth of the current point and the azimuth of the predicted value for the current point (which can be determined using inter or intra prediction). Instead, the G-PCC encoder 200 encodes the azimuth residual as a combination of an incremental phi multiplier and a residual. For example, azimSpeed can represent the azimuth difference between successive captures of a rotating LIDAR sensor (note that there are multiple sensors in each rotating LIDAR system). Thus, if each LIDAR sensor in a rotating LIDAR system captures 1000 points per rotation, azimSpeed will be (1 << azimBitDepth) / 1000, effectively the azimuth difference between adjacent point captures. Here, azimBitDepth is the bit depth used to represent the azimuth value at the G-PCC encoder 200. (1 << azimBitDepth) represents a full rotation (360 degrees). Thus, (1 << azimBitDepth) / 1000 represents the azimuth difference between two captures. Although the LIDAR sensors can be designed to sample at constant intervals during rotation, due to noise and other inaccuracies, the azimuth difference between adjacent capture azimuth values may not be azimSpeed, or a multiple of azimSpeed. A first azimuth residual resAz1, which is the difference between the azimuths of the current point and the predicted value, is encoded and decoded as a combination of an incremental phi multiplier (qphi) and a second azimuth residual (resAz2). qphi can be derived as follows: qphi = deltaPhi >= 0? (deltaPhi + (_geomAngularAzimuthSpeed >> 1)) / _geomAngularAzimuthSpeed : -(-deltaPhi + (_geomAngularAzimuthSpeed >> 1)) / _geomAngularAzimuthSpeed; The value of the coded azimuth residual resAz2 is obtained as resAz1−deltaPhi*azimSpeed. Both qphi and resAz2 are signaled in the bitstream. The G-PCC decoder 300 may decode resAz2 and qphi. The G-PCC decoder 300 may obtain the reconstructed azimuth residual (resAz1) as resAz2+qphi*azimSpeed.

[0199] Fig. 9 is a conceptual diagram showing an example additional inter-frame prediction value point 900 obtained from a first point oriented greater than the inter-frame prediction value point 914. Figure 7 In the described inter-frame prediction method for predicting geometry, when inter-frame coding is applied, the radius, orientation and laser ID of the current point (current point 700) are predicted based on points (inter-frame prediction point 704) near the collocated orientation position (reference position 710) in the reference frame (reference frame 706). Fig. 9 In the example of FIG. 1 , the G-PCC encoder 200 and the G-PCC decoder 300 may use the following steps to determine the additional inter-frame prediction value point 900: a) For a given point (current point 900 of current frame 904), determine the previous point 902 in current frame 904 ( Fig. 9 'previous decoding point' in ); b) determining a reference position 906 in a reference frame 908 having a similarity to the previous point 902 ( Fig. 9 the same scaled position and laser ID, c) determining a position in the reference frame 908 as a first point having an orientation (eg, a scaled orientation) greater than the reference position 906 determined in step b) to be used as an inter-frame prediction value point ( Fig. 9 The inter-frame prediction point 910 in .

[0200] This can be done by finding the value greater than Fig. 9 The first point ( Fig. 9 The additional inter-frame prediction value point is obtained by using the "additional inter-frame prediction point 912" in the example above. If inter-frame coding has been applied, additional signaling can be used to indicate which prediction value is selected. The additional inter-frame prediction value point can also be called the "next next" inter-frame prediction value.

[0201] In some examples, the G-PCC encoder 200 (e.g., the arithmetic coding unit 214 of the G-PCC encoder 200) and the G-PCC decoder 300 (e.g., the geometric arithmetic decoding unit 302 of the G-PCC decoder 300) can apply an improved context selection algorithm to encode and decode the inter prediction flag. The inter prediction flag values ​​of five previously encoded points can be used to select the context of the inter prediction flag in predictive geometry encoding and decoding.

[0202] The foregoing techniques may have one or more disadvantages. Inter-frame prediction uses points in a reference frame to predict points in a given frame. In some examples, a zero-compensated reference frame may be used; and in some cases, a global motion compensated reference frame may be used for prediction. In a zero-motion compensated frame, points in a reference frame are used without applying motion compensation; a global motion compensated reference frame is obtained from a (zero-compensated) reference frame by motion compensation based on a set of motion parameters; these parameters may be derived, for example, using motion estimation techniques or obtained from an external unit (e.g., using GPS parameters or from the motion of a car capturing a point cloud). When motion compensation is applied to points in a reference frame, the position of the points is modified, and this results in a reduction in encoding and decoding efficiency.

[0203] According to one or more techniques of the present disclosure, the G-PCC codec can perform resampling on a reference frame (e.g., a zero-compensated reference frame, an uncompensated reference frame, etc.). A zero-compensated reference generally refers to compensation of zero motion and zero rotation. In many cases, a zero-compensated reference and an uncompensated reference are equivalent.

[0204] The G-PCC codec can optionally perform resampling operations on points of the reference frame as follows: - For each point pt in the reference frame (zero compensated), the G-PCC codec can determine whether to apply resampling to this point. - If no resampling is applied, the G-PCC codec can keep the point as is. - Otherwise (re-sampling is applied), the G-PCC codec may select (eg, choose) a first set of points in a global compensation reference frame (GlobRef) associated with pt. - For one or more components of pt, the G-PCC codec may derive values ​​from the first set of points; and use the derived values ​​of pt for the corresponding components. o For example, if point pt is represented by (r, phi, laser ID) as components, and r_d is the radius derived from the first set of points, then point pt is updated to (r_d, phi, laser ID) and is retained.

[0205] The steps of an example resampling operation associated with a point pt are described below:

[0206] The G-PCC codec determines whether to apply resampling to a point: The G-PCC codec can use the following conditions to determine whether to apply resampling to a point: ○ Whether the point belongs to a road or whether the point belongs to a building / other object in the scene. A "road" point can be determined by one or more thresholds signaled in the bitstream; for example, if two thresholds a1 and a2 are two z-thresholds signaled to indicate road points, then for points with z-values ​​in the range [a1, a2), no resampling is applied; for other points, resampling is applied. - In some examples, all points may be resampled.

[0207] The G-PCC codec selects the first set of points associated with the current point pt in the zero-compensation frame as follows: - For a given point pt with components (r, phi, lId), a first set of points in GlobRef may be selected as points close to collocated points in GlobRef. o For example, points in GlobRef whose laser ID value is equal to lId and whose orientation is in the range [phi-increment, phi+increment], where increment is a fixed value based on the laser ID. o In another example, the azimuth scale value may be used to determine quantized azimuth values ​​for all points (e.g., phi may be quantized as phiQ). Points in the reference frame associated with quantized azimuth values ​​phiQ may be considered to be part of the bit associated with phiQ. A first set of points may be determined as points in GlobRef that have a laser ID equal to lId and belong to: ■The bits associated with phiQ, ■ the first non-empty bit associated with a quantized position value greater than phiQ, and ■ The first non-empty bit associated with a quantized position value less than phiQ. - The G-PCC codec prunes the first set of points to one or two points with bearing values ​​closest to phi. ○ In one example, if there is a point P1(r_a, phi_a, lId) belonging to GlobRef and associated with the bit phiQ: ■If phi_a=phi, only P1 is retained in the first group. ■ If phi_a>phi, then P1 is retained and another point P2 (r_b, phi_b, lId) is chosen such that phi_b <phi。 ■ If phi_a < phi, then P1 is retained and another point P3(r_c, phi_c, lId) is selected such that phi_c > phi. If no such point P1 exists, then two points P2(r_b, phi_b, lId) and point P3(r_c, phi_c, lId) are selected from the first group such that phi_b < phi and phi_c > phi. If one or more of the points P2 or P3 are not available in GlobRef, then the first group of points may not include the unavailable point(s).

[0208] The G-PCC codec can derive the radius value from the points in the first group as follows (this can also be applied to other components): - If there is only one point (r_x, phi_x, lId) in the first group, then the radius of point pt is updated to r_x. In other words, point pt is updated from (r, phi, lId) to (r_x, phi, lId). - If there are two or more points in the first group, then the radius of point pt is obtained as an interpolated value from the radius values of the points in the first group.

[0209] The following example of the technology of the present disclosure is provided for reference. In one example, the G-PCC codec can derive the global compensation reference frame GlobRef and use resampling based on the points in GlobRef to update the points in the zero compensation reference frame ZeroRef. The G-PCC codec can use the updated (resampled) zero compensation reference frame for inter-frame prediction.

[0210] Fig.10 is a flowchart showing an exemplary technique of how resampled points can be used for inter-frame prediction according to one or more techniques of the present disclosure. In Fig.10 the example, the G-PCC codec (e.g., G-PCC encoder 200 or G-PCC decoder 300) determines that an inter-frame prediction mode will be used to predict the current point in the current frame. In response to determining to use the inter-frame prediction mode to predict the current point in the current frame, the G-PCC codec identifies the zero compensation reference frame associated with the current frame, such as ZeroRef(1002). The G-PCC codec derives the motion compensation reference frame associated with the current frame, such as GlobRef(1004), and selectively resamples the points in the zero compensation reference frame based on one or more points in the motion compensation reference frame to generate a resampled zero compensation reference frame (1006). The G-PCC codec predicts the current point in the current frame based on the resampled zero compensation reference frame (1008).

[0211] The above operation (resampling) can also be implemented as resampling the points in GlobRef using the azimuth value from ZeroRef. For each point PT in ZeroRef, the G-PCC codec can determine whether to apply resampling to the point. If so, the G-PCC codec can select a first set of points from GlobRef; obtain an interpolation radius value r_int from the first set of points, and update the radius value of PT with the interpolation radius value r_int.

[0212] Fig.11 1 is a flow chart illustrating an example technique of resampling points in GlobRef using points from ZeroRef according to one or more techniques of the present disclosure. To selectively resample points in a zero compensated reference frame, the G-PCC codec selects a particular point in the zero compensated reference frame (1102) and determines whether to resample the particular point (1104). In response to determining to resample the particular point, the G-PCC codec determines a first set of points in a motion compensated reference frame associated with the particular point (1106) and derives a derived radius value based on the first set of points (1108). The G-PCC codec updates a radius value for the particular point based on the derived radius value (1110).

[0213] Figures 12A-12D is a conceptual diagram illustrating how a first set of points 1202 , 1204 , and 1206 in GlobRef may be selected for a point 1208 in ZeroRef in accordance with one or more aspects of the present disclosure. Figures 12A-12D All points shown in may have the same laser ID. Solid lines 1210, 1212, 1214, and 1216 may indicate the boundaries of the azimuth bit cells (all azimuth values ​​within a bit cell are quantized to the same value). Dashed line 1218 indicates an azimuth value equal to the azimuth of point 1208. Based on the availability of points in GlobRef in the same azimuth bit cell as point 1208, and the difference between points 1202, 1204, and 1206 (if available) and the azimuth value of point 1208, the G-PCC codec may select a first set of points from points 1202, 1204, and 1206.

[0214] exist Fig. 12A In the example of , point 1204 exists in the same position bit as point 1208, and the position of point 1204 is equal to the position of point 1208. Based on this, the G-PCC codec can select the first set of points as point 1204 (e.g., include point 1204 in the first set of points).

[0215] exist Fig. 12BIn the example of , point 1204 exists in the same azimuth bit as the PT, and the azimuth of point 1204 is greater than the azimuth of point 1208. Based on this, the G-PCC codec can select a first set of points as point 1202 and point 1204 (e.g., include point 1202 and point 1204 in the first set of points).

[0216] exist Fig. 12C In the example of , point 1204 exists in the same azimuth bit as point 1208, and the azimuth of point 1204 is less than the azimuth of point 1208. Based on this, the G-PCC codec can select a first set of points as point 1204 and point 1206 (e.g., include point 1204 and point 1206 in the first set of points).

[0217] exist Fig.12D In the example of , point 1204 does not exist in the same orientation bit as point 1208. Based on this, the G-PCC codec can select a first set of points as point 1202 and point 1206 (e.g., include point 1202 and point 1206 in the first set of points).

[0218] When the first set of points contains only one point (r_x, phi_x, lId), the radius of point PT can be set equal to r_x. When the first set of points contains two points (r_x, phi_x, lId) and (r_y, phi_y, lId), the radius of point PT can be set equal to r_xy, which can be derived as follows:

[0219] The above examples show linear interpolation of the radius points (eg, interpolation of the points of the first set of points), but higher order interpolators (eg, cubic interpolation, etc.) may also be used and the number of points in the first set may also be greater than two.

[0220] In some examples, such as when delRad is greater than a certain threshold (eg, 1024 or 2048), the G-PCC codec may not perform interpolation.

[0221] In some examples, such as when delAzim is greater than a certain threshold, or when the quantized value of delAzim is greater than a certain threshold (delAzim quantized with an azimuth scale value is greater than N, where N is a fixed number, such as 4), the G-PCC codec may not perform interpolation.

[0222] In some examples, such as when interpolation is not performed (e.g., because of a threshold), the G-PCC codec may not update the point in ZeroRef. In some examples, such as when interpolation is not performed (e.g., because of a threshold), the G-PCC codec may set the point to the radius of a point with an orientation closer to phi_PT (e.g., if abs(phi_PT-phi_x) is less than abs(phi_PT-phi_y), then r_xy may be set equal to r_x, otherwise r_xy is set equal to r_y).

[0223] Fig.14 is a flow chart illustrating an example technique for performing inter-frame prediction in accordance with one or more techniques of this disclosure. Fig.10 In the example of , a G-PCC codec (eg, G-PCC encoder 200 or G-PCC decoder 300) identifies a reference frame (1302) and derives one or more motion compensation reference points from the reference frame (1304).

[0224] The G-PCC codec selectively resamples points in a reference frame based on one or more motion compensated reference points to generate a resampled reference frame (1306). To selectively resample points in a reference frame, the G-PCC codec may be configured to: select a specific point in the reference frame; determine whether to resample the specific point; and, in response to determining to resample the specific point: determine a first set of points in the reference frame associated with the specific point, wherein the first set of points includes at least one of the one or more motion compensated reference points; derive a derived radius value based on the first set of points; and update the radius value of the specific point according to the derived radius value. To derive the derived radius value based on the first set of points, the G-PCC codec may be configured to: selectively interpolate the radius values ​​of a plurality of points to determine the derived radius value, wherein the first set of points includes a plurality of points. In order to selectively interpolate the radius values ​​of multiple points, the G-PCC codec can be configured to: in response to the difference between the radius values ​​of the first point and the second point among the multiple points being greater than a first threshold, or the difference between the azimuth values ​​of the first point and the second point being greater than one or both of a second threshold, determine not to interpolate the radius values ​​of the first point and the second point. The G-PCC codec may be configured to, in response to determining that the radius values ​​of the plurality of points are not to be interpolated, avoid updating the radius value of the particular point based on the derived radius value. The G-PCC codec may be configured to, in response to determining that the radius values ​​of the plurality of points are not to be interpolated, set the radius value of the particular point to the radius value of a single point in the first set of points. For example, the single point in the first set of points may be a point having an azimuth value that is closest to the azimuth value of the particular point.

[0225] The G-PCC codec predicts a current point in a current frame based on the resampled reference frame (1308). The G-PCC codec may reconstruct a point cloud including the current point.

[0226] Fig.14 is a conceptual diagram illustrating an example ranging system 1400 that can be used with one or more techniques of this disclosure. Fig.14 In the example of , ranging system 1400 includes illuminator 1402 and sensor 1404. Illuminator 1402 can emit light 1406. In some examples, illuminator 1402 can emit light 1406 as one or more laser beams. Light 1406 can be one or more wavelengths, such as infrared wavelengths or visible light wavelengths. In other examples, light 1406 is not a coherent laser. When light 1406 encounters an object such as object 1408, light 1406 produces return light 1410. Return light 1410 may include backscattered and / or reflected light. Return light 1410 may pass through lens 1411, which guides return light 1410 to create an image 1412 of object 1408 on sensor 1404. Sensor 1404 generates signal 1414 based on image 1412. Image 1412 may include a set of points (e.g., as represented by Fig.14 1412).

[0227] In some examples, illuminator 1402 and sensor 1404 can be mounted on a rotating structure so that illuminator 1402 and sensor 1404 capture a 360-degree view of the environment. In other examples, ranging system 1400 can include one or more optical components (e.g., mirrors, collimators, diffraction gratings, etc.) that enable illuminator 1402 and sensor 1404 to detect the range of objects within a certain range (e.g., up to 360 degrees). Although Fig.14 The example shows only a single illuminator 1402 and sensor 1404, but the ranging system 1400 may include multiple sets of illuminators and sensors.

[0228] In some examples, the illuminator 1402 generates a structured light pattern. In these examples, the ranging system 1400 may include a plurality of sensors 1404 on which respective images of the structured light pattern are formed. The ranging system 1400 may use the difference between the images of the structured light pattern to determine the distance to the object 1408 from which the structured light pattern is backscattered. When the object 1408 is relatively close to the sensor 1404 (e.g., 0.2 meters to 2 meters), the ranging system based on structured light can have a high level of accuracy (e.g., accuracy in the sub-millimeter range). This high level of accuracy can be used for facial recognition applications, such as unlocking mobile devices (e.g., mobile phones, tablet computers, etc.) and for security applications.

[0229] In some examples, ranging system 1400 is a time-of-flight (ToF) based system. In some examples where ranging system 1400 is a TOF based system, illuminator 1402 generates light pulses. In other words, illuminator 1402 can modulate the amplitude of emitted light 1406. In these examples, sensor 1404 detects return light 1410 from the light pulses 1406 generated by illuminator 1402. Then, ranging system 1400 can determine the distance to object 1408 from which light 1406 is backscattered based on the delay between when light 1406 is emitted and detected and the known speed of light in air. In some examples, illuminator 1402 can modulate the phase of emitted light 1406, instead of (or in addition to) modulating the amplitude of emitted light 1406. In these examples, sensor 1404 can detect the phase of return light 1410 from object 1408 and determine the distance to a point on object 1408 using the speed of light and based on the time difference between when illuminator 1402 generates light 1406 at a particular phase and when sensor 1404 detects return light 1410 at the particular phase.

[0230] In other examples, the point cloud can be generated without using the illuminator 1402. For example, in some examples, the sensor 1404 of the ranging system 1400 may include two or more optical cameras. In such an example, the ranging system 1400 may use the optical cameras to capture a stereoscopic image of the environment (including the object 1408). The ranging system 1400 may include a point cloud generator 1416, which can calculate the difference between the positions in the stereoscopic images. The ranging system 1400 can then use the difference to determine the distance to the position shown in the stereoscopic image. Based on these distances, the point cloud generator 1416 can generate a point cloud.

[0231] Sensor 1404 may also detect other properties of object 1408, such as color and reflectivity information. Fig.14In the example of , the point cloud generator 1416 can generate a point cloud based on the signal 1414 generated by the sensor 1404. The ranging system 1400 and / or the point cloud generator 1416 can form a data source 104 ( Figure 1 ). Therefore, according to any technique of the present disclosure, the point cloud generated by the ranging system 1400 can be encoded and / or decoded.

[0232] Fig.15 is a conceptual diagram illustrating an example vehicle-based scenario in which one or more techniques of the present disclosure may be used. Fig.15 In the example of FIG. 1 , vehicle 1500 includes ranging system 1502. Fig.15 The method discussed above implements the ranging system 1502. Fig.15 , but the vehicle 1500 may also include a data source (e.g., data source 104 ( Figure 1 )) and a G-PCC encoder (e.g., a G-PCC encoder 200 ( Figure 1 )).exist Fig.15 In the example of FIG. 1 , the ranging system 1502 emits a laser beam 1504 that reflects off a pedestrian 1506A or other object in the road. The data source of the vehicle 1500 can generate a point cloud based on the signal generated by the ranging system 1502. The G-PCC encoder of the vehicle 1500 can encode the point cloud to generate a bitstream 1508, such as a geometry bitstream ( Figure 2 ) and attribute bitstream ( Figure 2 ). Bitstream 1508 may include significantly fewer bits than the unencoded point cloud obtained by the G-PCC encoder. In some examples, the G-PCC encoder of vehicle 1500 may encode bitstream 1508 using radius interpolation as described above. In some examples, the G-PCC decoder of vehicle 1510 may decode bitstream 1508 using radius interpolation as described above.

[0233] The output interface of the vehicle 1500 (eg, the output interface 108 ( Figure 1 )) can send the bitstream 1508 to one or more other devices. The bitstream 1508 may include much fewer bits than the unencoded point cloud obtained by the G-PCC encoder. Therefore, the vehicle 1500 may be able to send the bitstream 1508 to other devices faster than the unencoded point cloud data. Additionally, the bitstream 1508 may require less data storage capacity.

[0234] exist Fig.15 In the example of FIG. 1 , vehicle 1500 may send a bitstream 1508 to another vehicle 1510. Vehicle 1510 may include a G-PCC decoder (e.g., G-PCC decoder 300 ( Figure 1 )). The G-PCC decoder of vehicle 1510 can decode bitstream 1508 to reconstruct the point cloud. Vehicle 1510 can use the reconstructed point cloud for various purposes. For example, vehicle 1510 can determine that pedestrian 1506 is located in the road in front of vehicle 1500 based on the reconstructed point cloud, and thus, for example, start to slow down even before the driver of vehicle 1510 realizes that pedestrian 1206 is located in the road. Therefore, in some examples, vehicle 1510 can perform autonomous navigation operations based on the reconstructed point cloud.

[0235] Additionally or alternatively, vehicle 1500 can send bitstream 1508 to server system 1512. Server system 1512 can use bitstream 1508 for various purposes. For example, server system 1512 can store bitstream 1508 for subsequent reconstruction of the point cloud. In this example, server system 1512 can use the point cloud and other data (e.g., vehicle telemetry data generated by vehicle 1500) to train an autonomous driving system. In other examples, server system 1512 can store bitstream 1508 for subsequent reconstruction for forensic accident investigation.

[0236] Fig.16 is a conceptual diagram illustrating an example extended reality system in which one or more techniques of the present disclosure may be used. Extended reality (XR) is a term used to encompass a range of technologies including augmented reality (AR), mixed reality (MR), and virtual reality (VR). Fig.16 In the example of FIG. 1 , user 1600 is located at a first location 1602. User 1600 wears an XR helmet 1604. As an alternative to XR helmet 1604, user 1600 may use a mobile device (e.g., a mobile phone, a tablet computer, etc.). XR helmet 1604 includes a depth detection sensor (such as a range finding system) that detects the location of a point on object 1606 at location 1602. A data source of XR helmet 1604 may generate a point cloud representation of object 1606 at location 1602 using a signal generated by the depth detection sensor. XR helmet 1604 may include a G-PCC encoder (e.g., Figure 1 1604 ), which is configured to encode the point cloud to generate a bitstream 1608. In some examples, as described above, the G-PCC encoder of the XR headset 1604 can use radius interpolation when encoding the point cloud.

[0237] The XR helmet 1604 may send a bitstream 1608 (e.g., via a network such as the Internet) to an XR helmet 1610 worn by a user 1612 at a second location 1614. The XR helmet 1610 may decode the bitstream 1608 to reconstruct the point cloud. In some examples, as described above, the G-PCC decoder of the XR headset 1604 may use radius interpolation when decoding the point cloud.

[0238] The XR helmet 1610 can use the point cloud to generate an XR visualization (e.g., AR, MR, VR visualization) representing an object 1606 at the location 1602. Thus, in some examples, for example, when the XR helmet 1610 generates a VR visualization, the user 1612 can have a 3D immersive experience of the location 1602. In some examples, the XR helmet 1610 can determine the location of the virtual object based on the reconstructed point cloud. For example, the XR helmet 1610 can determine that the environment (e.g., location 1602) includes a flat surface based on the reconstructed point cloud, and then determine that the virtual object (e.g., a cartoon character) will be positioned on the flat surface. The XR helmet 1610 can generate an XR visualization in which the virtual object is located at the determined location. For example, the XR helmet 1610 can show a cartoon character sitting on a flat surface.

[0239] Fig.17 is a conceptual diagram illustrating an example mobile device system in which one or more techniques of this disclosure may be used. Fig.17 In an example of FIG. 1 , a mobile device 1700 such as a mobile phone or tablet computer includes a range finding system such as a LIDAR system that detects the location of points on an object 1702 in the environment of the mobile device 1700. A data source of the mobile device 1700 may use a signal generated by a depth detection sensor to generate a point cloud representation of the object 1702. The mobile device 1700 may include a G-PCC encoder (e.g., Figure 1 G-PCC encoder 200), which is configured to encode the point cloud to generate a bitstream 1704.

[0240] exist Fig.17 In an example of , mobile device 1700 can send a bitstream to remote device 1706 (such as a server system or other mobile device). Remote device 1706 can decode bitstream 1704 to reconstruct the point cloud. In some examples, as described above, the G-PCC decoder of remote device 1706 can use radius interpolation when decoding the point cloud.

[0241] The remote device 1706 can use the point cloud for various purposes. For example, the remote device 1706 can use the point cloud to generate a map of the environment of the mobile device 1700. For example, the remote device 1706 can generate a map of the interior of a building based on the reconstructed point cloud. In another example, the remote device 1706 can generate an image (e.g., computer graphics) based on the point cloud. For example, the remote device 1706 can use the points of the point cloud as the vertices of a polygon and use the color attributes of the points as the basis for coloring the polygon. In some examples, the remote device 1706 can use the reconstructed point cloud for facial recognition or other security applications.

[0242] The following numbered clauses may describe one or more aspects of the present disclosure:

[0243] Item 1A: A method for processing a point cloud, the method comprising: in response to determining to use an inter-frame prediction mode to predict a current point in a current frame: identifying a zero-compensated reference frame; deriving a motion-compensated reference frame; selectively resampling points in the zero-compensated reference frame based on one or more points in the motion-compensated reference frame to generate a resampled zero-compensated reference frame; and, inter-predicting the current point in the current frame based on the resampled zero-compensated reference frame.

[0244] Clause 2A: The method of clause 1A, wherein the zero compensation reference frame is ZeroRef and the motion compensation reference frame is GlobRef.

[0245] Clause 3A: A method according to clause 1A or clause 2A, wherein selectively resampling a point in a zero-compensated reference frame includes: selecting a specific point in the zero-compensated reference frame; determining whether to resample the specific point; in response to determining that the specific point is to be resampled: determining a first set of points in a motion-compensated reference frame associated with the specific point; deriving a derived radius value based on the first set of points; and, based on the derived radius value, updating the radius value of the specific point.

[0246] Clause 4A: The method of clause 3A, wherein deriving the derived radius value based on a first set of points comprises selectively interpolating the derived radius value based on the plurality of points, wherein the first set of points comprises a plurality of points.

[0247] Clause 5A: The method of clause 4A, wherein selectively interpolating the derived radius value comprises determining not to interpolate the derived radius value if a difference between the radius values ​​of the plurality of points is greater than a threshold.

[0248] Clause 6A: The method of clause 4A, wherein selectively interpolating the derived radius value comprises determining not to interpolate the derived radius value if a difference between the orientation values ​​of the plurality of points is greater than a threshold.

[0249] Clause 7A: The method of any one of clauses 4A-6A, further comprising: in response to determining not to interpolate the derived radius value, avoiding updating the radius value of the particular point based on the derived radius value.

[0250] Clause 8A: The method of any one of clauses 4A-6A, further comprising: in response to determining not to interpolate the derived radius value, setting the radius value of the particular point to the radius value of a single point in the first set of points.

[0251] Clause 9A: The method of clause 8A, wherein a single point in the first set of points has a bearing value that is closest to the bearing value of the particular point.

[0252] Clause 10A: An apparatus for processing a point cloud, the apparatus comprising: one or more units for performing the method according to any one of clauses 1A-9A.

[0253] Clause 11A: The apparatus of clause 10A, wherein the one or more units comprise one or more processors implemented in circuitry.

[0254] Clause 12A: The apparatus of any of clauses 10A or 11A, further comprising: a memory for storing data representing a point cloud. Clause 13A: The apparatus of any of clauses 10A-12A, wherein the apparatus comprises a decoder.

[0255] Clause 14A: The apparatus of any of clauses 10A-13A, wherein the apparatus comprises an encoder.

[0256] Clause 15A: The apparatus of any of clauses 10A-14A, further comprising a device for generating a point cloud.

[0257] Clause 16A: The apparatus of any of clauses 10A-15A, further comprising: a display for presenting an image based on the point cloud.

[0258] Clause 17A: A computer-readable storage medium having stored thereon instructions which, when executed, cause one or more processors to perform the method of any of clauses 1-9A.

[0259] Item 1B: A device for processing point cloud data, the device comprising: a memory configured to store point cloud data; and one or more processors implemented in a circuit and configured to: in response to determining to use an inter-frame prediction mode to predict a current point in a current frame: identify a reference frame; derive one or more motion compensated reference points from the reference frame; selectively resample points in the reference frame based on the one or more motion compensated reference points to generate a resampled reference frame; and, based on the resampled reference frame, inter-frame predict the current point in the current frame.

[0260] Item 2B: A device according to Item 1B, wherein, in order to selectively resample a point in the reference frame, the one or more processors are further configured to: select a specific point in the reference frame; determine whether to resample the specific point; in response to determining that the specific point is to be resampled: determine a first group of points in the reference frame associated with the specific point, wherein the first group of points includes at least one of the one or more motion compensated reference points; derive a derived radius value based on the first group of points; and update the radius value of the specific point based on the derived radius value.

[0261] Clause 3B: A device according to clause 2B, wherein, in order to derive the derived radius value based on the first set of points, the one or more processors are further configured to: selectively interpolate the radius values ​​of multiple points to determine the derived radius value, wherein the first set of points includes the multiple points.

[0262] Item 4B: A device according to Item 3B, wherein, in order to selectively interpolate the radius values ​​of the multiple points, the one or more processors are further configured to: in response to the difference between the radius values ​​of a first point and a second point among the multiple points being greater than a first threshold, or the difference between the orientation values ​​of the first point and the second point being greater than one or both of a second threshold, determine not to interpolate the radius values ​​of the first point and the second point.

[0263] Clause 5B: The apparatus of clause 4B, wherein the one or more processors are further configured to: in response to determining not to interpolate the radius values ​​for the plurality of points, refrain from updating the radius value for the particular point based on the derived radius value.

[0264] Clause 6B: The apparatus of clause 4B, wherein the one or more processors are further configured to, in response to determining not to interpolate the radius values ​​of the plurality of points, set the radius value of the particular point to the radius value of a single point in the first set of points.

[0265] Clause 7B: The apparatus of clause 6B, wherein a single point in the first set of points has a bearing value that is closest to the bearing value of the particular point.

[0266] Clause 8B: The apparatus of any one of clauses 1B-7B, wherein the one or more processors are further configured to: reconstruct a point cloud including the current point to form a reconstructed point cloud.

[0267] Clause 9B: The apparatus of clause 8B, wherein the one or more processors are further configured to generate a map of an interior of a building based on the reconstructed point cloud.

[0268] Clause 10B: The apparatus of clause 8B or clause 9B, wherein the one or more processors are further configured to: perform an autonomous navigation operation based on the reconstructed point cloud.

[0269] Clause 11B: The apparatus of any of clauses 8B-10B, wherein the one or more processors are further configured to generate computer graphics based on the reconstructed point cloud.

[0270] Clause 12B: A device as described in any of clauses 8B-11B, wherein the one or more processors are configured to: determine a position of a virtual object based on the reconstructed point cloud; and generate an extended reality (XR) visualization, wherein the virtual object is at the determined position.

[0271] Clause 13B: The apparatus of any of Clauses 8B-12B, further comprising: a display for presenting an image based on the reconstructed point cloud.

[0272] Clause 14B: A device as described in any of clauses 1B-13B, wherein the device includes one or more of a mobile phone, a tablet, a vehicle, or an extended reality device.

[0273] Item 15B: A method for processing a point cloud, the method comprising: in response to determining to use an inter-frame prediction mode to predict a current point in a current frame: identifying a reference frame; deriving one or more motion compensation reference points based on the reference frame; selectively resampling points in the reference frame based on the one or more point motion compensation reference points to generate a resampled reference frame; and, based on the resampled reference frame, performing inter-frame prediction on the current point in the current frame.

[0274] Clause 16B: A method according to clause 15B, wherein selectively resampling a point in the reference frame comprises: selecting a specific point in the reference frame; determining whether to resample the specific point; in response to determining to resample the specific point: determining a first group of points in the reference frame associated with the specific point, wherein the first group of points includes at least one of the one or more motion compensated reference points; deriving a derived radius value based on the first group of points; and updating the radius value of the specific point based on the derived radius value.

[0275] Clause 17B: The method of clause 16B, wherein deriving the derived radius value based on the first set of points comprises selectively interpolating radius values ​​of a plurality of points to determine the derived radius value, wherein the first set of points includes the plurality of points.

[0276] Clause 18B: A method according to Clause 17B, wherein selectively interpolating the derived radius value includes: in response to the difference between the radius values ​​of the first point and the second point among the multiple points being greater than a first threshold, or the difference between the orientation values ​​of the first point and the second point being greater than one or both of a second threshold, determining not to interpolate the radius values ​​of the first point and the second point.

[0277] Clause 19B: The method of clause 18B, further comprising: in response to determining not to interpolate the radius values ​​of the plurality of points, refraining from updating the radius value of the particular point based on the derived radius value.

[0278] Clause 20B: The method of clause 18B, further comprising: in response to determining not to interpolate the radius values ​​of the plurality of points, setting the radius value of the particular point to the radius value of a single point of the first set of points.

[0279] Clause 21B: The method of clause 20B, wherein a single point in the first set of points has a bearing value that is closest to the bearing value of the particular point.

[0280] Clause 22B: The method of any one of clauses 15B-21B, wherein the method of processing the point cloud comprises: a method of encoding the point cloud.

[0281] Clause 23B: The method of any one of clauses 15B-21B, wherein the method of processing the point cloud comprises: a method of decoding the point cloud.

[0282] Item 24B: A computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to: in response to determining to use an inter-frame prediction mode to predict a current point in a current frame: identify a reference frame; derive one or more motion compensation reference points based on the reference frame; selectively resample points in the reference frame based on the one or more motion compensation reference points to generate a resampled reference frame; and, based on the resampled reference frame, perform inter-frame prediction on the current point in the current frame.

[0283] Clause 25B: A computer-readable storage medium according to clause 24B, wherein, in order to selectively resample a point in the reference frame, the instructions cause the one or more processors to: select a specific point in the reference frame; determine whether to resample the specific point; in response to determining that the specific point is to be resampled: determine a first group of points in the reference frame associated with the specific point, wherein the first group of points includes at least one of the one or more motion compensated reference points; derive a derived radius value based on the first group of points; and, update the radius value of the specific point based on the derived radius value.

[0284] Clause 26B: A computer-readable storage medium as described in clause 25B, wherein, in order to derive a derived radius value based on the first set of points, the instructions cause one or more processors to: selectively interpolate radius values ​​of multiple points to determine the derived radius value, wherein the first set of points includes the multiple points.

[0285] Item 27B: A computer-readable storage medium according to Item 26B, wherein, in order to selectively interpolate the derived radius value, the instructions cause the one or more processors to: in response to the difference between the radius values ​​of a first point and a second point among the multiple points being greater than a first threshold, or the difference between the orientation values ​​of the first point and the second point being greater than one or both of a second threshold, determine not to interpolate the radius values ​​of the first point and the second point.

[0286] Clause 28B: A computer-readable storage medium as described in clause 27B, wherein the instructions further cause the one or more processors to: in response to determining not to interpolate the radius values ​​of the multiple points, avoid updating the radius value of the particular point based on the derived radius value.

[0287] Clause 29B: A computer-readable storage medium according to clause 27B, wherein the instructions further cause the one or more processors to: in response to determining that the radius values ​​of the multiple points are not to be interpolated, set the radius value of the particular point to the radius value of a single point in the first set of points.

[0288] Clause 30B: The computer-readable storage medium of clause 29B, wherein the single point in the first set of points has a bearing value that is closest to the bearing value of the particular point.

[0289] It is to be appreciated that, depending on the examples, certain actions or events of any of the techniques described herein may be performed in a different order, may be added, combined, or omitted entirely (e.g., not all described actions or events are necessary to implement the techniques). In addition, in some examples, actions or events may be performed concurrently rather than sequentially, such as through multithreading, interrupt handling, or multiple processors.

[0290] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted through a computer-readable medium as one or more instructions or codes and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium or a communication medium, the communication medium including, for example, any medium that facilitates the transfer of a computer program from one place to another according to a communication protocol. In this manner, a computer-readable medium may generally correspond to (1) a non-temporary tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to obtain instructions, codes, and / or data structures for implementing the techniques described in the present disclosure. A computer program product may include a computer-readable medium.

[0291] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program codes in the form of instructions or data structures and can be accessed by a computer. In addition, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (e.g., infrared, radio and microwave), coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology (e.g., infrared, radio and microwave) are included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals or other temporary media, but are instead directed to non-temporary tangible storage media. As used herein, disks and optical disks include compact disks (CDs), laser optical disks, optical disks, digital versatile disks (DVDs), floppy disks and blue-ray disks, wherein disks usually copy data magnetically, while optical disks use lasers to optically copy data. Combinations of the above should also be included within the scope of computer-readable media.

[0292] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the terms "processor" and "processing circuitry" as used herein may refer to any of the aforementioned structures or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. In addition, the techniques may be implemented entirely in one or more circuits or logic elements.

[0293] The techniques of the present disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in the present disclosure to emphasize the functional aspects of a device configured to perform the disclosed techniques, but do not necessarily need to be implemented by different hardware units. Specifically, as described above, the various units may be combined in a codec hardware unit, or provided by a collection of interoperable hardware units (including one or more processors as described above) in combination with appropriate software and / or firmware.

[0294] Various examples have been described. These and other examples are within the scope of the following clauses.

Claims

1. A device for processing point cloud data, the device comprising: A memory configured to store the point cloud data; as well as One or more processors embodied in circuitry and configured to, in response to determining to use an inter-frame prediction mode to predict a current point in a current frame: Identify the reference frame; deriving one or more motion compensation reference points from the reference frame; selectively resampling points in the reference frame based on the one or more motion compensated reference points to generate a resampled reference frame; as well as Based on the resampled reference frame, inter-frame prediction is performed on the current point in the current frame.

2. The device according to claim 1, wherein: To selectively resample the points in the reference frame, the one or more processors are further configured to: selecting a particular point in the reference frame; determining whether to resample the specific point; In response to determining to resample the specific point: determining a first set of points in the reference frame associated with the particular point, wherein the first set of points includes at least one of the one or more motion compensated reference points; deriving a derived radius value based on the first set of points; and The radius value of the specific point is updated based on the derived radius value.

3. The device according to claim 2, wherein: To derive the derived radius value based on the first set of points, the one or more processors are further configured to: The radius values ​​of a plurality of points are selectively interpolated to determine the derived radius value, wherein the first set of points includes the plurality of points.

4. The device according to claim 3, wherein: To selectively interpolate the radius values ​​of the plurality of points, the one or more processors are further configured to: In response to the difference between the radius values ​​of the first point and the second point among the multiple points being greater than a first threshold, or the difference between the orientation values ​​of the first point and the second point being greater than a second threshold, one or both, it is determined not to interpolate the radius values ​​of the first point and the second point.

5. The device according to claim 4, wherein: The one or more processors are further configured to: In response to determining not to interpolate the radius values ​​for the plurality of points, updating the radius value for the particular point based on the derived radius value is avoided.

6. The device according to claim 4, wherein: The one or more processors are further configured to: In response to determining not to interpolate the radius values ​​of the plurality of points, the radius value of the particular point is set to a radius value of a single point in the first set of points.

7. The device according to claim 6, wherein: The single point in the first set of points has a bearing value that is closest to the bearing value of the particular point.

8. The device according to claim 1, wherein: The one or more processors are further configured to reconstruct a point cloud including the current point to form a reconstructed point cloud.

9. The device according to claim 8, wherein: The one or more processors are further configured to generate a map of the interior of the building based on the reconstructed point cloud.

10. The device according to claim 8, wherein: The one or more processors are further configured to perform autonomous navigation operations based on the reconstructed point cloud.

11. The device according to claim 8, wherein: The one or more processors are further configured to generate computer graphics based on the reconstructed point cloud.

12. The device according to claim 8, wherein: The one or more processors are configured to: determining a position of a virtual object based on the reconstructed point cloud; and An extended reality (XR) visualization is generated, wherein the virtual object is at the determined location.

13. The apparatus according to claim 8, further comprising: A display for presenting an image based on the reconstructed point cloud.

14. The apparatus according to claim 1, wherein: The device includes one or more of a mobile phone, a tablet computer, a vehicle, or an extended reality device.

15. A method for processing a point cloud, the method comprising: In response to determining to predict the current point in the current frame using an inter-frame prediction mode: Identify the reference frame; deriving one or more motion compensation reference points from the reference frame; selectively resampling points in the reference frame based on the one or more point motion compensated reference points to generate a resampled reference frame; as well as Based on the resampled reference frame, inter-frame prediction is performed on the current point in the current frame.

16. The method according to claim 15, wherein: Selectively resampling the points in the reference frame comprises: selecting a particular point in the reference frame; determining whether to resample the specific point; In response to a determination to resample the particular point: determining a first set of points in the reference frame associated with the particular point, wherein the first set of points includes at least one of the one or more motion compensated reference points; deriving a derived radius value based on the first set of points; and The radius value of the specific point is updated based on the derived radius value.

17. The method according to claim 16, wherein: Deriving the derived radius value based on the first set of points includes: The radius values ​​of a plurality of points are selectively interpolated to determine the derived radius value, wherein the first set of points includes the plurality of points.

18. The method according to claim 17, wherein: Selectively interpolating the derived radius value comprises: In response to the difference between the radius values ​​of the first point and the second point among the multiple points being greater than a first threshold, or the difference between the orientation values ​​of the first point and the second point being greater than a second threshold, one or both, it is determined not to interpolate the radius values ​​of the first point and the second point.

19. The method according to claim 18, further comprising: In response to determining not to interpolate the radius values ​​for the plurality of points, updating the radius value for the particular point based on the derived radius value is avoided.

20. The method of claim 18, further comprising: In response to determining not to interpolate the radius values ​​of the plurality of points, the radius value of the particular point is set to a radius value of a single point in the first set of points.

21. The method according to claim 20, wherein: The single point in the first set of points has a bearing value that is closest to the bearing value of the particular point.

22. The method according to claim 15, wherein: The method for processing the point cloud includes: a method for encoding the point cloud.

23. The method according to claim 15, wherein: The method for processing the point cloud includes: a method for decoding the point cloud.

24. A computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to: In response to determining to use an inter-prediction mode to predict a current point in a current frame: Identify the reference frame; deriving one or more motion compensation reference points from the reference frame; selectively resampling points in the reference frame based on the one or more point motion compensated reference points to generate a resampled reference frame; and Inter-frame prediction is performed on the current point in the current frame based on the resampled reference frame.

25. The computer-readable storage medium of claim 24, wherein: To selectively resample the points in the reference frame, the instructions cause the one or more processors to: selecting a particular point in the reference frame; determining whether to resample the specific point; In response to determining to resample the specific point: determining a first set of points in the reference frame associated with the particular point, wherein the first set of points includes at least one of the one or more motion compensated reference points; deriving a derived radius value based on the first set of points; and The radius value of the specific point is updated based on the derived radius value.

26. The computer-readable storage medium of claim 25, wherein: To derive the derived radius value based on the first set of points, the instructions cause the one or more processors to: The derived radius value is determined by selectively interpolating radius values ​​of a plurality of points, wherein the first set of points includes the plurality of points.

27. The computer-readable storage medium of claim 26, wherein: To selectively interpolate the derived radius values, the instructions cause the one or more processors to: In response to the difference between the radius values ​​of the first point and the second point among the multiple points being greater than a first threshold, or the difference between the orientation values ​​of the first point and the second point being greater than a second threshold, one or both, it is determined not to interpolate the radius values ​​of the first point and the second point.

28. The computer-readable storage medium of claim 27, wherein: The instructions further cause the one or more processors to: In response to determining not to interpolate the radius values ​​for the plurality of points, updating the radius value for the particular point based on the derived radius value is avoided.

29. The computer-readable storage medium of claim 27, wherein: The instructions further cause the one or more processors to: In response to determining not to interpolate the radius values ​​of the plurality of points, the radius value of the particular point is set to a radius value of a single point in the first set of points.

30. The computer-readable storage medium of claim 29, wherein: The single point in the first set of points has a bearing value that is closest to the bearing value of the particular point.

Images