Reference frame for point cloud compression

Abstract

An apparatus for decoding point cloud data includes: one or more memories configured to store the point cloud data; and processing circuitry coupled to the one or more memories, where the processing circuitry is configured to: apply a first process to the reference point cloud frame to generate a first level of processed frame; applying a second process to the first level processed frame to generate a second level processed frame; performing inter-frame prediction decoding on geometric data of points of the current point cloud frame by using the first-level processed frame; and performing inter-frame prediction decoding on the attribute data of the points of the current point cloud frame by using the second-level processed frame.

Description

Reference frames for point cloud compression

[0001] This application claims priority to U.S. Patent Application No. 18 / 911,872, filed October 10, 2024; U.S. Provisional Patent Application No. 63 / 589,602, filed October 11, 2023; and U.S. Provisional Patent Application No. 63 / 620,580, filed January 12, 2024, the entire contents of each of which are incorporated herein by reference. U.S. Patent Application No. 18 / 911,872, filed October 10, 2024, claims the benefit of U.S. Provisional Patent Application No. 63 / 589,602, filed October 11, 2023; and U.S. Provisional Patent Application No. 63 / 620,580, filed January 12, 2024. Technical Field

[0002] This disclosure relates to point cloud encoding and decoding. Background Technology

[0003] A point cloud is a collection of points in three-dimensional space. These points can correspond to points on objects within that space. Therefore, point clouds can be used to represent the physical content of three-dimensional space. Point clouds have practical applications in a wide variety of situations. For example, point clouds can be used in the context of autonomous vehicles to represent the position of objects on a road. In another example, point clouds can be used in the context of representing the physical content of an environment to locate virtual objects in augmented reality (AR) or mixed reality (MR) applications. Point cloud compression is the process of encoding and decoding point clouds. Encoding point clouds reduces the amount of data required to store and transmit them. Summary of the Invention

[0004] Overall, this disclosure describes techniques for generating predictive samples for decoding (e.g., encoding or decoding) a current frame of point cloud data (e.g., improving inter-frame prediction for predictive geometric decoding of point clouds). For example, an encoder or decoder may perform one or more levels of processing on a reference frame of the point cloud data and decode (e.g., encode or decode) the current frame of the point cloud data based on the processed reference frame. Utilizing the example techniques described in this disclosure, the amount of information stored for decoding can be reduced while minimizing image quality loss. Therefore, the example techniques can improve buffer usage in practical applications for encoding or decoding point cloud data (e.g., reducing the amount of information that needs to be stored).

[0005] In one example, this disclosure describes an apparatus for decoding point cloud data, the apparatus comprising: one or more memories configured to store the point cloud data; and processing circuitry coupled to the one or more memories, wherein the processing circuitry is configured to: apply a first process to a reference point cloud frame to generate a first-level processed frame; apply a second process to the first-level processed frame to generate a second-level processed frame; perform inter-frame prediction decoding on geometric data of points in a current point cloud frame using the first-level processed frame; and perform inter-frame prediction decoding on attribute data of points in the current point cloud frame using the second-level processed frame.

[0006] In one example, this disclosure describes an apparatus for encoding point cloud data, the apparatus comprising: one or more memories configured to store the point cloud data; and processing circuitry coupled to the one or more memories, wherein the processing circuitry is configured to: apply a first process to a reference point cloud frame to generate a first-level processed frame; apply a second process to the first-level processed frame to generate a second-level processed frame; perform inter-frame predictive coding on geometric data of points in a current point cloud frame using the first-level processed frame; and perform inter-frame predictive coding on attribute data of points in the current point cloud frame using the second-level processed frame.

[0007] In one example, this disclosure describes a method for decoding point cloud data, the method comprising: applying a first process to a reference point cloud frame to generate a first-level processed frame; applying a second process to the first-level processed frame to generate a second-level processed frame; performing inter-frame prediction decoding on the geometric data of points in a current point cloud frame using the first-level processed frame; and performing inter-frame prediction decoding on the attribute data of points in the current point cloud frame using the second-level processed frame.

[0008] In one example, this disclosure describes one or more computer-readable storage media storing instructions that, when executed, cause one or more processors to: apply a first process to a reference point cloud frame to generate a first-level processed frame; apply a second process to the first-level processed frame to generate a second-level processed frame; perform inter-frame predictive coding on the geometric data of points in a current point cloud frame using the first-level processed frame; and perform inter-frame predictive coding on the attribute data of points in the current point cloud frame using the second-level processed frame.

[0009] Details of one or more examples are set forth in the accompanying drawings and the following description. Other features, objects, and advantages will be apparent from the description, drawings, and claims. Attached Figure Description

[0010] Figure 1 is a block diagram illustrating an example encoding and decoding system that can perform the techniques of this disclosure.

[0011] Figure 2 is a block diagram illustrating an example geometric point cloud compression (G-PCC) encoder.

[0012] Figure 3 is a block diagram illustrating an example G-PCC decoder.

[0013] Figure 4 is a conceptual diagram illustrating an example of octree partitioning used for geometric decoding.

[0014] Figure 5 is a conceptual diagram illustrating an example of a prediction tree.

[0015] Figures 6A and 6B are conceptual diagrams illustrating examples of rotating LiDAR acquisition models.

[0016] Figure 7 is a conceptual diagram illustrating an example of inter-frame prediction of the current point (curPoint) from a point (interPredPt) in the reference frame.

[0017] Figure 8 is a block diagram illustrating the example geometric coding unit of Figure 2 in more detail.

[0018] Figure 9 is a block diagram illustrating the example attribute encoding unit of Figure 2 in more detail.

[0019] Figure 10 is a block diagram illustrating the example geometric decoding unit of Figure 3 in more detail.

[0020] Figure 11 is a block diagram illustrating the example attribute decoding unit of Figure 3 in more detail.

[0021] Figure 12 is a flowchart illustrating an example of decoding point cloud data.

[0022] Figure 13 is a conceptual diagram illustrating an example of identifying additional inter-frame prediction sub-points obtained from a first point having an azimuth angle greater than that of the inter-frame prediction point.

[0023] Figure 14 is a flowchart illustrating an example of generating a compensated reference frame from a reference frame.

[0024] Figure 15 is a conceptual diagram illustrating an example of azimuth resampling for motion-compensated reference.

[0025] Figure 16 is a flowchart illustrating an example of spherical coordinate transformation.

[0026] Figure 17 is a flowchart illustrating examples of values ​​used for geometry and attribute prediction.

[0027] Figure 18 is a flowchart illustrating an example of generating a spherical table for inter-frame prediction.

[0028] Figure 19 is a flowchart illustrating an example of applying the first process to a reference frame.

[0029] Figure 20 is a flowchart illustrating another example of applying the first process to a reference frame.

[0030] Figure 21 is a flowchart illustrating another example of generating a spherical table for inter-frame prediction.

[0031] Figure 22 is a flowchart illustrating an example of applying scaling and offset.

[0032] Figure 23 is a flowchart illustrating an example of generating a processed frame based on a spherical table.

[0033] Figure 24 is a flowchart illustrating another example of values ​​used for geometry and attribute prediction.

[0034] Figure 25 is a flowchart illustrating an example of a point being discarded.

[0035] Figure 26 is a flowchart illustrating an example of attaching back to a discarded point.

[0036] Figure 27 is a flowchart illustrating an example of outputting a list of discarded points.

[0037] Figure 28 is a conceptual diagram illustrating an example ranging system that can be used with one or more technologies of this disclosure.

[0038] Figure 29 is a conceptual diagram illustrating an example vehicle-based scenario in which one or more technologies of this disclosure may be used.

[0039] Figure 30 is a conceptual diagram illustrating an example extended reality system in which one or more technologies of the present disclosure may be used.

[0040] Figure 31 is a conceptual diagram illustrating an example mobile device system in which one or more technologies of this disclosure may be used.

[0041] Figure 32 is a flowchart illustrating an example technique for encoding point cloud data.

[0042] Figure 33 is a flowchart illustrating an example technique for decoding point cloud data. Detailed Implementation

[0043] A point cloud contains a set of points in 3D space and may have attributes associated with those points. Attributes may be color information such as R, G, B or Y, Cb, Cr, or reflectivity information, or other attributes. Point clouds can be captured by various cameras or sensors (such as LiDAR, 3D scanners) and can also be generated by computers. Point cloud data refers to attribute data (e.g., color, reflectivity, etc.) and / or geometric data (e.g., coordinates). Point cloud data is used in a variety of applications, including but not limited to architecture (e.g., modeling), graphics (e.g., 3D models for visualization and animation), and the automotive industry (e.g., LiDAR sensors for navigation aids).

[0044] Point cloud data can be represented as frames, where a frame includes point cloud data captured over time (e.g., a frame includes point data of points captured during one rotation of a LiDAR sensor). A point cloud encoder can be configured to encode the point cloud data to compress the amount of information communicated for a frame by sending signals, and a point cloud decoder can be configured to decode the point cloud data to reconstruct the frame.

[0045] Examples of encoding and decoding techniques include inter-frame predictive coding or decoding and intra-frame predictive coding or decoding. In inter-frame predictive coding or decoding, a point cloud encoder or decoder determines the geometric and attribute data of a point in the current frame based on the geometric and attribute data of points in a reference frame. The point used as a reference for the geometric and attribute data in the reference frame can be the same point or can be different points (e.g., one point serves as the reference for the geometric data, while another point serves as the reference for the attribute data). In intra-frame predictive coding or decoding, a point cloud encoder or decoder determines the geometric and attribute data of a point in a frame based on the geometric and attribute data of one or more points in the same frame.

[0046] In one or more examples, a point cloud encoder and a point cloud decoder may apply a first process to a reference point cloud frame to generate a first-level processed frame. The point cloud encoder may perform inter-frame predictive coding of the geometric data, and the point cloud decoder may use the first-level processed frame to perform inter-frame predictive decoding of the geometric data of points in the current point cloud frame.

[0047] According to one or more examples described in this disclosure, a point cloud encoder and a point cloud decoder may apply a second process to the first-level processed frame to generate a second-level processed frame, instead of returning a reference point cloud frame to generate information for attribute data encoding or decoding. The point cloud encoder may perform inter-frame predictive coding on the attribute data, and the point cloud decoder may use the second-level processed frame to perform inter-frame predictive decoding on the attribute data of points in the current point cloud frame.

[0048] In one or more examples, the second-level processed frame may also include coordinate information. This coordinate information is used to encode or decode attribute data. For example, actual attribute data (e.g., color, reflectivity, etc.) may be associated with the coordinate information of points in the second-level processed frame. Therefore, both the first-level processed frame and the second-level processed frame may include coordinate information, but the coordinate information in the first-level processed frame can be used for inter-frame predictive coding or decoding of geometric data, and the coordinate information in the second-level processed frame can be used for inter-frame predictive coding or decoding of attribute data.

[0049] In this way, instead of using two separate buffers—one for inter-frame prediction geometry data and another for inter-frame prediction attribute data—the point cloud encoder and decoder can maintain a buffer storing the first-level processed frames used for inter-frame prediction geometry data. For inter-frame prediction attribute data, the point cloud encoder and decoder can generate the second-level processed frames from the first-level processed frames.

[0050] In other words, since the point cloud encoder and decoder generate the second-level processed frame from the first-level processed frame, the amount of data maintained from the completion of encoding or decoding of the reference point cloud frame to the start of encoding or decoding of the current point cloud frame is reduced. For example, it is not necessary to store both the first-level processed frame and the second-level processed frame from the completion of encoding or decoding of the reference point cloud frame to the start of encoding or decoding of the current point cloud frame. In this way, the example technique can reduce the amount of data maintained frame by frame, thereby reducing memory storage requirements.

[0051] As described above, the point cloud encoder and decoder can apply the first process to the reference point cloud frame to generate the first-level processed frame. Furthermore, the point cloud encoder and decoder can apply the second process to the first-level processed frame to generate the second-level processed frame. In one or more examples, points in the current frame and the reference frame can be represented by spherical coordinates including one or more of a radius component, an azimuth component (sometimes called phi), and a laser identification component. This can be useful because (e.g., in a LiDAR system) each laser used to capture a point may be at a different elevation or height. The point cloud encoder and decoder can use these spherical coordinates to apply the first and second processes.

[0052] As an example, to apply this first process, the point cloud encoder and decoder may, for each of the plurality of quantized azimuth components and for the laser marker component, store the radius and azimuth components of k points (e.g., greater than or equal to 1) of the reference point cloud frame associated with the laser marker component in a table. In this example, each of the plurality of quantized azimuth components is an index to the table, and the table is at least a portion of the first-level processed frames. In some examples, the value of k may be communicated or received by sending a signal, but in some examples, it may be preset.

[0053] As an example, to apply this second process, the point cloud encoder and decoder can apply offsets and scaling to one or more of the radius component, azimuth component, and laser marking component for each point in the first-level processed frame to generate the second-level processed frame. In some examples, the values ​​of the offset and scaling can be notified or received by sending signals, but in some examples, they can be preset.

[0054] Figure 1 is a block diagram illustrating an example encoding and decoding system 100 that can perform the techniques of this disclosure. The techniques of this disclosure generally relate to decoding (encoding and / or decoding) point cloud data, i.e., supporting point cloud compression. Generally, point cloud data includes any data used for processing point clouds. This decoding can be effective in compressing and / or decompressing point cloud data.

[0055] As shown in Figure 1, system 100 includes a source device 102 and a destination device 116. The source device 102 provides encoded point cloud data for decoding by the destination device 116. Specifically, in the example of Figure 1, the source device 102 provides point cloud data to the destination device 116 via a computer-readable medium 110. The source device 102 and destination device 116 can include any of a wide range of devices, including desktop computers, laptop computers, tablet computers, set-top boxes, mobile phones (such as smartphones), televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, land or sea vehicles, spacecraft, aircraft, robots, LiDAR devices, satellites, etc. In some cases, the source device 102 and destination device 116 may be equipped for wireless communication.

[0056] In the example of Figure 1, source device 102 includes a data source 104, a memory 106, a geometric point cloud compression (G-PCC) encoder 200, and an output interface 108. Destination device 116 includes an input interface 122, a G-PCC decoder 300, a memory 120, and a data consumer 118. According to this disclosure, the G-PCC encoder 200 of source device 102 and the G-PCC decoder 300 of destination device 116 can be configured to apply the techniques of this disclosure related to the decoding of point cloud data. Thus, source device 102 represents an example of an encoding device, while destination device 116 represents an example of a decoding device. In other examples, source device 102 and destination device 116 may include other components or arrangements. For example, source device 102 may receive data (e.g., point cloud data) from an internal or external source. Similarly, destination device 116 may interface with an external data consumer without including the data consumer in the same device.

[0057] The system 100 shown in Figure 1 is merely an example. Typically, other digital encoding and / or decoding devices can perform the techniques of this disclosure related to the decoding of point cloud data. Source device 102 and destination device 116 are merely examples of such devices, where source device 102 generates decoded data for transmission to destination device 116. This disclosure refers to a “decoding” device as a device that performs the decoding (e.g., encoding and / or decoding) of data. Thus, G-PCC encoder 200 and G-PCC decoder 300 represent examples of decoding devices, specifically, encoder and decoder, respectively. In some examples, source device 102 and destination device 116 can operate in a substantially symmetrical manner, such that each of source device 102 and destination device 116 includes both encoding and decoding components. Therefore, system 100 can support one-way or two-way transmission between source device 102 and destination device 116, for example, for streaming, playback, broadcasting, telephone, navigation, and other applications.

[0058] Generally, data source 104 represents the source of data (i.e., raw, unencoded point cloud data) and can provide a series of sequential "frames" of data to G-PCC encoder 200, which encodes the data in the frames. Data source 104 of source device 102 may include point cloud capture devices such as any of a variety of cameras or sensors (e.g., a 3D scanner or a light detection and ranging (LIDAR) device, one or more cameras), archives containing previously captured data, and / or data feed interfaces receiving data from data content providers. Alternatively or additionally, the point cloud data may be computer-generated from a scanner, camera, sensor, or other data source. For example, data source 104 may generate computer graphics-based data as source data, or produce a combination of real-time data, archived data, and computer-generated data. In each case, G-PCC encoder 200 encodes the captured data, pre-captured data, or computer-generated data. G-PCC encoder 200 may rearrange the frames from the received order (sometimes referred to as "display order") to a decoding order for decoding. The G-PCC encoder 200 can generate one or more bit streams including encoded data. The source device 102 can then output the encoded data to a computer-readable medium 110 via the output interface 108 for reception and / or retrieval by, for example, the input interface 122 of the destination device 116.

[0059] The memory 106 of the source device 102 and the memory 120 of the destination device 116 may represent general-purpose memory. In some examples, memory 106 and memory 120 may store raw data, such as raw data from data source 104 and raw decoded data from G-PCC decoder 300. Additionally or alternatively, memory 106 and memory 120 may store software instructions that can be executed by, for example, G-PCC encoder 200 and G-PCC decoder 300. Although memory 106 and memory 120 are shown separately from G-PCC encoder 200 and G-PCC decoder 300 in this example, it should be understood that G-PCC encoder 200 and G-PCC decoder 300 may also include internal memory for functionally similar or equivalent purposes. Furthermore, memory 106 and memory 120 may store encoded data, such as output from G-PCC encoder 200 and input to G-PCC decoder 300. In some examples, portions of memory 106 and memory 120 may be allocated as one or more buffers, for example, to store raw, decoded data and / or encoded data. For instance, memory 106 and memory 120 may store data representing point clouds.

[0060] Computer-readable medium 110 can represent any type of medium or device capable of transmitting encoded data from source device 102 to destination device 116. In one example, computer-readable medium 110 represents a communication medium enabling source device 102 to transmit encoded data directly to destination device 116 in real time, for example, via a radio frequency network or a computer-based network. Output interface 108 can modulate the transmitted signal including the encoded data, and input interface 122 can demodulate the received transmitted signal according to a communication standard, such as a wireless communication protocol. The communication medium can include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form part of a packet-based network such as a local area network, a wide area network, or a global network (such as the Internet). The communication medium can include a router, switch, base station, or any other equipment that may be useful for facilitating communication from source device 102 to destination device 116.

[0061] In some examples, source device 102 can output encoded data from output interface 108 to storage device 112. Similarly, destination device 116 can access encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as hard disk drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded data.

[0062] In some examples, source device 102 may output encoded data to file server 114 or another intermediate storage device that may store the encoded data generated by source device 102. Destination device 116 may access the stored data from file server 114 via streaming or downloading. File server 114 may be any type of server device capable of storing encoded data and sending it to destination device 116. File server 114 may represent a web server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Destination device 116 may access the encoded data from file server 114 via any standard data connection, including an internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both, suitable for accessing encoded data stored on file server 114. File server 114 and input interface 122 may be configured to operate according to a streaming protocol, a downloading protocol, or a combination thereof.

[0063] Output interface 108 and input interface 122 can represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 can be configured to transmit data (such as encoded data) according to cellular communication standards (such as 4G, 4G-LTE (Long Term Evolution), Advanced LTE, 5G, etc.). In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 can be configured according to specifications such as IEEE 802.11, IEEE 802.15 (e.g., ZigBee). ™ ),Bluetooth ™ Other wireless standards, such as the standard, are used to transmit data (such as encoded data). In some examples, source device 102 and / or destination device 116 may include corresponding system-on-chip (SoC) devices. For example, source device 102 may include an SoC device for performing functions belonging to G-PCC encoder 200 and / or output interface 108, and destination device 116 may include an SoC device for performing functions belonging to G-PCC decoder 300 and / or input interface 122.

[0064] The technology disclosed herein can be applied to encoding and decoding point cloud data to support any of a variety of applications, such as: communication between autonomous vehicles; communication between scanners, cameras, sensors and processing devices (such as local or remote servers, geographic servers).

[0065] The input interface 122 of the destination device 116 receives an encoded bitstream from a computer-readable medium 110 (e.g., a communication medium, storage device 112, file server 114, etc.). The encoded bitstream may include signaling information defined by the G-PCC encoder 200 and also used by the G-PCC decoder 300, such as syntax elements having values ​​describing the characteristics and / or processing of the decoded units (e.g., slices, pictures, picture groups, sequences, etc.). The data consumer 118 uses the decoded data. For example, the data consumer 118 may use the decoded data to determine the location of a physical object. In some examples, the data consumer 118 may include a display for presenting an image based on a point cloud.

[0066] The G-PCC encoder 200 and G-PCC decoder 300 can each be implemented as any of a variety of suitable encoder and / or decoder circuit systems, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is partially implemented in software, the device may store instructions for the software in a suitable non-transitory computer-readable medium and execute the instructions in hardware using one or more processors to perform the technology of this disclosure. Each of the G-PCC encoder 200 and G-PCC decoder 300 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) in the respective device. Devices including the G-PCC encoder 200 and / or G-PCC decoder 300 may include one or more integrated circuits, microprocessors, and / or other types of devices.

[0067] The G-PCC encoder 200 and G-PCC decoder 300 can operate according to decoding standards such as the Video Point Cloud Compression (V-PCC) standard or the Geometric Point Cloud Compression (G-PCC) standard. This disclosure can generally relate to the decoding (e.g., encoding and decoding) of images, thus including processes for encoding or decoding data. Encoded bitstreams typically include a series of values ​​for syntax elements representing decoding decisions (e.g., decoding modes).

[0068] This disclosure may generally relate to "signaling" certain information (such as syntax elements). The term "signaling" can generally refer to communication of values ​​for syntax elements and / or other data for decoding encoded data. That is, the G-PCC encoder 200 may signal values ​​for syntax elements in the bitstream. Generally, signaling refers to generating values ​​in the bitstream. As noted above, source device 102 may transmit the bitstream to destination device 116 substantially in real time or not in real time (such as when syntax elements are stored in storage device 112 for later retrieval by destination device 116).

[0069] ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) and the latest ISO / IEC MPEG 3DG (JTC 1 / SC29 / WG 7) are investigating the potential need for standardization of point cloud decoding techniques with compression capabilities significantly exceeding current methods, and are working towards establishing such a standard. This exploration is being conducted collaboratively within a team called the 3D Graphics Team (3DG) to evaluate compression technology designs proposed by experts in the field.

[0070] Point cloud compression activities are categorized into two distinct approaches. The first is “Video Point Cloud Compression” (V-PCC), which segments a 3D object and projects these segments onto multiple 2D planes (represented as “patches” in 2D frames), which are then decoded by an older 2D video codec, such as the High Efficiency Video Decoding (HEVC) (ITU-T H.265) codec. The second approach is “Geometry-Based Point Cloud Compression” (G-PCC), which directly compresses the 3D geometry (i.e., the location of a set of points in 3D space) and associated attribute values ​​(for each point associated with the 3D geometry). G-PCC addresses point cloud compression in both Category 1 (static point clouds) and Category 3 (dynamically acquired point clouds). The latest draft of the G-PCC standard is available in the G-PCC DIS (ISO / IEC JTC1 / SC29 / WG11 w19088, Brussels, Belgium, January 2020), and the codec specification is available in the G-PCC Codec Specification v6 (ISO / IEC JTC1 / SC29 / WG11 w19091, Brussels, Belgium, January 2020).

[0071] A point cloud contains a set of points in 3D space and can have attributes associated with those points. Attributes can be color information such as R, G, B or Y, Cb, Cr, or reflectivity information, or other attributes. Point clouds can be captured by various cameras or sensors, such as LiDAR sensors and 3D scanners, and can also be computer-generated. Point cloud data is used in a variety of applications, including but not limited to architecture (modeling), graphics (3D models for visualization and animation), and the automotive industry (LiDAR sensors for navigation aids).

[0072] The 3D space occupied by point cloud data can be enclosed by virtual bounding boxes. The position of a point within the bounding box can be represented with a certain precision; therefore, the position of one or more points can be quantized based on this precision. At the smallest level, the bounding box is divided into voxels, which are the smallest spatial units represented by a unit cube. A voxel in the bounding box can be associated with zero, one, or more points. The bounding box can be divided into multiple cubic / cuboid regions, which can be called tiles. Each tile can be decoded into one or more slices. Dividing the bounding box into slices and tiles can be based on the number of points in each partition, or on other considerations (e.g., a specific region can be decoded into a tile). Slice regions can be further subdivided using splitting decisions similar to those in video codecs.

[0073] Figure 2 provides an overview of the G-PCC encoder 200. Figure 3 provides an overview of the G-PCC decoder 300. The modules shown are logical and do not necessarily correspond one-to-one with the implemented code. In the example of Figure 2, the G-PCC encoder 200 may include a geometry encoding unit 250 and an attribute encoding unit 260. Generally, the geometry encoding unit 250 is configured to encode the positions of points in a point cloud frame to produce a geometry bitstream 203. The attribute encoding unit 260 is configured to encode the attributes of the points in the point cloud frame to produce an attribute bitstream 205. As will be explained below, the attribute encoding unit 260 may also use the positions and the encoded geometry (e.g., reconstructed) from the geometry encoding unit 250 to encode the attributes.

[0074] In the example of Figure 3, the G-PCC decoder 300 may include a geometry decoding unit 350 and an attribute decoding unit 360. Generally, the geometry decoding unit 350 is configured to decode the geometry bitstream 203 to recover the position of points in the point cloud frame. The attribute decoding unit 360 is configured to decode the attribute bitstream 205 to recover the attributes of the points in the point cloud frame. As will be explained below, the attribute decoding unit 360 may also use this position from the decoded geometry (e.g., reconstructed) from the geometry decoding unit 350 to encode the attribute.

[0075] In both the G-PCC encoder 200 and the G-PCC decoder 300, the point cloud positions are decoded first. Attribute decoding depends on the decoding geometry. In Figures 8 through 11 of this disclosure, the decoding unit with vertically shaded lines is generally an option for Category 1 data. The diagonally shaded decoding unit is generally an option for Category 3 data. All other modules are shared between Category 1 and Category 3.

[0076] For Category 3 data, the compressed geometry is typically represented as a leaf-level octree from the root down to the individual voxels. For Category 1 data, the compressed geometry is typically represented by a pruned octree (i.e., a leaf-level octree from the root down to blocks larger than voxels) plus a model approximating the surface of each leaf of the pruned octree. In this way, both Category 1 and Category 3 data share the octree decoding mechanism, while Category 1 data can additionally approximate each voxel within a leaf using a surface model. The surface model used is a triangulation of 1-10 triangles per block, forming a triangle soup. Therefore, the Category 1 geometry codec is called a Trisoup geometry codec, while the Category 3 geometry codec is called an octree geometry codec.

[0077] At each node of the octree, occupancy is signaled for one or more of its child nodes (up to eight nodes) when not inferred. Multiple neighborhoods are specified, including (a) nodes sharing a face with the current octree node, (b) nodes sharing a face, edge, or vertex with the current octree node, etc. Within each neighborhood, occupancy of the current node or its child nodes can be predicted using the occupancy of the node and / or its child nodes. For points sparsely filled in some nodes of the octree, the codec also supports a direct decoding mode, in which the 3D position of the point is directly encoded. A signaling flag can be sent to indicate that the direct mode is signaled. At the lowest level, the number of points associated with an octree node / leaf node can also be decoded.

[0078] Once the geometry is decoded, the attributes corresponding to that geometry point are decoded. When there are multiple attribute points corresponding to a reconstructed / decoded geometry point, the attribute values ​​representing the reconstructed point can be derived. For example, Figure 4 illustrates an octree segment 400 used for geometry decoding, where points 402 to 408 each represent points at different levels of the octree segment.

[0079] There are three attribute decoding methods in G-PCC: Region Adaptive Hierarchical Transform (RAHT) decoding, interpolation-based hierarchical nearest neighbor prediction (prediction transform), and interpolation-based hierarchical nearest neighbor prediction (lifting transform) with update / lifting steps. RAHT and lifting are typically used for Class 1 data, while prediction is typically used for Class 3 data. However, any method can be used for any data, and like the geometry codec in G-PCC, the attribute decoding method used to decode point clouds is specified in the bitstream.

[0080] Attribute decoding can be performed at each level of detail (LoD), where a finer representation of the point cloud attribute is obtained at each level of detail. Each level of detail can be specified based on a distance metric from neighboring nodes or based on the sampling distance.

[0081] At the G-PCC encoder 200, the residual obtained as the output of the attribute decoding method can be quantized. The residual can be obtained by subtracting the attribute value from the prediction derived from the attribute values ​​of the points in the neighborhood of the current point and based on the attribute values ​​of the previously encoded points. The quantized residual can be decoded using context-adaptive arithmetic decoding.

[0082] The G-PCC encoder 200 and G-PCC decoder 300 can be configured to use predictive geometry decoding as an alternative to octree geometry decoding to decode point cloud data. In predictive tree decoding, the nodes of the point cloud are arranged in a tree structure (which defines the prediction structure), and various prediction strategies are used to predict the coordinates of each node in the tree relative to its predictors.

[0083] Figure 5 is a conceptual diagram illustrating an example of a prediction tree. For example, Figure 5 is a directed graph where arrows point in the prediction direction. One example node is the root vertex and has no predictors; another example node has two children; another example node has three children; another blue example node has one child; and another example node is a leaf node and has no children. Each node has only one parent node. In other words, it is the root vertex and has no predictors. Other nodes may have one, two, three, or more children. Other nodes may be leaf nodes with no children. In one example, each predicted node has only one parent node.

[0084] In one example, four prediction strategies are assigned to each node based on its parent node (p0), grandparent node (p1), and great-grandparent node (p2):

[0085] • No prediction / zero prediction (0)

[0086] • Incremental forecast (p0)

[0087] • Linear prediction (2 p0 – p1)

[0088] • Parallelogram prediction (2) p0 + p1 – p2)

[0089] The G-PCC encoder 200 can employ any algorithm to generate the prediction tree; the algorithm used is determined based on the application / use case, and several strategies can be used. For each node, the residual coordinate values ​​are decoded in the bitstream in a depth-first manner, starting from the root node. Predictive geometry decoding can be particularly useful for Category 3 (LIDAR-acquired) point cloud data (e.g., for low-latency applications).

[0090] Figures 6A and 6B are conceptual diagrams illustrating examples of rotating LiDAR acquisition models. The LiDAR 602 can be used in automotive, mobile computing, aviation, and other applications. In some examples, the angular pattern can be used for predictive geometry decoding, where the characteristics of the LiDAR sensor can be utilized more efficiently when decoding the prediction tree. The coordinates of the position are converted... (Radius, azimuth, and laser index) domain 600 and perform predictions in this domain 600 (the residuals are decoded in...) (Domain). The laser index is also known as the laser identifier component. Therefore, the coordinates of a position include one or more of the following: a radius component (also known as rad), an azimuth component (also known as phi), and an identifier component (also known as laserID or laser index). Due to rounding errors, The decoding in the model is not lossless, therefore the second set of residuals corresponding to Cartesian coordinates is decoded. The following provides a description of the encoding and decoding strategies for the angular patterns used in predictive geometric decoding.

[0091] The angular patterns used for predicting geometric decoding can be used in conjunction with point clouds obtained using a rotated LiDAR model. Here, the LiDAR 602 has patterns based on the azimuth angle. N lasers (e.g., N=16, 32, 64) rotate around the Z-axis. Each laser may have a different elevation. and height In one example, the Cartesian integer coordinates of the model's coordinate system are obtained using a rotated LiDAR as shown in Figures 6A and 6B. ,laser Irradiate to a point .

[0092] The angle pattern used for predicting geometric decoding can include three parameters. The position of M is modeled and calculated as follows:

[0093] •

[0094] •

[0095] • ,

[0096] More precisely, the angular pattern of predictive geometry decoding uses a quantized version. , which is represented as , where three integers and The calculation is as follows:

[0097] •

[0098] •

[0099] •

[0100] in

[0101] • ( and They are control and The quantization parameters for the accuracy.

[0102] • It is a function that returns 1 if t is positive, otherwise returns (-1).

[0103] • yes The absolute value of.

[0104] To avoid reconstruction mismatches due to the use of floating-point operations, and It can be pre-calculated and quantified as follows:

[0105]

[0106]

[0107] in

[0108] • ( and They are control and The quantization parameters for the accuracy.

[0109] The reconstructed Cartesian coordinates are obtained as follows:

[0110] •

[0111] •

[0112] • ,

[0113] in and yes and An approximation can be made using fixed-point representation, lookup tables, and / or linear interpolation.

[0114] Due to the following reasons, The value may be related to different:

[0115] - Quantification

[0116] - Approximate

[0117] - Model inaccuracy

[0118] - Model parameter inaccuracy

[0119] make The reconstruction residual is defined as follows:

[0120] -

[0121] -

[0122] -

[0123] In this method, the G-PCC encoder 200 can be operated as follows:

[0124] • For model parameters and quantization parameters Encode

[0125] • Representation Application of geometric prediction scheme

[0126] ○ New predictors that utilize the characteristics of LiDAR can be introduced. For example, the rotational speed of a LiDAR scanner about the z-axis is typically constant. Therefore, the G-PCC encoder 200 can predict the current... as follows:

[0127]

[0128] in

[0129] ■ This is a set of potential speeds selectable by the encoder. Index It can be explicitly written to the bitstream, or it can be inferred from the context based on a deterministic strategy applied by both the G-PCC encoder 200 and the G-PCC decoder 300.

[0130] ■ It is the number of skip points that can be explicitly written to the bitstream or inferred from the context based on a deterministic strategy applied by both the encoder and decoder. It is also referred to as the "phi multiplier". Note that it is currently used only in conjunction with the incremental predictor.

[0131] • Use each node to reconstruct the residual Encode

[0132] The G-PCC decoder 300 can be operated as follows:

[0133] • For model parameters and quantization parameters Decode

[0134] • Based on the geometric prediction scheme used by the G-PCC encoder 200, the nodes are associated with... The parameters are decoded.

[0135] • Calculate the reconstructed coordinates as described above. .

[0136] • Decode the residual

[0137] As discussed in the following section, residuals can be reconstructed through quantification. To support lossy compression

[0138] • Calculate the original coordinates as follows

[0139] ○

[0140] ○

[0141] ○

[0142] Lossy compression can be achieved by applying quantization to reconstruct the residuals. Alternatively, this can be achieved by discarding points.

[0143] The quantified reconstruction residuals can be calculated as follows:

[0144] •

[0145] •

[0146] •

[0147] in , It is controlled separately. , and The quantization parameters for the accuracy.

[0148] Mesh quantization can be used to further improve RD (rate distortion) performance results. Quantization parameters can be changed at the sequence / frame / slice / block level to achieve region-adaptive quality and for rate control purposes.

[0149] Attribute decoding, octree geometry decoding, and prediction tree geometry decoding techniques can be implemented as intra-frame prediction decoding techniques. That is, the G-PCC encoder 200 and G-PCC decoder 300 can decode attribute and location data using only information from the point cloud data frame being decoded. In other examples, the G-PCC encoder 200 and G-PCC decoder 300 can use inter-frame prediction techniques to decode attributes, octree geometry, and / or prediction tree geometry. That is, the G-PCC encoder 200 and G-PCC decoder 300 can use information from the point cloud data frame being decoded as well as information from previously decoded point cloud data frames to decode attribute and location data.

[0150] As described above, one example of predictive geometry decoding uses a prediction tree structure to predict the location of points. When corner decoding is enabled, the x, y, z coordinates are transformed into radius, azimuth, and laserID, and residuals are emitted in these three coordinates as well as in the x, y, z dimensions. Intra-frame predictions for radius, azimuth, and laserID can be one of four modes, and the predictor is a node classified as parent, grandparent, or great-grandparent in the prediction tree relative to the current node. In one example, predictive geometry decoding can be configured as an intra-frame decoding tool because it only uses points within the same frame for prediction. However, using points from previously decoded frames (e.g., inter-frame prediction) can provide better predictions, thus offering better compression performance in some cases.

[0151] For predictive geometry decoding using inter-frame prediction, one technique involves predicting the radius of a point from a reference frame. For each point in the prediction tree, it is determined whether the point is inter-frame predicted or intra-frame predicted (indicated by a flag). When intra-frame prediction is performed, the intra-frame prediction mode of predictive geometry decoding is used. When inter-frame prediction is used, intra-frame prediction is still used to predict the azimuth and laserID, while the radius is predicted from a point in the reference frame that has the same laserID as the current point and the azimuth closest to the current azimuth. Another example implementation of this method performs inter-frame prediction of azimuth and laserID in addition to radius prediction. When inter-frame decoding is applied, the radius, azimuth, and laserID of the current point are predicted based on points located near the azimuth positions of previously decoded points in the reference frame. Furthermore, separate sets of contexts are used for inter-frame and intra-frame prediction.

[0152] Figure 7 illustrates one method. Figure 7 is a conceptual diagram illustrating an example of inter-frame prediction from a point (interPredPt) 702 in a reference frame to a current point (curPoint) 700 in the current frame. Inter-frame prediction to azimuth, radius, and laserID extensions may include the following steps:

[0153] • For a given point, select the previously decoded point (prevDecP0) 704.

[0154] • Select position point (refFrameP0) 706 in the reference frame, which has the same scaling azimuth and laserID as prevDecP0 704.

[0155] • In the reference frame, find the first point (interPredPt) 702 with an azimuth angle greater than refFrameP0 706. Point interPredPt 702 can also be called the "next" inter-frame predictor.

[0156] Figure 8 is a block diagram illustrating in more detail an example of the geometric coding unit 250 of Figure 2. The geometric coding unit 250 may include a coordinate transformation unit 202, a voxelization unit 206, a prediction tree construction unit 207, an octree analysis unit 210, a surface approximation analysis unit 212, an arithmetic coding unit 214, and a geometric reconstruction unit 216.

[0157] As shown in the example of Figure 8, the geometry encoding unit 250 can obtain a set of locations of points in the point cloud. In one example, the geometry encoding unit 250 can obtain a set of locations and a set of attributes of points in the point cloud from a data source 104 (Figure 1). These locations may include the coordinates of the points in the point cloud. The geometry encoding unit 250 can generate a geometric bitstream 203 that includes the encoded representation of the locations of the points in the point cloud.

[0158] The coordinate transformation unit 202 can apply a transformation to the coordinates of a point to transform the coordinates from the initial domain to the transformation domain. The transformed coordinates may be referred to as transformed coordinates in this disclosure. The voxelization unit 206 can voxelize the transformed coordinates. Voxelization of the transformed coordinates may include quantization and removal of some points in the point cloud. In other words, multiple points in the point cloud can be grouped into a single "voxel," which can subsequently be considered as a single point in some respects.

[0159] The prediction tree building unit 207 can be configured to generate a prediction tree based on voxelized transform coordinates. The prediction tree building unit 207 can be configured to perform either the prediction tree decoding technique described above in intra-frame prediction mode or inter-frame prediction mode. To perform prediction tree decoding using inter-frame prediction, the prediction tree building unit 207 can access points from previously encoded frames from the geometry reconstruction unit 216. The dashed lines from the geometry reconstruction unit 216 indicate the data path when performing inter-frame prediction. The arithmetic coding unit 214 can entropy-encode the syntax elements representing the encoded prediction tree.

[0160] Instead of performing prediction tree-based decoding, the geometric encoding unit 250 can perform octree-based decoding. The octree analysis unit 210 can generate an octree based on voxelized transformed coordinates. The surface approximation analysis unit 212 can analyze these points to potentially determine a surface representation of the set of points. The arithmetic encoding unit 214 can entropy-encode the syntax elements representing information about the octree and / or the surface determined by the surface approximation analysis unit 212. The geometric encoding unit 250 can output these syntax elements in a geometric bitstream 203. The geometric bitstream 203 may also include other syntax elements, including syntax elements that are not arithmetically encoded.

[0161] Octree-based decoding can be performed as an intra-frame prediction technique or an inter-frame prediction technique. To perform octree decoding using inter-frame prediction, octree analysis unit 210 and surface approximation analysis unit 212 can access points from previously encoded frames from geometry reconstruction unit 216. The dashed lines from geometry reconstruction unit 216 illustrate the data path when performing inter-frame prediction.

[0162] The geometric reconstruction unit 216 can reconstruct the transformed coordinates of points in the point cloud based on an octree, a prediction tree, data indicating the surface determined by the surface approximation analysis unit 212, and / or other information. Due to voxelization and surface approximation, the number of transformed coordinates reconstructed by the geometric reconstruction unit 216 may differ from the original number of points in the point cloud. The points obtained in this disclosure may be referred to as reconstructed points.

[0163] Figure 9 is a block diagram illustrating in more detail an example of the attribute encoding unit 260 of Figure 2. The attribute encoding unit 250 may include a color transformation unit 204, an attribute transfer unit 208, a RAHT unit 218, a LoD generation unit 220, a boosting unit 222, a coefficient quantization unit 224, an arithmetic encoding unit 226, and an attribute reconstruction unit 228. The attribute encoding unit 260 may encode the attributes of points in the point cloud to generate an attribute bitstream 205 including the encoded representation of the attribute set. Attributes may include information about points in the point cloud, such as the color associated with a point in the point cloud.

[0164] Color transformation unit 204 can apply transformations to convert the color information of attributes to different domains. For example, color transformation unit 204 can transform color information from the RGB color space to the YCbCr color space. Attribute transfer unit 208 can transfer attributes of original points in the point cloud to reconstructed points in the point cloud. Attribute transfer unit 208 can use the original position of the point as well as the position generated from attribute encoding unit 250 (e.g., from geometric reconstruction unit 216) for transfer.

[0165] RAHT unit 218 can apply RAHT decoding to the attributes of the reconstructed points. In some examples, according to RAHT, the attributes of the block at the 2x2x2 point location are obtained and transformed along one direction to obtain four low-frequency nodes (L) and four high-frequency nodes (H). Subsequently, the four low-frequency nodes (L) are transformed in a second direction to obtain two low-frequency nodes (LL) and two high-frequency nodes (LH). The two low-frequency nodes (LL) are transformed along a third direction to obtain one low-frequency node (LLL) and one high-frequency node (LLH). The low-frequency node LLL corresponds to the DC coefficients, and the high-frequency nodes H, LH, and LLH correspond to the AC coefficients. The transformation in each direction can be a 1-D transformation with two coefficient weights. The low-frequency coefficients can be obtained as coefficients for the next higher-level 2x2x2 block for the RAHT transformation, and the AC coefficients are encoded without modification; such transformations continue until the top root node. The weights to be used for these coefficients are calculated from top to bottom using a tree traversal for encoding; the transformation order is bottom to top. These coefficients can then be quantized and decoded.

[0166] Alternatively or additionally, LoD generation unit 220 and lifting unit 222 may apply LoD processing and lifting to the attributes of the reconstructed points, respectively. LoD generation is used to break down the attributes into different refinement levels. Each refinement level provides a refinement of the attributes of the point cloud. The first refinement level provides a coarse approximation and contains few points; subsequent refinement levels typically contain more points, and so on. Refinement levels can be constructed using distance-based metrics, or one or more other classification criteria (e.g., subsampling from a specific order). Thus, all reconstructed points can be included in the refinement levels. Each level of detail is generated by taking the union of all points up to a specific refinement level: for example, LoD1 is obtained based on refinement level RL1, LoD2 is obtained based on RL1 and RL2, ..., LoDN is obtained by the union of RL1, RL2, ..., RLN. In some cases, LoD generation may be followed by a prediction scheme (e.g., a prediction transform) in which the attributes associated with each point in the LoD are predicted based on a weighted average of the previous points, and the residuals are quantized and entropy-decoded. The enhancement scheme is built on the predictive transformation mechanism, in which update operators are used to update the coefficients and adaptive quantization of the coefficients is performed.

[0167] RAHT unit 218 and lifting unit 222 can generate coefficients based on these attributes. Coefficient quantization unit 224 can quantize the coefficients generated by RAHT unit 218 or lifting unit 222. Arithmetic coding unit 226 can apply arithmetic decoding to the syntax elements representing the quantized coefficients. G-PCC encoder 200 can output these syntax elements in attribute bitstream 205. Attribute bitstream 205 may also include other syntax elements, including syntax elements that are not arithmetically encoded.

[0168] Similar to the geometry coding unit 250, the attribute coding unit 260 can use intra-frame prediction or inter-frame prediction techniques to encode attributes. The above description of the attribute coding unit 260 generally describes intra-frame prediction techniques. In other examples, the RAHT unit 215, the LoD generation unit 220, and / or the boosting unit 222 can also use attributes from previously encoded frames to further encode the attributes of the current frame. In this regard, the attribute reconstruction unit 228 can be configured to reconstruct the encoded attributes and store these encoded attributes for potential future use in inter-frame prediction coding. The dashed lines from the attribute reconstruction unit 228 illustrate the data path when inter-frame prediction is performed.

[0169] Figure 10 is a block diagram illustrating the example geometry decoding unit 350 of Figure 3 in more detail. The geometry decoding unit 350 can be configured to perform a process that is the inverse of the process performed by the geometry encoding unit 250 of Figure 8. The geometry decoding unit 350 receives a geometry bitstream 203 and generates the positions of points in a point cloud frame. The geometry decoding unit 350 may include a geometry arithmetic decoding unit 302, an octree synthesis unit 306, a prediction tree synthesis unit 307, a surface approximation synthesis unit 310, a geometry reconstruction unit 312, and an inverse coordinate transformation unit 320.

[0170] The geometric decoding unit 350 can receive the geometric bitstream 203. The geometric arithmetic decoding unit 302 can apply arithmetic decoding (e.g., context-adaptive binary arithmetic decoding (CABAC) or other types of arithmetic decoding) to the syntax elements in the geometric bitstream 203.

[0171] The octree synthesis unit 306 can synthesize an octree based on the syntax elements parsed from the geometric bitstream 203. Starting from the root node of the octree, it signals in the bitstream the occupancy of each of the eight child nodes at each octree level. When signaling indicates that a child node at a particular octree level is occupied, it signals the occupancy of the child nodes of that child node. Before proceeding to subsequent octree levels, it signals the nodes at each octree level.

[0172] At the final level of the octree, each node corresponds to a voxel location; when a leaf node is occupied, one or more points can be designated as occupied at a voxel location. In some instances, due to quantization, some branches of the octree may terminate earlier than the final level. In such cases, leaf nodes are considered occupied nodes without children. In the instance of using surface approximation in the geometric bitstream 203, the surface approximation synthesis unit 310 can determine the surface model based on the syntax elements parsed from the geometric bitstream 203 and based on the octree.

[0173] Octree-based decoding can be performed as either intra-frame prediction or inter-frame prediction. To perform octree decoding using inter-frame prediction, octree synthesis unit 306 and surface approximation synthesis unit 310 can access points from previously decoded frames from geometry reconstruction unit 312. The dashed lines from geometry reconstruction unit 312 illustrate the data path when inter-frame prediction is performed.

[0174] The prediction tree synthesis unit can synthesize a prediction tree based on the syntax elements parsed from the geometric bitstream 203. The prediction tree synthesis unit 307 can be configured to synthesize the prediction tree using any of the techniques described above, including both intra-frame prediction and inter-frame prediction techniques. To perform prediction tree decoding using inter-frame prediction, the prediction tree synthesis unit 307 can access points from previously decoded frames from the geometric reconstruction unit 312. The dashed lines from the geometric reconstruction unit 312 indicate the data path when inter-frame prediction is performed.

[0175] Furthermore, the geometric reconstruction unit 312 can perform reconstruction to determine the coordinates of points in the point cloud. For each location at a leaf node of the octree, the geometric reconstruction unit 312 can reconstruct the node location using a binary representation of the leaf node in the octree. At each corresponding leaf node, a signal is used to indicate the number of points at that corresponding leaf node; this indicates the number of replicated points at the same voxel location. When geometric quantization is used, the point locations are scaled to determine the reconstructed point location values.

[0176] The inverse transform coordinate unit 320 can apply an inverse transform to the reconstructed coordinates to convert the reconstructed coordinates (positions) of points in the point cloud from the transform domain back to the initial domain. The positions of points in the point cloud can be in the floating-point domain, but the point positions in the G-PCC codec are decoded in the integer domain. The inverse transform can be used to convert these positions back to the original domain.

[0177] Figure 11 is a block diagram illustrating the example attribute decoding unit 360 of Figure 3 in more detail. The attribute decoding unit 360 can be configured to perform a process that is the inverse of the process performed by the attribute encoding unit 260 of Figure 9. The attribute decoding unit 360 receives the attribute bitstream 205 and generates the attributes of the points in the point cloud frame. The attribute decoding unit 356 may include an attribute arithmetic decoding unit 304, an inverse quantization unit 308, an inverse RAHT unit 314, a LoD generation unit 316, an inverse boosting unit 318, an inverse color transformation unit 322, and an attribute reconstruction unit 328.

[0178] The attribute arithmetic decoding unit 304 can apply arithmetic decoding to the syntax elements in the attribute bitstream 205. The inverse quantization unit 308 can inverse quantize the attribute values. The attribute values ​​can be based on the syntax elements obtained from the attribute bitstream 205 (e.g., including syntax elements decoded by the attribute arithmetic decoding unit 304).

[0179] Depending on the encoding method of the attribute values, the inverse RAHT unit 314 can perform RAHT decoding to determine the color values ​​of points in the point cloud based on the inverse-quantized attribute values. RAHT decoding is performed from the top to the bottom of the tree. At each level, composition values ​​are derived using low-frequency and high-frequency coefficients derived from the inverse quantization process. At leaf nodes, the derived values ​​correspond to the attribute values ​​of the coefficients. The point weight derivation process is similar to that used at the G-PCC encoder 200. Alternatively, the LoD generation unit 316 and the inverse lifting unit 318 can use level-of-detail techniques to determine the color values ​​of points in the point cloud. The LoD generation unit 316 decodes each LoD, thus giving a progressively finer representation of the point's attributes. In the case of prediction transform, the LoD generation unit 316 derives the prediction of the point from a weighted sum of points previously reconstructed in the same LoD or earlier. The LoD generation unit 316 can add the prediction to the residual (obtained after inverse quantization) to obtain the reconstructed values ​​of the attributes. When using an enhancement scheme, the LoD generation unit 316 may also include an update operator to update the coefficients used to derive the attribute values. In this case, the LoD generation unit 316 may also apply inverse adaptive quantization.

[0180] Furthermore, in the example of Figure 11, the inverse color transformation unit 322 can apply an inverse color transformation to the color values. The inverse color transformation can be the reverse of the color transformation applied by the color transformation unit 204 of the encoder 200. For example, the color transformation unit 204 can transform color information from the RGB color space to the YCbCr color space. Correspondingly, the inverse color transformation unit 322 can transform color information from the YCbCr color space to the RGB color space.

[0181] The attribute reconstruction unit 328 can be configured to store attributes from previously decoded frames. Attribute decoding can be performed as an intra-frame prediction technique or an inter-frame prediction technique. To perform attribute decoding using inter-frame prediction, the inverse RAHT unit 314 and / or the LoD generation unit 316 can access the attributes of the previously decoded frames from the attribute reconstruction unit 328. The dashed lines from the attribute reconstruction unit 328 indicate the data path when inter-frame prediction is performed.

[0182] Various units illustrated in Figures 8 through 11 are provided to aid in understanding the operations performed by the G-PCC encoder 200 and the G-PCC decoder 300. Units can be implemented as fixed-function circuits, programmable circuits, or combinations thereof. A fixed-function circuit is a circuit that provides specific functionality and is pre-defined for the operations that can be performed. A programmable circuit is a circuit that can be programmed to perform various tasks and provides flexible functionality for the operations that can be performed. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. A fixed-function circuit can execute software instructions (e.g., to receive or output parameters), but the type of operation performed by a fixed-function circuit is generally immutable. In some examples, one or more units in the unit may be different circuit blocks (fixed-function or programmable), and in some examples, one or more units in the unit may be integrated circuits.

[0183] Figure 12 is a flowchart illustrating an example of decoding point cloud data. For example, Figure 12 illustrates the decoding process associated with an “inter_flag” that is notified by sending a signal for each point. The example in Figure 12 may be similar to the examples available in InterEM-v3.0.

[0184] For example, in Figure 12, the G-PCC decoder 300 can determine, based on a flag, whether to use inter-frame prediction or intra-frame prediction to decode the current point (1200). If it is inter-frame prediction ("Yes" in 1200), the G-PCC decoder 300 can select a previous point in the decoding order of the current frame, including the radius component (r), azimuth component (phi), and laser identifier component (laserID) (1202), and derive the quantized azimuth (e.g., phi) (1204). The G-PCC decoder 300 can examine a point in the reference frame with a larger quantized azimuth (e.g., the point is interPredPt) (1206), and use one or more of interPredPt as the inter-frame predictor (1208). The G-PCC decoder 300 can add the incremental azimuth multiplier to the main residual (1212).

[0185] If it is not inter-frame prediction (No in 1200), the G-PCC decoder 300 may select an intra-frame prediction candidate (e.g., pred_mode) (1210). The G-PCC decoder 300 may add an incremental azimuth multiplier to the primary residual (1212). The G-PCC decoder 300 may add a secondary residual after the transformation back to Cartesian coordinates.

[0186] The following describes an example of additional prediction sub-candidates. In the inter-frame prediction method for predictive geometry decoding described above, when applying inter-frame decoding, the following steps are used to predict the radius, azimuth, and laserID of the current point based on points near the common azimuth position in the reference frame: (a) for a given point, select a previously decoded point; (b) select a position in the reference frame with the same scaled azimuth and laserID as in (a); and (c) starting from the first point, select positions in the reference frame with an azimuth greater than that of the position in (b) as inter-frame prediction sub-points.

[0187] The example technique adds an additional inter-predictor point, which is obtained by finding a first point with an azimuth angle greater than that of the inter-predictor point in (c), as shown in Figure 13. If inter-frame decoding has been applied, the additional signaling is used to indicate which predictor to select. The additional inter-predictor point is also referred to as the "NextNext" inter-predictor.

[0188] For example, in Figure 13, the current frame 1302 includes the current point 1306 along with the previously decoded point 1308. Point 1310 in the reference frame 1300 is a reference point with the same scaled azimuth and laserID as the current frame 1302. Additional inter-frame prediction point 1312 of the reference frame 1300 can also be used for inter-frame prediction of the current point 1306.

[0189] The improved inter-frame prediction flag decoding is described below. An improved context selection algorithm is applied to decode the inter-frame prediction flag. The inter-frame prediction flag values ​​of five previously decoded points are used to select the context for the inter-frame prediction flag during predictive geometry decoding.

[0190] Global motion compensation is described below. When global motion (GM) parameters are available, a reference frame can be used to apply inter-frame prediction, which uses the GM parameters for motion compensation. GM parameters may include rotation and / or translation parameters. Typically, global motion compensation is applied in the Cartesian domain. In some cases, global motion compensation may also be performed in the spherical domain. Depending on which domain the reference frame is stored in and in which domain the reference frame is compensated, one or more of a Cartesian-to-spherical or spherical-to-Cartesian transformation may be applied. For example, when the reference frame is stored in the spherical domain and motion compensation is performed in the Cartesian domain, the motion compensation process may involve one or more of the steps illustrated in Figure 14.

[0191] For example, in Figure 14, a reference frame (1400) in the spherical domain is input for a spherical-to-Cartesian domain conversion (1402). The output can be a reference frame in the Cartesian domain (1404). The reference frame in the Cartesian domain can be motion-compensated (1406), and the output can be a compensated reference frame in the Cartesian domain (1408). The compensated reference frame in the Cartesian domain (1408) can be input for a Cartesian-to-spherical domain conversion (1410). The output can be a compensated reference frame in the spherical domain (1412).

[0192] In such cases, a compensated reference frame can be used for inter-frame prediction. Given its position in a Cartesian coordinate system... The corresponding radius and azimuth are calculated (floating-point implementation) as follows (similar to the CartesianToSpherical conversion function):

[0193] int64_t r0 = int64_t(std::round(hypot(xyz[0], xyz[1])));

[0194] auto phi0 = std::round((atan2(xyz[1], xyz[0]) / (2.0 M_PI)) scalePhi);

[0195] Specifically, `scalePhi` modifies different rate points in a lossy configuration; when performing lossless decoding of the geometry, the maximum value of the azimuth angle is 24 bits. The specific implementation of the azimuth angle positioning can be used in the `convertXyZToRpl` function.

[0196] radius:

[0197]

[0198]

[0199] The following describes the resampling of the reference frame. When global motion compensation is applied, the azimuth position of a point is modified according to the motion parameters. Therefore, resampling may be necessary to align the azimuth point before and after compensation, as illustrated in Figure 15.

[0200] The unfilled ellipse represents point 1500 in the uncompensated reference frame (e.g., a reference frame without any global motion compensation applied or before any global motion compensation was applied). The diagonally filled ellipse represents point 1502 in the globally motion compensated version of the reference frame. The horizontally filled ellipse represents the resampled point 1504 in the globally motion compensated version of the reference frame. Therefore, point 1500 has no global motion compensation applied, point 1502 has global motion compensation applied, and point 1504 has both global motion compensation and resampling applied. It can be seen that the application of global motion compensation may cause the azimuth position of one or more points in point 1502 to become misaligned with the corresponding point in point 1500. By resampling, the G-PCC encoder 200 or the G-PCC decoder 300 can realign point 1502 (e.g., shown as resampled point 1504) with its corresponding point 1500.

[0201] The resampling process can be applied to each point P in the uncompensated reference frame, as follows:

[0202] a. Let A_ref be the azimuth value and L be the laser ID value associated with point P.

[0203] b. If there exists a point P1 in the (global motion) compensation reference frame with an azimuth value equal to A_ref and a laser ID equal to L, then the radius of point P is set to be equal to the radius of point P1.

[0204] c. Additionally, in the (global motion) compensation reference frame with laser IDL, two points P2 and P3 are selected such that the azimuth angle of P2 is less than A_ref, and the azimuth angle of P3 is greater than A_ref. The radius of point P is set as a weighted interpolation of the radii of points P2 and P3; the weight used for interpolation depends on the difference between A_ref and the azimuth angle values ​​of P2 and P3.

[0205] The resulting reference frame (referred to as the resampled reference frame) (obtained by resampling the uncompensated reference frame using the radius value from the compensated reference frame) is used to predict inter-frame prediction candidates. Therefore, two inter-frame prediction sub-candidates can be indicated as [Res-Next, Res-NextNext], where the first part "Res" indicates that the candidate was obtained from the resampled reference frame, and the second part "Next" / "NextNext" indicates a specific candidate within the reference frame, as described above.

[0206] The following describes additional candidates for inter-frame prediction. In the modified inter-frame prediction sublist, four inter-frame prediction candidates are specified as follows: [Zero-Next, Zero-NextNext, Glob-Next, Glob-NextNext].

[0207] The prefix "Zero" for the first two candidates indicates that the candidates were obtained directly from the uncompensated reference frame (without motion compensation or resampling), while the prefix "Glob" for the last two candidates indicates that the candidates were obtained directly from the global motion-compensated reference frame.

[0208] The following describes the flags used to notify resampling by sending a signal, with `gm` indicating 2 / 4 candidates. The flag is enabled to indicate whether resampling is enabled. Furthermore, only two inter-frame prediction candidates are allowed when global motion of the sequence is disabled. Therefore, the inter-frame prediction candidates for predictive geometry decoding are selected as follows:

[0209] a. Disable global motion:

[0210] i. [Zero-Next, Zero-NextNext]

[0211] b. Enable global motion

[0212] i. Enable resampling

[0213] 1. [Res-Next, Res-NextNext, Glob-Next, Glob-NextNext]

[0214] ii. Disable resampling

[0215] 1. [Zero-Next, Zero-NextNext, Glob-Next, Glob-NextNext]

[0216] When both global motion and resampling are enabled, the prefix "Res" of the first two candidates indicates that the candidate was obtained from a resampled reference frame.

[0217] The following describes spherical coordinate transformation. Spherical coordinate transformation is a technique used in G-PCC where geometry represented in spherical coordinates is used during attribute decoding. Attribute decoding generally involves generating a level of detail (for predicting / lifting transforms) or generating a RAHT tree (for RAHT transforms), both of which utilize geometry. When spherical coordinate transformation is not used, attribute decoding is performed using geometry represented in Cartesian coordinates; the Morton scan order is chosen for resolving points. For sparse data (such as data obtained using LiDAR sensors), using Cartesian coordinates results in suboptimal relationships of points in the Morton order. Because spherical coordinates utilize the sensor scan characteristics, geometry converted to spherical coordinates provides a more efficient point representation. The Morton scan order in this domain provides more meaningful point relationships, and this improves the efficiency of attribute decoding. Generally, spherical coordinate transformation is only used when angular mode (for decoding geometry) is enabled.

[0218] The spherical coordinate representation (posSph0) used for attribute decoding is obtained by applying offset and scaling to the actual spherical coordinate representation (posSph0) of the geometry. In one or more examples, posSph0 can be considered a reference point cloud frame (e.g., a coordinate representation of the geometry of the reference point cloud frame). In some examples, the G-PCC encoder 200 and the G-PCC decoder 300 can apply processes (e.g., apply offset and scaling) to the reference point cloud frame (e.g., a coordinate representation of the geometry of the reference point cloud frame, such as posSph0) to generate a processed frame (e.g., posSph0). This processed frame is used to encode or decode the attribute data of the point. In one or more examples, posSph0 It can be used to encode or decode the attribute data of points in a reference frame (e.g., intra-frame prediction or inter-frame prediction), while another processed frame can be used to perform inter-frame prediction encoding or decoding of the attribute data of points in the current point cloud frame, as described below with respect to Figure 24.

[0219] Because the applied offset / scaling is a linear transformation, Figure 16 illustrates how the radius (rad), azimuth (phi), and laser ID (laserId) that together form the spherical representation posSph0 are transformed into the spherical representation posSph0 used for attribute prediction. rad phi and laserId The offset and scaling values ​​for each dimension are communicated by sending signals in the Attribute Parameter Set (APS).

[0220] For example, in Figure 16, the reference point cloud frame (posSph0) 1600 is in spherical coordinates and includes a radius (rad) component 1602, an azimuth (phi) component 1604, and a laser ID (laserId) component 1606. As described, the reference point cloud frame 1600 can be used as a coordinate representation of the geometry of the reference point cloud frame 1600. Each of rad 1602, phi 1604, and laserID 1606 can undergo offset and scaling processes 1608, 1610, and 1612, respectively. The result can be a coordinate representation with a radius (rad) ) Component 1614, azimuth (phi) ) Component 1616 and laser marking (laserId) Processed frame (posSph0) of component 1618 1620. In one or more examples, posSph0 Intra-frame prediction of attribute data for points in a reference frame is possible, but inter-frame prediction of attribute data for points in the current frame is possible using another processed frame.

[0221] The following describes an example of an inter-frame prediction buffer. Some example techniques use two reference frame buffers for the same reference frame. One buffer is used for inter-frame prediction of geometry, and the other buffer is used for inter-frame prediction of attributes, as illustrated in Figure 17. The contents of the two buffers are different.

[0222] For example, consider reference frame 0. The reconstructed spherical coordinates posSph0 1700 of frame 0 are used to generate posSph0 using spherical coordinate transformation. 1702, as described above with reference to Figure 16. In some techniques, this represents posSph0. It is used for both intra-frame and inter-frame attribute prediction. That is, posSph0 1702 is used both for intra-frame predictive coding and decoding of attribute data for points in a reference point cloud frame (e.g., reference frame 0) and for inter-frame predictive coding and decoding of attribute data for points in the current point cloud frame. As described in more detail, this may be inefficient from a memory storage perspective because posSph0 1702 is retained frame by frame (e.g., it is kept in storage after the encoding or decoding of reference frame 0 is completed and the encoding or decoding of the current frame begins).

[0223] In parallel, posSph0 is also used to generate a spherical table SphTable0 1704, which is used for inter-frame prediction of geometry via the method illustrated in Figure 18. SphTable0 1704 can be considered as a first-level processed frame. For example, G-PCC encoder 200 and G-PCC decoder 300 can apply the first process to a reference point cloud frame (e.g., posSph0 1700) to generate a first-level processed frame (e.g., SphTable0 1704). An example of the first process is illustrated in Figure 18.

[0224] For example, in Figure 18, the G-PCC encoder 200 and G-PCC decoder 300 can apply a first process to a reference point cloud frame (posSph0) 1800 to generate a first-level processed frame (SphTable0) 1812. For example, the reference point cloud frame 1800 may include a radius (rad) component 1802, an azimuth (phi) component 1804, and a laser identifier (laserId) component 1806. The G-PCC encoder 200 and G-PCC decoder 300 can quantize (1808) the azimuth component 1804 to generate a quantized azimuth component 1810.

[0225] For each of the multiple quantized azimuth components 1810 and for the laser identification component 1806, the G-PCC encoder 200 and the G-PCC decoder 300 can store the radius component 1802 and azimuth component 1804 of "k" points of the reference point cloud frame 1800 associated with the laser identification component 1806 to generate SphTable0 1812. As illustrated, the quantized azimuth qPhi 1810 and laserID 1806 are used as lookup values ​​in a spherical table (SphTable0 1812) storing points in spherical coordinates. That is, each of the multiple quantized azimuth components 1810, together with the laser identification component 1806, is an index to the table (SphTable0 1812), and the table (SphTable0 1812) is at least a portion of the first-level processed frames.

[0226] Techniques for decoding the current frame of point cloud data may have some issues. Some specific implementations of inter-frame prediction for geometry use two reference frame buffers for the same reference frame in inter-frame prediction: one buffer for geometry and one buffer for attributes. These buffers are not identical and therefore cannot be reused. For example, in Figure 17, SphTable01704 and posSph0... 1702 is stored in two separate buffers and maintained frame-by-frame (e.g., from the reference frame to the current frame). This increases the cost of storing inter-frame predicted frames. Using less buffer space would be more advantageous.

[0227] One or more of the example techniques described in this disclosure may be applied independently or in combination.

[0228] The reference frame for geometric inter-frame prediction is indicated to the G-PCC decoder 300 by applying a first process (or process 1) to the reference frame. To distinguish between the reference frames before and after processing, the following terminology is used:

[0229] a. Unprocessed Reference Frame (UPRF) – This is a reference that has not yet been processed; the reference frame can be in Cartesian coordinates or spherical coordinates (for geometry). In some cases, the UPRF may appear in both Cartesian and spherical coordinates (because both representations are possible). An unprocessed Cartesian reference frame can be output from the decoder. In some cases (e.g., spherical coordinate transformation), the UPRF in spherical coordinates can be transformed into another reference frame (e.g., using scaling and offset), which is used to derive the properties of the reference frame. In this case, the transformed reference frame will also be referred to as the UPRF (or the transformed UPRF) because, from the perspective of inter-frame prediction, the transformed frame (after the reference frame has been decoded) has not yet been processed.

[0230] b. Processed Reference Frame (PRF) – One or more techniques in this disclosure can be used to derive a PRF from a UPRF. The PRF can be used for inter-frame prediction, or can be further processed for inter-frame prediction of the current and subsequent frames in the order of decoding. A second processed reference frame, PRF-A, can be generated from the PRF; PRF-A can be used to decode attributes. The PRF can be referred to as a first-level processed frame, and PRF-A can be referred to as a second-level processed frame. Examples of PRFs (e.g., first-level processed frames) include SphTable0, and examples of PRF-A (e.g., second-level processed frames) include posSph0 generated from SphTable0. x.

[0231] The following diagram illustrates the various terms defined above, such as those in Figure 19. Parallelogram blocks indicate data / buffers, and rectangles represent processes. When a parallelogram is immediately adjacent to the left (above or below) of a rectangle, the parallelogram is the input to the rectangle. When a parallelogram is immediately adjacent to the right (above or below) of a rectangle, the parallelogram is the output of the rectangle.

[0232] For example, for frame 0, the geometric data of the points (stored as UPRF 1902) can be processed via Frame0-Geom 1900 using scaling and offset, and stored as transformed UPRF 1906. The attribute data of frame 0 can be processed via Frame0-Attr1904 to generate attr0 1908 associated with the transformed UPRF 1906.

[0233] For frame 1, the transformed UPRF 1906 can be processed via Frame1-Geom 1910 to generate a PRF 1912 associated with attr0 1908. For attributes, frame 1 can be processed via Frame1-Attr 1914 to generate a PRF-A 1916 associated with attr0.

[0234] The first process may include generating a PRF from a UPRF. This generation may involve one or more steps. Some example steps or processes for this generation are listed below. These steps may be applied individually or combined in some way.

[0235] Quantization can be applied to points in UPRF 1906 to obtain PRF 1912. Quantization can be applied to geometry, attributes, or both. Further details on quantization are provided below. In some cases, when coordinates are quantized, a subset of points with the same quantized azimuth value may be discarded / not included.

[0236] You can apply offsets and scaling to geometry, attributes, or both to derive processed frames from unprocessed frames. You can apply filtering operations to geometry, attributes, or both to derive processed frames from unprocessed frames. You can copy processed frames from unprocessed frames.

[0237] Downsampling / dropping points—One or more points in the UPRF can be skipped / not included to obtain the PRF. In some cases, this process is also referred to as quantization.

[0238] In some examples, PRF 1912 can be generated and represented as a table indexed by laser ID and quantized azimuth values, where each occupied entry in the table contains a point (coordinates and attributes). That is, an example of PRF 1912 is SphTable0 2012 in Figure 20, and it is generated using the example technique described above with respect to Figure 18. As described, SphTable0 1812 is indexed using laserId 1806 and quantized phi 1810.

[0239] The third process can be defined to specify how entries are populated in the PRF. The third process is described below, followed by a further description of the second process.

[0240] In some examples, the PRF can be generated and represented as a table indexed by laser IDs and quantized azimuth values, where each occupied entry in the table contains one or more points (coordinates and attributes). Therefore, each entry can contain a list or array of points. For example, as described above with respect to Figure 18, in some examples, SphTable0 1812 of Figure 18 may include k points, where k is greater than or equal to 1. Similarly, as described above, SphTable0 2012 of Figure 20 may include k points, where k is greater than or equal to 1.

[0241] One or more syntax elements can be signaled, allowing specification of whether entries in a table can have more than one entry, and if so, specifying the maximum number of points per table entry. For example, when the signaled syntax element `maxPointsPerEntryMinus1` has a value of 0, each entry can have at most one point; when the signaled syntax element `maxPointsPerEntryMinus1` has a value of n (n > 0), each entry can have at most (n+1) points. A `SphTable0` 1812 or 2012 including "k" points can be defined by the `maxPointsPerEntryMinus1` syntax element, but in some examples, `maxPointsPerEntryMinus1` can be a default value, rather than being signaled and received.

[0242] As an example, when the maximum number of points per entry is one (e.g., k = 1), the third process can specify the following:

[0243] a. If only one point has a specific index (i.e., the same laser ID and quantized azimuth value), then that point is added as a table entry for that index.

[0244] b. If there are more points (the first group) with the same index (i.e., the same laser ID and quantized azimuth value), the first point in the first group that has been processed / decoded is included as a table entry for that index.

[0245] c. In some cases, if there are more points (the first group) with the same index (i.e., the same laser ID and quantized azimuth value), the point in the first group with the smallest radius value is included as a table entry for that index.

[0246] As another example, when the maximum number of points per entry is greater than 1 (e.g., k > 1), the third process can specify the following (assuming each table entry can have at most n+1 points):

[0247] a. If only one point has a specific index (i.e., the same laser ID and quantized azimuth value), then that point is added as a table entry for that index.

[0248] b. If there are more points (the first group) with the same index (i.e., the same laser ID and quantized azimuth value), then the first (n+1) points in the first group that have been processed / decoded are included as table entries for that index (in the same order in which the points were processed).

[0249] c. In some cases, if more than one point (the first group) has the same index (i.e., the same laser ID and quantized azimuth value), then at most (n+1) points with the smallest radius are included as table entries for that index (in non-decreasing radius order).

[0250] d. In some cases, if more than one point (the first group) have the same index (i.e., the same laser ID and quantized azimuth value), the first (n+1) points processed / decoded in the first group are included as table entries for that index (in the same order in which the point was processed). If the point P with the smallest radius in the first group is included in the first (n+1) points, then point P is placed as the first point in the table entry, and the remaining points are shifted to the right as needed. Otherwise (the point P with the smallest radius in the first group may not be included in the first (n+1) points), then (a) the last point in the table entry is removed, (b) the remaining points are shifted one position to the right, and (c) point P is added as the first point in the entry.

[0251] Entries with more than one point can also use an intermediate buffer (e.g., to generate a resampled reference image).

[0252] Instructions for applying the first process to obtain the PRF may involve signaling a syntax element in the bitstream. This syntax element may be a flag indicating the application of the first process, and / or a mode value indicating / specifying which type of first process (e.g., quantization, offset + scaling, filtering, downsampling, etc.) to apply. In some examples, the first process may be applied when inter-frame prediction is enabled (thus relying on a syntax element indicating inter-frame prediction, rather than an explicit syntax element). In some examples, the default mode may be selected by the G-PCC encoder 200 and the G-PCC decoder 300.

[0253] That is, the G-PCC encoder 200 and the G-PCC decoder 300 can apply a first process to the reference point cloud frame to generate a first-level processed frame. The first process can be an UPRF to PRF conversion. For example, if the reference point cloud frame (e.g., UPRF) is posSph0 (e.g., a coordinate representation of the geometry of the reference point cloud frame), then the PRF can be SphTable0 generated using the example technique of Figure 18.

[0254] In the second process, the geometry-generated PRF (e.g., a first-level processed frame or SphTable0) may not be directly used for inter-frame prediction of attributes. The second process can be applied to the PRF to generate different processed reference frame PRF-A (e.g., a second-level processed frame, also referred to as posSph0) for inter-frame prediction of attributes. or posSph0 x (with the suffix "A" to indicate the attribute).

[0255] That is, the G-PCC encoder 200 and the G-PCC decoder 300 can apply a second process to the first-level processed frame (e.g., PRF or SphTable0) to generate a second-level processed frame (e.g., posSph0). Generate posSph0 The second process can be similar to the example illustrated in Figure 16, or described in more detail below with reference to Figure 23. For example, the G-PCC encoder 200 and the G-PCC decoder 300 can apply offsets and scaling to one or more of the radius component, azimuth component, and laser marking component for each point in the first-level processed frame to generate the second-level processed frame.

[0256] The G-PCC encoder 200 and G-PCC decoder 300 can use the first-level processed frame to perform inter-frame predictive coding or decoding on the geometric data of the points in the current point cloud frame. The G-PCC encoder 200 and G-PCC decoder 300 can use the second-level processed frame to perform inter-frame predictive coding or decoding on the attribute data of the points in the current point cloud frame.

[0257] In one or more examples, the second-level processed frame may also include coordinate information. In such examples, this coordinate information is used to encode or decode attribute data. For example, actual attribute data (e.g., color, reflectivity, etc.) may be associated with the coordinate information of points in the second-level processed frame. Therefore, both the first-level processed frame and the second-level processed frame may include coordinate information, but the coordinate information in the first-level processed frame can be used for inter-frame predictive coding or decoding of geometric data, and the coordinate information in the second-level processed frame can be used for inter-frame predictive coding or decoding of attribute data.

[0258] Using a PRF (e.g., a first-level processed frame) to generate a PRF-A (e.g., a second-level processed frame) has the following advantages: the PRF size may not be as large as the UPRF, so storing only the PRF may be advantageous in terms of storage. This storage advantage is maintained when the PRF is used to generate the PRF-A; however, the efficiency of inter-frame prediction may decrease. If a UPRF (e.g., posSph0) is used to generate the PRF-A, both the UPRF and the PRF can be stored during inter-frame prediction of subsequent frames. For example, as illustrated in Figure 17, SphTable0 1704 and posSph0 Both are maintained for inter-frame prediction in subsequent frames. Utilizing the example technique, because posSph0 (For example, the second-level processed frames for inter-frame prediction of attribute data) are generated from SphTable0 (for example, the first-level processed frames for inter-frame prediction of geometry data), so the amount of data stored can be reduced frame by frame.

[0259] In some examples, a second process from the UPRF can be used to generate the PRF-A. In some examples, the second process may correspond to the first process. For example, the second process may be specified to reverse the effect of the first process. The first process may not be reversible; in such cases, the second process may apply steps that reverse the effect of the first process to some extent. However, in general, the second process may not correspond to the first process and may be specified independently.

[0260] One or more parameters associated with the first process can be notified by sending signals in the bitstream. The number of parameters notified by sending signals can be determined based on the type of the first process being applied.

[0261] When the first process involves scaling and offset, scaling and offset can be notified by sending signals. When the first process involves quantization, the quantization scaling value can be notified by sending signals; in some cases, the value can be notified by sending signals and the quantization scaling value can be derived from that value.

[0262] Parameters can be notified by signaling in a parameter set (SPS, APS, GPS, etc.), a slice header, or another syntax element in the bitstream. Parameters can be notified / applied by signaling on a frame-by-frame or sequence-by-sequence basis. That is, when applied on a frame-by-frame basis, parameters can be notified by signaling for each frame, and when applied on a sequence-by-sequence basis, parameters can be notified by signaling once for each sequence.

[0263] In some examples, one or more parameters may be notified for the second process by sending signals. When the second process is associated with the first process, the parameters of the second process can be derived from the parameters of the first process. The parameters of the first and second processes may differ for each coordinate. In this case, separate coordinates may be notified by sending signals. In some examples, one or more parameters may be inferred from other parameters / syntax elements notified by sending signals. One or more techniques mentioned in this application can also be applied to multiple reference frames and bidirectional prediction.

[0264] The signal indicates whether quantization is applied to the reference frame; this quantization may refer to the quantization of point coordinates, downsampling of points, or both. The signal also indicates that points not included in the generation of the PRF are stored as a list that can be used for later processing (e.g., the generation of PRF-A).

[0265] Although the reference frames (e.g., processed frames, first-level processed frames, and / or second-level processed frames) mentioned above are used for inter-frame prediction, the reference frames can also be used for other decoding modes, or more generally, for decoding one or more other frames.

[0266] The following describes example techniques for implementing several methods described in this disclosure. In some examples, predictive geometry decoding is used to decode the geometry, and inter-frame prediction is enabled for a subset of frames. Angle mode is enabled (i.e., the laser parameters of the sensor are available), and spherical coordinate transformation is enabled.

[0267] Figure 20 illustrates the overall generation of UPRF and PRF during frame decoding. Consider two frames: frame 0 and frame 1. Frame0-Geom 2000 indicates the decoding of the geometry of frame 0, Frame0-Attr 2004 indicates the decoding of the attributes of frame 0, Frame1-Geom 2010 indicates the decoding of the geometry of frame 1, and Frame1-Attr indicates the decoding of the attributes of frame 1.

[0268] The parallelograms depict the data / buffers used as inputs to or outputs to the block. Figure 20 only depicts the data parallelogram relevant to this example. There may be other inputs / outputs for each block.

[0269] Frame0-Geom 2000 decodes the geometry of frame 0. The output of Frame0-Geom 2000 includes a reconstructed spherical coordinate representation of geometry posSph0 (UPRF) 2002. The transformed spherical coordinate representation posSph0 is obtained from the spherical coordinate representation posSph0. (The transformed UPRF, or still referred to as UPRF) 2006.

[0270] Frame0-Attr 2004 describes the decoding of the attributes of frame 0. The transformed spherical coordinates are represented as posSph0. This is the input to Frame0-Attr 2004. The output of this block includes the transformed spherical coordinate representation of the geometry, posSph0. 2006 and the corresponding attribute attr0 2008. In this example, posSph0 2006 will be UPRF of frame 1.

[0271] The spherical table representation of reference frame SphTable0 2012 is the PRF used for the geometry of frame 1. It is derived from posSph0 using the first process described above (such as the process in Figure 18). It was derived.

[0272] Frame1-Geom 2010 depicts the decoding of the geometry of Frame 1 using inter-frame prediction. The input to Frame1-Geom 2010 includes the spherical table representation of the reference frame SphTable0.

[0273] Reference frame posSph0 The spherical coordinate representation of 2016 is the PRF (referred to as PRF-A) for the attributes of frame 1. It is derived from SphTable0 using the second process described above.

[0274] Frame1-Attr 2014 describes the decoding of attributes of Frame 1 using inter-frame prediction. The input to Frame1-Attr 2014 includes the spherical coordinate representation posSph0. 2016.

[0275] The following is an example of implementing the first process (e.g., UPRF to PRF), illustrated in Figure 21. Figure 21 details the process of scaling and offsetting from the transformed UPRF SphTable0. 2124 Generate PRF SphTable0 The method. UPRF is stored as a spherical table (e.g., SphTable0). 2124), where laser IDx 2122 and quantized azimuth 2120 are used as lookup parameters; radius, azimuth, and attributes are stored as entries in table SphTable0. 2124.

[0276] Spherical coordinates are represented as posSph0 2100 includes radius (rad) 2102, Azimuth (phi) 2104 and Laser ID 2106. posSph0 can be obtained by applying offset and scaling to the reconstructed spherical coordinates. 2100 (Scaling and offset may vary for different coordinates). In this example, the first process may involve scaling InvScale and offsetting InvOffset 2108, 2110, and 2112 respectively to obtain the radius radx 2114, azimuth phix 2116, and laser ID LaserIdx 2122 in the original representation. The quantized azimuth qPhix 2120 can be derived from the azimuth Phix 2116. qPhix 2120 and laserIdx 2122 can be used as two coordinates to generate a spherical lookup table (SphTable0). )2124 is used for lookup, and points radx 2114, phix 2116, laserIdx 2122 and attributes are stored as entries (laserIdx does not need to be stored because laserIdx can be used as an index in the index of the table's bins).

[0277] In the foregoing, the terms InvScale and InvOffset were used; however, when describing the example techniques of this disclosure, scaling and offset are further used. This is because scaling and offset can themselves be viewed as the inverse process of spherical coordinate transformation. Generally, InvScale and InvOffset can be considered as some numbers used to perform the example techniques.

[0278] In some examples, the parameters used in the first process can be derived from the parameters used in the spherical coordinate transformation or other syntax elements in the bitstream.

[0279] The following describes an example of a method for generating a PRF from a transformed PRF by applying scaling and offset operations, followed by quantization of the azimuth angle. The output of this process is an UPRF, referred to as posSph0-PRF, as illustrated in Figure 22.

[0280] In this example, the first process may involve scaling and offsetting to use InvScale and InvOffset 2210, 2212, and 2214 respectively from posSph0 2200 rad. 2204, phi 2206 and laserId 2208 retrieves the radius Radx 2210, azimuth Phix 2218, and laser ID LaserIdx 2220 from the original representation. The quantized azimuth qPhix 2224 can be derived from the azimuth Phix 2218. The UPRF posSph0-PRF 2226 can be obtained by storing each point as Radx 2216, qPhix 2224, and laserIdx 2226 (and attribute 2202). In some examples, this UPRF is not stored as a spherical table.

[0281] The following describes an example of a method for generating a second PRF (PRF-A) that can be used in inter-frame prediction of attributes. As described above, a PRF can be generated from the UPRF using a first process; this PRF (e.g., a first-level processed frame) can be used to decode geometry. To decode attributes, a second-level processed frame PRF-A (e.g., a second-level processed frame) can be derived from the PRF.

[0282] In this example, PRF-A is generated from PRF by simply reversing the operations of the first process described above. Since the complete azimuth points are stored in the spherical table, additional azimuth scaling may not be necessary. All three coordinates are scaled back to the original domain using offset and scaling to obtain PRF-A, posSph0. It can be used for inter-frame prediction of attributes, as illustrated in Figure 23.

[0283] For example, in Figure 23, SphTable0 2300 includes rad Component 2302, phi Component 2304 and laserID Component 2306. Apply offsets and scaling to 2308, 2310, and 2312 to include rad. Component 2302, phi Component 2304 and laserID Component 2306 to generate posSph0 2320 rad Component 2314, phi Component 2316 and laserID Quantity 2318.

[0284] In the example of Figure 23, by simply reversing the operations of the first process described above, from PRF (e.g., SphTable0) 2302) Generate PRF-A (e.g., posSph0) 2320). First, the quantized azimuth angle is scaled back to the original domain; then scaling and offset can be applied to obtain PRF-A, posSph0-PRF-A, which can be used for inter-frame prediction of attributes.

[0285] In one or more examples, the PRF can be generated directly from the UPRF (unlike the transformed UPRF in the previous examples), as shown in Figure 24. The method for generating the PRF SphTable0 can be the same as the currently used method. (Using the second process) the PRF-A posSph0 for inter-frame prediction of attributes is derived from the PRF SphTable0. x.

[0286] For example, G-PCC encoder 200 and G-PCC decoder 300 may apply a first process to a reference point cloud frame (e.g., posSph0 2400) to generate a first-level processed frame (e.g., SphTable0 2404). As an example, G-PCC encoder 200 and G-PCC decoder 300 may perform the example techniques of Figures 18, 21, or 22.

[0287] For example, to apply the first process to the reference point cloud frame to generate the first-level processed frame, the G-PCC encoder 200 and G-PCC decoder 300 may store the radius components (e.g., rad) and azimuth components (e.g., phi) of k points of the reference point cloud frame associated with the laser identifier component in a table for each of the plurality of quantized azimuth components (e.g., qphi) and for the laser identifier component (e.g., laserID). The value of k may be greater than or equal to 1 and may be notified by sending a signal, received, or preset. In some examples, each of the plurality of quantized azimuth components is an index of the table, and the table is at least a portion of the first-level processed frame. For example, SphTable0 2404 may be at least a portion of the first-level processed frame.

[0288] The G-PCC encoder 200 and G-PCC decoder 300 can apply a second process to the first-level processed frame (e.g., SphTable02404) to generate a second-level processed frame (e.g., posSph0). x 2406, also known as posSph0 For example, to apply the second process to the first-level processed frame to generate the second-level processed frame, the G-PCC encoder 200 and the G-PCC decoder 300 may apply offsets and scaling to one or more of the radius component, azimuth component, and laser marking component for each point in the first-level processed frame (e.g., SphTable0 2404) to generate the second-level processed frame (e.g., posSph0). x 2406). As an example, the G-PCC encoder 200 and G-PCC decoder 300 can perform the example technique illustrated in Figure 23 to generate posSph0. x 2406.

[0289] As described, the example techniques facilitate efficient memory usage. For example, the G-PCC encoder 200 and G-PCC decoder 300 can store a first-level processed frame (e.g., SphTable0 2404) in a buffer, and apply the second process to that first-level processed frame (e.g., SphTable0 2404) to generate the second-level processed frame (e.g., posSph0). (x 2406), the G-PCC encoder 200 and the G-PCC decoder 300 can access the first-level processed frame (e.g., SphTable0 2404) from this buffer. In this way, a buffer storing SphTable0 2404 can be maintained frame by frame.

[0290] The G-PCC encoder 200 can use the first-level processed frame (e.g., SphTable0 2404) to perform inter-frame predictive coding on the geometric data of the points in the current point cloud frame, and the G-PCC decoder 300 can use the first-level processed frame to perform inter-frame predictive decoding on the geometric data of the points in the current point cloud frame. The G-PCC encoder 200 can use the second-level processed frame (posSph0) x 2406) performs inter-frame prediction coding on the attribute data of the points in the current point cloud frame, and the G-PCC decoder 300 can use the second-level processed frame to perform inter-frame prediction decoding on the attribute data of the points in the current point cloud frame.

[0291] Geometric data may include at least coordinate data, and attribute data may include at least color data, reflectance data, or both color data and reflectance data. In one or more examples, the first-level processed frame (e.g., SphTable02404) and the second-level processed frame (posSph0) x 2406) may include coordinate information. The coordinate information in SphTable0 2404 is used for inter-frame predictive coding or decoding of geometric data, and posSph0 The coordinate information in x 2406 is used to encode or decode attribute data. For example, actual attribute data (e.g., color, reflectivity, etc.) can be compared with the second-level processed frame (e.g., posSph0). The coordinate information of the points in x 2406 is associated. Therefore, both the first-level processed frame and the second-level processed frame may include coordinate information, but the coordinate information in the first-level processed frame can be used for inter-frame predictive coding or decoding of geometric data, and the coordinate information in the second-level processed frame can be used for inter-frame predictive coding or decoding of attribute data.

[0292] As illustrated in Figure 24, in one or more examples, the G-PCC encoder 200 and G-PCC decoder 300 may apply a third process to generate information (e.g., posSph0) for encoding or decoding attribute data of points in a reference point cloud frame (e.g., inter-frame or intra-frame predictive encoding or decoding). 2402). For example, the G-PCC encoder 200 and G-PCC decoder 300 can perform the example technique of Figure 16 to generate posSph0. 2402. The G-PCC encoder 200 can use this information (e.g., posSph0). 2402) The attribute data of the points in the reference point cloud frame is encoded, and the G-PCC decoder 300 can use this information to decode the attribute data of the points in the reference point cloud frame.

[0293] In one or more examples, the G-PCC encoder 200 and G-PCC decoder 300 may perform example techniques when inter-frame prediction is enabled. For example, the G-PCC encoder 200 and G-PCC decoder 300 may determine whether inter-frame prediction encoding or decoding is enabled for the current point cloud frame. In this case, in order to apply both the first and second processes, the G-PCC encoder 200 and G-PCC decoder 300 may apply both processes only if it is determined that inter-frame prediction encoding or decoding is enabled for the current point cloud frame.

[0294] The syntax element `inter_azim_scale_log2` controls the quantization of azimuth values ​​used for inter-frame prediction in predictive geometry decoding. This element quantizes the azimuth values, which are then used as indices in the inter-frame predictive geometry decoding reference (along with the laser index) `SphTable0`; these two indices together specify a cell in the reference table, with each cell containing one point. The minimum azimuth scale value (corresponding to `inter_azim_scale_log2` equal to 0) specifies quantization with no points. However, since each cell contains only one point, some points with the same azimuth value will not be stored in the reference table. This leads to a loss of reference frame fidelity.

[0295] Having at least one mode that does not quantize a reference frame or does not affect fidelity is desirable. Some signaling techniques do not support this.

[0296] In one or more examples, the G-PCC encoder 200 may signal an instruction not to perform quantization on points in the reference frame. Quantization in this case may also include dropping points. This signaling instruction can be implemented in one of several ways:

[0297] a. A flag can be signaled to indicate that quantization of points in the reference frame should not be performed. When this flag indicates that quantization should not be performed, other syntax elements related to quantization (e.g., inter_azim_scale_log2) may not be signaled.

[0298] b. The syntax element inter_azim_scale_log2 can be signaled as inter_azim_scale_log2_plus1; if inter_azim_scale_log2_plus1 is equal to 0, no quantization is performed; otherwise, quantization is performed, where inter_azim_scale_log2 is set to equal inter_azim_scale_log2_plus1 – 1.

[0299] In some examples, points in the reference frame may be discarded / not included, and the remaining points can be used for inter-frame prediction (e.g., inter-frame prediction of geometry); however, the discarded points are kept in a separate list so that they can be used for later decoding (e.g., inter-frame prediction of attributes). Figures 25 through 27 illustrate this.

[0300] In Figure 25, process 1 2502 is used from UPRF posSph0 2500 generates a spherical table (SphericalTable0). 2504; Not included in the spherical table from posSph0 Point 2500 is output as discarded point 2506. In this case, a spherical table with quantized azimuth angles can be generated for lookup.

[0301] Figure 26 illustrates how the PRF SphericalTable0 can be processed using a second process (process 2 2604). 2600 to obtain PRF-A posSph0 2606. However, instead of directly using posSph0... Instead of performing inter-frame prediction of attributes using 2606, the discarded points from process 1 are also appended (e.g., appended point 2608) to obtain the spherical coordinate representation posSph0. _final 2610. In some cases, the second process can be redefined to include both process 2 and the additional point. Therefore, posSph0 _final 2610 is a first-level processed frame (e.g., SphericalTable0) used for inter-frame predictive encoding or decoding of the attribute data of points in the current point cloud frame. Example of a second-level processed frame generated (2600).

[0302] Figure 27 illustrates how to modify process 1 to output a list of discarded points again. The example in Figure 27 begins at start 2700, and the G-PCC encoder 200 and G-PCC decoder 300 determine posSph0. Are there more points in 2702? If there are no more points ("No" in 2702), the example technique can end 2712. If there are more points ("Yes" in 2702), the G-PCC encoder 200 and G-PCC decoder 300 can determine whether point P will be added to the spherical table (2704).

[0303] If no addition is made ("No" in 2704), the example returns to the beginning at 2700. If addition is made ("Yes" in 2704), the G-PCC encoder 200 and G-PCC decoder 300 determine whether point P should replace point Q in the Spherical Table (2706). If replacement is made ("Yes" in 2706), the G-PCC encoder 200 and G-PCC decoder 300 add P to SphericalTable0. (2710), and return to the beginning 2700. If no replacement is made ("No" in 2706), then the G-PCC encoder 200 and the G-PCC decoder 300 can add Q to the list of discarded points (2708) and add P to SphericalTable0. (2710), and return to the beginning 2700.

[0304] In this way, UPRF posSph0 is parsed based on one or more conditions. The points in the table can be used to determine whether to add point P from the UPRF table to SphericalTable0. If point P will replace another point Q already existing in the sphere table, then Q will be added to the list of discarded points. If one or more conditions are met, then P will be added to the sphere table.

[0305] The above-described example technique applies to both the G-PCC encoder 200 and the G-PCC decoder 300. The difference in processing at the encoder and decoder will be regarding the indication of points being discarded in process 1 (e.g., a flag can be sent to notify), which can then be used in generating processed reference frames for inter-frame attribute prediction.

[0306] Figure 28 is a conceptual diagram illustrating an example ranging system 2800 that can be used with one or more techniques of this disclosure. In the example of Figure 28, the ranging system 2800 includes an illuminator 2802 and a sensor 2804. The illuminator 2802 may emit light 2806. In some examples, the illuminator 2802 may emit light 2806 as one or more laser beams. Light 2806 may be one or more wavelengths, such as infrared wavelengths or visible light wavelengths. In other examples, light 2806 is not a coherent laser. When light 2806 encounters an object (such as object 2808), light 2806 produces reflected light 2810. The reflected light 2810 may include backscattered light and / or reflected light. The reflected light 2810 may pass through a lens 2811, which guides the reflected light 2810 to generate an image 2812 of object 2808 on sensor 2804. Sensor 2804 generates a signal 2814 based on the image 2812. Image 2812 may include a set of points (e.g., points represented by small dots in image 2812 of FIG28).

[0307] In some examples, illuminator 2802 and sensor 2804 may be mounted on a rotating structure, allowing illuminator 2802 and sensor 2804 to capture a 360-degree view of the environment (e.g., a rotating LIDAR sensor). In other examples, ranging system 2800 may include one or more optical components (e.g., mirrors, collimators, diffraction gratings, etc.) that enable illuminator 2802 and sensor 2804 to detect objects within a specific range (e.g., up to 360 degrees). Although the example in Figure 28 shows only a single illuminator 2802 and sensor 2804, ranging system 2800 may include multiple sets of illuminators and sensors.

[0308] In some examples, illuminator 2802 generates a structured light pattern. In such examples, ranging system 2800 may include multiple sensors 2804 on which corresponding images of the structured light pattern are formed. Ranging system 2800 can use the differences between the images of the structured light pattern to determine the distance to object 2808 from which the structured light pattern backscatters. When object 2808 is relatively close to sensor 2804 (e.g., 0.2 meters to 2 meters), the structured light-based ranging system can have a high level of accuracy (e.g., sub-millimeter accuracy). This high level of accuracy can be useful in facial recognition applications such as unlocking mobile devices (e.g., mobile phones, tablets, etc.) and for security applications.

[0309] In some examples, the ranging system 2800 is a time-of-flight (ToF) based system. In some examples of the ToF-based ranging system 2800, an illuminator 2802 generates pulses of light. In other words, the illuminator 2802 can modulate the amplitude of the emitted light 2806. In such examples, a sensor 2804 detects the return light 2810 from the pulse of light 2806 generated by the illuminator 2802. The ranging system 2800 can then determine the distance to an object 2808 from which the light 2806 backscatters, based on the delay between the emission of the light 2806 and the detection of the light, and the known speed of light in air. In some examples, the illuminator 2802 can modulate the phase of the emitted light 2806 instead of (or in addition to) modulating the amplitude of the emitted light 2806. In such an example, sensor 2804 can detect the phase of the return light 2810 from object 2808, and use the speed of light and the time difference between when illuminator 2802 generates light 2806 at a specific phase and when sensor 2804 detects the return light 2810 at that specific phase to determine the distance to a point on object 2808.

[0310] In other examples, point clouds can be generated without using illuminator 2802. For example, in some examples, sensor 2804 of ranging system 2800 may include two or more optical cameras. In such examples, ranging system 2800 may use optical cameras to capture stereo images of the environment including object 2808. Ranging system 2800 may include point cloud generator 2816 that can calculate differences between locations in the stereo image. Ranging system 2800 can then use these differences to determine distances to locations shown in the stereo image. Based on these distances, point cloud generator 2816 can generate point clouds.

[0311] Sensor 2804 can also detect other properties of object 2808, such as color and reflectivity information. In the example of FIG. 28, point cloud generator 2816 can generate a point cloud based on signal 2814 generated by sensor 2804. Ranging system 2800 and / or point cloud generator 2816 can form part of data source 104 (FIG. 1). Therefore, the point cloud generated by ranging system 2800 can be encoded and / or decoded according to any techniques of the present disclosure. Inter-frame prediction and residual prediction as described in the present disclosure can reduce the size of the encoded data.

[0312] Figure 29 is a conceptual diagram illustrating an example vehicle-based scenario in which one or more technologies of this disclosure may be used. In the example of Figure 29, vehicle 2900 includes a ranging system 2902. The ranging system 2902 may be implemented in a manner discussed with respect to Figure 28. Although not shown in the example of Figure 29, vehicle 2900 may also include a data source (such as data source 104 (Figure 1)) and a G-PCC encoder (such as G-PCC encoder 200 (Figure 1)). In the example of Figure 29, the ranging system 2902 emits laser beams 2904 that are reflected from pedestrians 2906 or other objects on the road. The data source of vehicle 2900 may generate a point cloud based on the signal generated by the ranging system 2902. The G-PCC encoder of vehicle 2900 may encode the point cloud to generate a bitstream 2908, such as a geometric bitstream (Figure 2) and an attribute bitstream (Figure 2). Inter-frame prediction and residual prediction as described in this disclosure may reduce the size of the geometric bitstream. Bitstream 2908 may contain far fewer bits than the uncoded point cloud obtained by the G-PCC encoder.

[0313] The output interface of vehicle 2900 (e.g., output interface 108 (FIG. 1)) can send bit stream 2908 to one or more other devices. Bit stream 2908 may include significantly fewer bits than the uncoded point cloud obtained by the G-PCC encoder. Therefore, vehicle 2900 may be able to send bit stream 2908 to other devices much faster than uncoded point cloud data. Additionally, bit stream 2908 may require less data storage capacity on the device.

[0314] In the example of Figure 29, vehicle 2900 may send bitstream 2908 to another vehicle 2910. Vehicle 2910 may include a G-PCC decoder, such as G-PCC decoder 300 (Figure 1). The G-PCC decoder of vehicle 2910 may decode bitstream 2908 to reconstruct a point cloud. Vehicle 2910 may use the reconstructed point cloud for various purposes. For example, vehicle 2910 may determine, based on the reconstructed point cloud, that pedestrian 2906 is on the road ahead of vehicle 2900, and thus begin to decelerate, for example, even before the driver of vehicle 2910 becomes aware of pedestrian 2906 on the road. Therefore, in some examples, vehicle 2910 may perform autonomous navigation operations based on the reconstructed point cloud.

[0315] Additionally or alternatively, vehicle 2900 may send bitstream 2908 to server system 2912. Server system 2912 may use bitstream 2908 for various purposes. For example, server system 2912 may store bitstream 2908 for subsequent reconstruction of the point cloud. In this example, server system 2912 may use the point cloud together with other data (e.g., vehicle telemetry data generated by vehicle 2900) to train an autonomous driving system. In other examples, server system 2912 may store bitstream 2908 for subsequent reconstruction for accident forensic investigations.

[0316] Figure 30 is a conceptual diagram illustrating an example extended reality system in which one or more technologies of this disclosure may be used. Extended reality (XR) is a term used to cover a range of technologies including augmented reality (AR), mixed reality (MR), and virtual reality (VR). In the example of Figure 30, user 3000 is located at a first location 3002. User 3000 wears an XR headset 3004. Alternatively, user 3000 may use a mobile device (e.g., a mobile phone, tablet computer, etc.) as an alternative to the XR headset 3004. The XR headset 3004 includes a depth sensor, such as a ranging system, which detects the position of a point on object 3006 at location 3002. The data source of the XR headset 3004 may use the signal generated by the depth sensor to generate a point cloud representation of object 3006 at location 3002. XR head-mounted device 3004 may include a G-PCC encoder (e.g., G-PCC encoder 200 of FIG1) configured to encode point clouds to generate a bitstream 3008. Inter-frame prediction and residual prediction as described in this disclosure can reduce the size of the bitstream 3008.

[0317] XR headset 3004 can send bitstream 3008 (e.g., via a network, such as the Internet) to XR headset 3010 worn by user 3012 at second location 3014. XR headset 3010 can decode bitstream 3008 to reconstruct a point cloud. XR headset 3010 can use the point cloud to generate an XR visualization (e.g., AR visualization, MR visualization, VR visualization) representing object 3006 at location 3002. Thus, in some examples, such as when XR headset 3010 generates a VR visualization, user 3012 can have a 3D immersive experience of location 3002. In some examples, XR headset 3010 can determine the location of virtual objects based on the reconstructed point cloud. For example, XR headset 3010 can determine, based on the reconstructed point cloud, that the environment (e.g., location 3002) includes flat surfaces, and then determine that virtual objects (e.g., cartoon characters) should be positioned on that flat surface. The XR headset 3010 can generate XR visualizations in which virtual objects are located at a defined position. For example, the XR headset 3010 can show a cartoon character sitting on a flat surface.

[0318] Figure 31 is a conceptual diagram illustrating an example mobile device system in which one or more technologies of this disclosure may be used. In the example of Figure 31, a mobile device 3100, such as a mobile phone or tablet computer (e.g., a wireless communication device), includes a ranging system (such as a LiDAR system) that detects the location of points on an object 3102 within the environment of the mobile device 3100. The data source of the mobile device 3100 may use signals generated by a depth detection sensor to generate a point cloud representation of the object 3102. The mobile device 3100 may include a G-PCC encoder (e.g., the G-PCC encoder 200 of Figure 1) configured to encode the point cloud to generate a bitstream 3104. In the example of Figure 31, the mobile device 3100 may send the bitstream to a remote device 3106 (such as a server system or other mobile device). Inter-frame prediction and residual prediction as described in this disclosure may reduce the size of the bitstream 3104. The remote device 3106 may decode the bitstream 3104 to reconstruct the point cloud. The remote device 3106 may use the point cloud for various purposes. For example, remote device 3106 can use point clouds to generate an environmental map of mobile device 3100. For example, remote device 3106 can generate a map of the interior of a building based on the reconstructed point cloud. In another example, remote device 3106 can generate an image (e.g., computer graphics) based on the point cloud. For example, remote device 3106 can use points in the point cloud as vertices of polygons and use the color attributes of the points as the basis for coloring the polygons. In some examples, remote device 3106 can use the reconstructed point cloud for facial recognition or other security applications.

[0319] Figure 32 is a flowchart illustrating an example technique for decoding point cloud data. The example in Figure 32 is described in relation to processing circuitry, such as the processing circuitry of a G-PCC encoder 200. A device including the G-PCC encoder 200 may also include one or more LiDAR sensors configured to capture points of the current point cloud frame for which the G-PCC encoder 200 will encode its attribute and geometric data.

[0320] The processing circuitry of the G-PCC encoder 200 can be configured to apply a first process to a reference point cloud frame to generate a first-level processed frame (3200). The processing circuitry of the G-PCC encoder 200 can be configured to store the radius and azimuth components of k points of the reference point cloud frame associated with the laser marker component in a table for each of a plurality of quantized azimuth components and for the laser marker component. In this example, k can be greater than or equal to 1, and the value of k can be communicated, received, or preset by transmitting a signal. Each of the plurality of quantized azimuth components can be an index of the table, and the table is at least a portion of the first-level processed frame. Example techniques for generating the first-level processed frame include those illustrated in Figures 18 and 21, as two non-limiting examples, where SphTable0 is an example of the first-level processed frame.

[0321] The processing circuitry of the G-PCC encoder 200 can apply a second process to the first-level processed frame to generate a second-level processed frame (3202). For example, as illustrated in Figure 24, posSph0 is generated from SphTable0. x 2406, which is an example of the second-level processed frame. As an example, to apply the second process to the first-level processed frame to generate the second-level processed frame, the processing circuitry of the G-PCC encoder 200 can be configured to apply offset and scaling to one or more of the radius component, azimuth component, and laser marking component for each point in the first-level processed frame to generate the second-level processed frame. Example techniques for generating the second-level processed frame are illustrated in Figures 16 and 23 as two non-limiting examples, where posSph0 x or posSph0 This is an example of a frame that has been processed at the second level.

[0322] As an example, the processing circuitry of the G-PCC encoder 200 may store the first-level processed frame in a buffer. In order to apply the second process to the first-level processed frame to generate the second-level processed frame, the processing circuitry of the G-PCC encoder 200 may be configured to access the first-level processed frame from the buffer.

[0323] The processing circuitry of the G-PCC encoder 200 can be configured to perform inter-frame predictive coding (3204) on the geometric data of points in the current point cloud frame using the first-level processed frame. This geometric data may include at least coordinate data.

[0324] The processing circuitry of the G-PCC encoder 200 can be configured to perform inter-frame predictive coding (3206) on the attribute data of points in the current point cloud frame using the second-level processed frame. The attribute data may include at least color data, reflectance data, or both color data and reflectance data.

[0325] In some examples, the processing circuitry of the G-PCC encoder 200 can be configured to apply a third process to generate information for encoding attribute data of points in the reference point cloud frame (e.g., inter-frame or intra-frame predictive coding), and to use that information to encode the attribute data of points in the reference point cloud frame. For example, as illustrated in Figure 24, the processing circuitry of the G-PCC encoder 200 can generate posSph0 from posSph0 2400. 2402. The processing circuit of the G-PCC encoder 200 can use posSph0. 2402 is used to encode the attribute data of the points in the reference point cloud frame (e.g., intra-frame predictive coding), but posSph0 is used. x 2406 is used to perform inter-frame predictive coding on the attribute data of the points in the current point cloud frame. As illustrated in Figure 24, the processing circuit of the G-PCC encoder 200 can generate posSph0 from SphTable0 2404 (e.g., the first-level processed frame). x 2406 (e.g., second-level processed frames), where SphTable0 2404 is generated from posSph0 (e.g., reference point cloud).

[0326] Figure 33 is a flowchart illustrating an example technique for decoding point cloud data. The example in Figure 33 is described in relation to processing circuitry, such as the G-PCC decoder 300. A device including the G-PCC decoder 300 may also include a display configured to present an image based on points of the current point cloud frame from which the G-PCC decoder 300 decodes its attribute and geometric data.

[0327] The processing circuitry of the G-PCC decoder 300 can be configured to apply a first process to a reference point cloud frame to generate a first-level processed frame (3300). The processing circuitry of the G-PCC decoder 300 can be configured to store the radius and azimuth components of k points of the reference point cloud frame associated with the laser marker component in a table for each of a plurality of quantized azimuth components and for the laser marker component. In this example, k can be greater than or equal to 1, and the value of k can be communicated, received, or preset by transmitting a signal. Each of the plurality of quantized azimuth components can be an index of the table, and the table is at least a portion of the first-level processed frame. Example techniques for generating the first-level processed frame include those illustrated in Figures 18 and 21, as two non-limiting examples, where SphTable0 is an example of the first-level processed frame.

[0328] The processing circuitry of the G-PCC decoder 300 can apply a second process to the first-level processed frame to generate a second-level processed frame (3302). For example, as illustrated in Figure 24, posSph0 is generated from SphTable0. x 2406, which is an example of the second-level processed frame. As an example, to apply the second process to the first-level processed frame to generate the second-level processed frame, the processing circuitry of the G-PCC decoder 300 can be configured to apply offset and scaling to one or more of the radius component, azimuth component, and laser marker component for each point in the first-level processed frame to generate the second-level processed frame. Example techniques for generating the second-level processed frame are illustrated in Figures 16 and 23 as two non-limiting examples, where posSph0 x or posSph0 This is an example of a frame that has been processed at the second level.

[0329] As an example, the processing circuitry of the G-PCC decoder 300 may store the first-level processed frame in a buffer. In order to apply the second process to the first-level processed frame to generate the second-level processed frame, the processing circuitry of the G-PCC decoder 300 may be configured to access the first-level processed frame from the buffer.

[0330] The processing circuitry of the G-PCC decoder 300 can be configured to perform inter-frame predictive decoding (3304) on the geometric data of points in the current point cloud frame using the first-level processed frame. The geometric data may include at least coordinate data.

[0331] The processing circuitry of the G-PCC decoder 300 can be configured to perform inter-frame predictive decoding (3306) on the attribute data of points in the current point cloud frame using the second-level processed frame. The attribute data may include at least color data, reflectance data, or both color data and reflectance data.

[0332] In some examples, the processing circuitry of the G-PCC decoder 300 can be configured to apply a third process to generate information for decoding the attribute data of the points in the reference point cloud frame (e.g., inter-frame or intra-frame predictive decoding), and to use that information to decode the attribute data of the points in the reference point cloud frame. For example, as illustrated in Figure 24, the processing circuitry of the G-PCC decoder 300 can generate posSph0 from posSph0 2400. 2402. The processing circuitry of the G-PCC decoder 300 can use posSph0. 2402 is used to decode the attribute data of the points in the reference point cloud frame, but posSph0 is used. x 2406 is used to perform inter-frame prediction decoding on the attribute data of the points in the current point cloud frame. As illustrated in Figure 24, the processing circuitry of the G-PCC decoder 300 can generate posSph0 from SphTable0 2404 (e.g., the first-level processed frame). x 2406 (e.g., second-level processed frames), where SphTable0 2404 is generated from posSph0 (e.g., reference point cloud).

[0333] Examples from the various aspects of this disclosure may be used individually or in any combination.

[0334] Clause 1A. A method for decoding point cloud data, the method comprising: applying a process to a reference frame to generate a processed frame; and decoding a current frame based on the processed frame, wherein the reference frame and the current frame comprise point cloud data frames.

[0335] Clause 2A. The method according to Clause 1A, wherein the process is a first process, wherein the processed frame is a first-level processed frame, the method further comprising: applying a second process to the first-level processed frame to generate a second-level processed frame, wherein decoding includes decoding the current frame based on the second-level processed frame.

[0336] Clause 3A. The method according to Clause 2A, wherein decoding includes: decoding the geometric data of the current frame using the first level processed frames; and decoding the attribute data of the current frame using the second level processed frames.

[0337] Clause 4A. The method according to any one of Clauses 2A to 3A, the method further comprising: notifying or receiving one or more parameters for the first process by sending a signal; and deriving one or more parameters for the second process based on the one or more parameters for the first process.

[0338] Clause 5A. The method according to any one of Clauses 2A to 3A, the method further comprising: notifying or receiving one or more parameters for the second process by sending a signal.

[0339] Clause 6A. The method according to any one of Clauses 1A to 5A, the method further comprising: notifying by sending a signal or receiving an indication of whether to apply the procedure to the reference frame by one or more syntax elements, wherein applying the procedure includes applying the procedure if the one or more syntax elements indicate that the procedure should be applied.

[0340] Clause 7A. The method according to any one of Clauses 1A to 6A, wherein decoding includes decoding at least one of geometric data of points in the point cloud of the current frame or attribute data of points in the point cloud of the current frame.

[0341] Clause 8A. The method according to any one of Clauses 1A to 7A, wherein decoding includes inter-frame predictive decoding.

[0342] Clause 9A. The method according to any one of Clauses 1A to 8A, wherein decoding includes encoding.

[0343] Clause 10A. The method according to any one of Clauses 1A to 8A, wherein decoding includes decoding.

[0344] Clause 11A. The method according to any one of Clauses 1A to 9A, the method further comprising generating the point cloud data.

[0345] Clause 12A. An apparatus for decoding point cloud data, the apparatus comprising: one or more memories configured to store the point cloud data; and processing circuitry configured to perform a method according to any one or more of Clauses 1A to 11A.

[0346] Clause 13A. The device as described in Clause 12A, the device further includes a display for presenting an image based on the point cloud.

[0347] Clause 14A. A computer-readable storage medium having instructions stored thereon, which, when executed, cause one or more processors to perform the method according to any one of Clauses 1A to 11A.

[0348] Clause 15A. An apparatus for decoding point cloud data, the apparatus comprising components for performing the method of any one of Clauses 1A to 11A.

[0349] Clause 1. An apparatus for decoding point cloud data, the apparatus comprising: one or more memories configured to store the point cloud data; and processing circuitry coupled to the one or more memories, wherein the processing circuitry is configured to: apply a first process to a reference point cloud frame to generate a first-level processed frame; apply a second process to the first-level processed frame to generate a second-level processed frame; perform inter-frame prediction decoding on geometric data of points in a current point cloud frame using the first-level processed frame; and perform inter-frame prediction decoding on attribute data of points in the current point cloud frame using the second-level processed frame.

[0350] Clause 2. The apparatus according to Clause 1, wherein, in order to apply the first process to the reference point cloud frame to generate the first-level processed frame, the processing circuitry is configured to: for each of a plurality of quantized azimuth components and for a laser marker component, store the radius and azimuth components of k points of the reference point cloud frame associated with the laser marker component in a table, wherein k is greater than or equal to 1, wherein each of the plurality of quantized azimuth components is an index of the table, and wherein the table is at least a portion of the first-level processed frame.

[0351] Clause 3. The device as described in Clause 2, wherein the value of k is received.

[0352] Clause 4. The device according to any one of Clauses 1 to 3, wherein, in order to apply the second process to the first level processed frame to generate the second level processed frame, the processing circuit is configured to: for each point in the first level processed frame, apply offset and scaling to one or more of the radius component, azimuth component, and laser marking component to generate the second level processed frame.

[0353] Clause 5. The device according to any one of Clauses 1 to 5, wherein the processing circuitry is configured to: store the first level processed frame in a buffer, wherein in order to apply the second process to the first level processed frame to generate the second level processed frame, the processing circuitry is configured to access the first level processed frame from the buffer.

[0354] Clause 6. The apparatus according to any one of Clauses 1 to 5, wherein the processing circuitry is configured to: apply a third process to generate information for decoding attribute data of points in the reference point cloud frame; and use the information to decode the attribute data of the points in the reference point cloud frame.

[0355] Clause 7. The device according to any one of Clauses 1 to 6, wherein the geometric data includes coordinate data, and wherein the attribute data includes color data, reflectance data, or both color data and reflectance data.

[0356] Clause 8. The device according to any one of Clauses 1 to 7, the device further comprising a display configured to render an image based on the current point cloud frame.

[0357] Clause 9. An apparatus for encoding point cloud data, the apparatus comprising: one or more memories configured to store the point cloud data; and processing circuitry coupled to the one or more memories, wherein the processing circuitry is configured to: apply a first process to a reference point cloud frame to generate a first-level processed frame; apply a second process to the first-level processed frame to generate a second-level processed frame; perform inter-frame predictive coding on geometric data of points in a current point cloud frame using the first-level processed frame; and perform inter-frame predictive coding on attribute data of points in the current point cloud frame using the second-level processed frame.

[0358] Clause 10. The apparatus of Clause 9, wherein, in order to apply the first process to the reference point cloud frame to generate the first-level processed frame, the processing circuitry is configured to: for each of a plurality of quantized azimuth components and for a laser marker component, store the radius and azimuth components of k points of the reference point cloud frame associated with the laser marker component in a table, wherein k is greater than or equal to 1, wherein each of the plurality of quantized azimuth components is an index of the table, and wherein the table is at least a portion of the first-level processed frame.

[0359] Clause 11. The device according to Clause 10, wherein the value of k is notified by sending a signal.

[0360] Clause 12. The apparatus according to any one of Clauses 9 to 11, wherein, in order to apply the second process to the first level processed frame to generate the second level processed frame, the processing circuitry is configured to: for each point in the first level processed frame, apply offset and scaling to one or more of the radius component, azimuth component, and laser marking component to generate the second level processed frame.

[0361] Clause 13. The device according to any one of Clauses 9 to 12, wherein the processing circuitry is configured to: store the first level processed frame in a buffer, wherein in order to apply the second process to the first level processed frame to generate the second level processed frame, the processing circuitry is configured to access the first level processed frame from the buffer.

[0362] Clause 14. The apparatus according to any one of Clauses 9 to 13, wherein the processing circuitry is configured to: apply a third process to generate information for encoding attribute data of points in the reference point cloud frame; and use the information to encode the attribute data of the points in the reference point cloud frame.

[0363] Clause 15. The device according to any one of Clauses 9 to 14, wherein the geometric data comprises coordinate data, and wherein the attribute data comprises color data, reflectance data, or both color data and reflectance data.

[0364] Clause 16. The device according to any one of Clauses 9 to 15, the device further comprising one or more LiDAR sensors configured to capture the point of the current point cloud frame.

[0365] Clause 17. A method for decoding point cloud data, the method comprising: applying a first process to a reference point cloud frame to generate a first-level processed frame; applying a second process to the first-level processed frame to generate a second-level processed frame; performing inter-frame prediction decoding on geometric data of points in a current point cloud frame using the first-level processed frame; and performing inter-frame prediction decoding on attribute data of points in the current point cloud frame using the second-level processed frame.

[0366] Clause 18. The method according to Clause 17, wherein applying the first process to the reference point cloud frame to generate the first-level processed frame comprises: for each of a plurality of quantized azimuth components and for a laser identification component, storing the radius and azimuth components of k points of the reference point cloud frame associated with the laser identification component in a table, wherein k is greater than or equal to 1, wherein each of the plurality of quantized azimuth components is an index of the table, and wherein the table is at least a portion of the first-level processed frame.

[0367] Clause 19. The method described in Clause 18, wherein a value of k is received.

[0368] Clause 20. The method according to any one of Clauses 17 to 19, wherein applying the second process to the first level processed frame to generate the second level processed frame comprises: for each point in the first level processed frame, applying offset and scaling to one or more of the radius component, azimuth component, and laser marking component to generate the second level processed frame.

[0369] Clause 21. The method according to any one of Clauses 17 to 20, the method further comprising: storing the first level processed frame in a buffer, wherein applying the second process to the first level processed frame to generate the second level processed frame includes accessing the first level processed frame from the buffer.

[0370] Clause 22. The method according to any one of Clauses 17 to 21, the method further comprising: applying a third process to generate information for decoding attribute data of points in the reference point cloud frame; and using the information to decode the attribute data of the points in the reference point cloud frame.

[0371] Clause 23. The method according to any one of Clauses 17 to 22, wherein the geometric data comprises coordinate data, and wherein the attribute data comprises color data, reflectance data, or both color data and reflectance data.

[0372] Clause 24. One or more computer-readable storage media storing instructions that, when executed, cause one or more processors to: apply a first process to a reference point cloud frame to generate a first-level processed frame; apply a second process to the first-level processed frame to generate a second-level processed frame; perform inter-frame predictive coding on geometric data of points in a current point cloud frame using the first-level processed frame; and perform inter-frame predictive coding on attribute data of points in the current point cloud frame using the second-level processed frame.

[0373] Clause 25. One or more computer-readable storage media as described in Clause 24, wherein the instructions for causing the one or more processors to apply the first process to the reference point cloud frame to generate a first-level processed frame include causing the one or more processors to: for each of a plurality of quantized azimuth components and for a laser identification component, store in a table the radius and azimuth components of k points of the reference point cloud frame associated with the laser identification component, wherein k is greater than or equal to 1, wherein each of the plurality of quantized azimuth components is an index of the table, and wherein the table is at least a portion of the first-level processed frame.

[0374] Clause 26. One or more computer-readable storage media according to any one of Clauses 24 and 25, wherein the instructions for causing the one or more processors to apply the second process to the first level processed frame to generate the second level processed frame include causing the one or more processors to: for each point in the first level processed frame, apply offset and scaling to one or more of the radius component, azimuth component, and laser marking component to generate the second level processed frame.

[0375] It should be recognized that, based on the examples, certain actions or events of any technique described herein may be performed in a different sequence, and may be added, combined, or omitted entirely (e.g., not all actions or events described are necessary for implementing the technique). Furthermore, in some examples, actions or events may be performed concurrently (e.g., through multithreading, interrupt handling, or multiple processors) rather than sequentially.

[0376] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on a computer-readable medium or transmitted via a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium (which corresponds to a tangible medium such as a data storage medium) or a communication medium, including, for example, any medium that facilitates the transfer of a computer program from one place to another according to a communication protocol. In this way, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described in this disclosure. Computer program products may include computer-readable media.

[0377] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium capable of storing desired program code in the form of instructions or data structures and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies (such as infrared, radio, and microwave), then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies (such as infrared, radio, and microwave) are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but instead refer to non-transient tangible storage media. As used herein, disks and optical discs include compact optical discs (CDs), laser optical discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs utilize lasers to optically reproduce data. The combinations described above should also be included within the scope of computer-readable media.

[0378] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the terms "processor" and "processing circuit" as used herein can refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein can be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into combined codecs. Furthermore, these techniques can be fully implemented in one or more circuit or logic elements.

[0379] The techniques disclosed herein can be implemented in a wide variety of devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or IC sets (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but implementation by different hardware units is not necessarily required. Rather, as described above, various units can be combined in a codec hardware unit, or various units can be provided by a collection of interoperable hardware units (including one or more processors as described above) combined with appropriate software and / or firmware.

[0380] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. An apparatus for decoding point cloud data, the apparatus comprising: One or more memories, the one or more memories being configured to store the point cloud data; and processing circuitry coupled to the one or more memories, wherein the processing circuitry is configured to apply a first process to a reference point cloud frame to generate a first-level processed frame; A second process is applied to the first-level processed frames to generate second-level processed frames; Inter-frame prediction decoding is performed using the geometric data of points in the current point cloud frame from the first-level processed frames. And use the second-level processed frames to perform inter-frame prediction decoding on the attribute data of the points in the current point cloud frame.

2. The apparatus of claim 1, wherein, in order to apply the first process to the reference point cloud frame to generate the first-level processed frame, the processing circuitry is configured to: for each of a plurality of quantized azimuth components and for a laser marker component, store the radius and azimuth components of k points of the reference point cloud frame associated with the laser marker component in a table, wherein k is greater than or equal to 1, wherein each of the plurality of quantized azimuth components is an index of the table, and wherein the table is at least a portion of the first-level processed frame.

3. The device according to claim 2, wherein the value of k is received.

4. The apparatus of claim 1, wherein, in order to apply the second process to the first-level processed frame to generate the second-level processed frame, the processing circuit is configured to: for each point in the first-level processed frame, apply offset and scaling to one or more of the radius component, azimuth component, and laser marking component to generate the second-level processed frame.

5. The apparatus of claim 1, wherein the processing circuitry is configured to: store the first-level processed frame in a buffer, wherein in order to apply the second process to the first-level processed frame to generate the second-level processed frame, the processing circuitry is configured to access the first-level processed frame from the buffer.

6. The apparatus of claim 1, wherein the processing circuitry is configured to: apply a third process to generate information for decoding attribute data of points in the reference point cloud frame; and use the information to decode the attribute data of the points in the reference point cloud frame.

7. The device of claim 1, wherein the geometric data includes coordinate data, and wherein the attribute data includes color data, reflectance data, or both color data and reflectance data.

8. The device of claim 1, further comprising a display configured to render an image based on the current point cloud frame.

9. An apparatus for encoding point cloud data, the apparatus comprising: One or more memories, the one or more memories being configured to store the point cloud data; and processing circuitry coupled to the one or more memories, wherein the processing circuitry is configured to apply a first process to a reference point cloud frame to generate a first-level processed frame; A second process is applied to the first-level processed frames to generate second-level processed frames; Inter-frame predictive coding is performed on the geometric data of the points in the current point cloud frame using the first-level processed frames. And use the second-level processed frames to perform inter-frame predictive coding on the attribute data of the points in the current point cloud frame.

10. The apparatus of claim 9, wherein, in order to apply the first process to the reference point cloud frame to generate the first-level processed frame, the processing circuitry is configured to: for each of a plurality of quantized azimuth components and for a laser marker component, store the radius and azimuth components of k points of the reference point cloud frame associated with the laser marker component in a table, wherein k is greater than or equal to 1, wherein each of the plurality of quantized azimuth components is an index of the table, and wherein the table is at least a portion of the first-level processed frame.

11. The device of claim 10, wherein the value of k is notified by sending a signal.

12. The apparatus of claim 9, wherein, in order to apply the second process to the first-level processed frame to generate the second-level processed frame, the processing circuitry is configured to: for each point in the first-level processed frame, apply offset and scaling to one or more of the radius component, azimuth component, and laser marking component to generate the second-level processed frame.

13. The apparatus of claim 9, wherein the processing circuitry is configured to: store the first-level processed frame in a buffer, wherein in order to apply the second process to the first-level processed frame to generate the second-level processed frame, the processing circuitry is configured to access the first-level processed frame from the buffer.

14. The apparatus of claim 9, wherein the processing circuitry is configured to: apply a third process to generate information for encoding attribute data of points in the reference point cloud frame; and use the information to encode the attribute data of the points in the reference point cloud frame.

15. The device of claim 9, wherein the geometric data comprises coordinate data, and wherein the attribute data comprises color data, reflectance data, or both color data and reflectance data.

16. The device of claim 9, further comprising one or more LiDAR sensors configured to capture the points of the current point cloud frame.

17. A method for decoding point cloud data, the method comprising: The first process is applied to the reference point cloud frame to generate a first-level processed frame; A second process is applied to the first-level processed frames to generate second-level processed frames; Inter-frame prediction decoding is performed using the geometric data of points in the current point cloud frame from the first-level processed frames. And use the second-level processed frames to perform inter-frame prediction decoding on the attribute data of the points in the current point cloud frame.

18. The method of claim 17, wherein applying the first process to the reference point cloud frame to generate the first-level processed frame comprises: For each of the plurality of quantized azimuth components and for the laser identification component, the radius and azimuth components of k points of the reference point cloud frame associated with the laser identification component are stored in a table, where k is greater than or equal to 1, wherein each of the plurality of quantized azimuth components is an index of the table, and wherein the table is at least a portion of the first-level processed frames.

19. The method of claim 18, wherein the value of k is received.

20. The method of claim 17, wherein applying the second process to the first-level processed frame to generate the second-level processed frame comprises: For each point in the first-level processed frame, offset and scaling are applied to one or more of the radius component, azimuth component, and laser marking component to generate the second-level processed frame.

21. The method according to claim 17, further comprising: Storing the first-level processed frame in a buffer, wherein applying the second process to the first-level processed frame to generate the second-level processed frame includes accessing the first-level processed frame from the buffer.

22. The method of claim 17, further comprising: A third process is applied to generate information for decoding the attribute data of the points in the reference point cloud frame; And use the information to decode the attribute data of the points in the reference point cloud frame.

23. The method of claim 17, wherein the geometric data comprises coordinate data, and wherein the attribute data comprises color data, reflectance data, or both color data and reflectance data.

24. One or more computer-readable storage media storing instructions that, when executed, cause one or more processors to: apply a first process to a reference point cloud frame to generate a first-level processed frame; and apply a second process to the first-level processed frame to generate a second-level processed frame. Inter-frame predictive coding is performed on the geometric data of the points in the current point cloud frame using the first-level processed frames. And use the second-level processed frames to perform inter-frame predictive coding on the attribute data of the points in the current point cloud frame.

25. One or more computer-readable storage media of claim 24, wherein the instructions causing the one or more processors to apply the first process to the reference point cloud frame to generate a first-level processed frame include instructions that cause the one or more processors to: for each of a plurality of quantized azimuth components and for a laser identification component, store in a table the radius and azimuth components of k points of the reference point cloud frame associated with the laser identification component, wherein k is greater than or equal to 1, wherein each of the plurality of quantized azimuth components is an index of the table, and wherein the table is at least a portion of the first-level processed frame.

26. The computer-readable storage medium of claim 24, wherein the instructions that cause the one or more processors to apply the second process to the first-level processed frame to generate the second-level processed frame include instructions that cause the one or more processors to: for each point in the first-level processed frame, apply offset and scaling to one or more of a radius component, an azimuth component, and a laser marking component to generate the second-level processed frame.

Images