Attribute coding for point cloud compression

Abstract

A method of encoding point cloud data includes: for a first encoding process, receiving geometric data of point cloud data of a source point cloud; encoding the geometric data according to the first encoding process to generate encoded geometric data and a geometric bit stream of a target point cloud; decoding the encoded geometry data to generate reconstructed geometry data; performing an attribute recalculation process on attribute data of the point cloud data of the source point cloud based on the reconstructed geometric data to generate recalculated reconstructed point cloud data of the target point cloud; and encoding the recalculated reconstructed point cloud data according to a second encoding process to generate an attribute bitstream.

Description

[0001] This application claims priority to U.S. Patent Application No. 18 / 490,467, filed on October 19, 2023, and U.S. Provisional Application No. 63 / 380,522, filed on October 21, 2022, the entire contents of which are incorporated herein by reference. U.S. Patent Application No. 18 / 490,467, filed on October 19, 2023, claims the benefit of U.S. Provisional Application No. 63 / 380,522, filed on October 21, 2022. Technical Field

[0002] The present disclosure relates to point cloud encoding and decoding. Background Art

[0003] A point cloud is a collection of points in 3-dimensional space. A point may correspond to a point on an object within the 3-dimensional space. Therefore, a point cloud may be used to represent the physical contents of a 3-dimensional space. Point clouds may have utility in a wide variety of situations. For example, a point cloud may be used in the context of an autonomous vehicle to represent the location of an object on a road. In another example, a point cloud may be used in the context of representing the physical contents of an environment for the purpose of locating virtual objects in an augmented reality (AR) or mixed reality (MR) application. Point cloud compression is a process used to encode and decode a point cloud. Encoding a point cloud may reduce the amount of data required to store and transmit the point cloud. Summary of the invention

[0004] In general, the present disclosure describes techniques for encoding (e.g., encoding or decoding) geometry data and encoding (e.g., disassociating) attribute data for point cloud data, utilizing encoding techniques that are effective for each of the geometry data and the attribute data while maintaining correspondence (e.g., association) between the geometry data and the attribute data. For example, a first encoding process (e.g., a machine learning-based encoding technique) for the geometry data may be efficient, but the first encoding process may be less efficient for the attribute data (or vice versa). However, encoding the geometry data using the first encoding process (e.g., a machine learning-based encoding technique) without considering the attribute data may result in a disconnection (e.g., disassociation) between the geometry data and the attribute data. Using the example techniques described in the present disclosure, different encoding techniques for the geometry data and the attribute data are possible while maintaining correspondence between the geometry data and the attribute data for the point cloud data.

[0005] Example techniques provide a way in which different encoding and decoding processes can be applied to geometry data and attribute data to provide better compression gain while maintaining the correspondence of geometry data and attribute data. As an example, example techniques can utilize learning-based geometry compression and other compression schemes for attribute compression, use learning-based compression (e.g., different training models) for both geometry compression and attribute compression, or use learning-based attribute compression and other compression schemes for geometry compression. In this way, example techniques improve the overall operation of a point cloud encoding or decoding system by enabling the practical application of different encoding and decoding processes for geometry data and attribute data.

[0006] In one example, the present disclosure describes a method for encoding point cloud data, the method comprising: for a first encoding process, receiving geometric data of point cloud data of a source point cloud; encoding the geometric data according to the first encoding process to generate encoded geometric data and a geometric bit stream; decoding the encoded geometric data to generate reconstructed geometric data of a target point cloud; performing an attribute recalculation process on attribute data of the point cloud data of the source point cloud based on the reconstructed geometric data to generate recalculated reconstructed point cloud data of the target point cloud; and encoding the recalculated reconstructed point cloud data according to a second encoding process to generate an attribute bit stream.

[0007] In one example, the present disclosure describes a method for decoding point cloud data, the method comprising: receiving a geometry bitstream comprising encoded geometry data of point cloud data encoded according to a first encoding process; decoding the encoded geometry data using a first decoding process opposite to the first encoding process to generate reconstructed geometry data; receiving an attribute bitstream comprising encoded attribute data of the point cloud data encoded according to a second encoding process; and decoding the encoded attribute data based on the reconstructed geometry data using a second decoding process opposite to the second encoding process to generate decoded attribute data and reconstructed geometry data of the point cloud data.

[0008] In one example, the present disclosure describes a system for encoding point cloud data, the system comprising: one or more memories configured to store point cloud data; and a processing circuit coupled to the one or more memories and configured to: receive geometric data of point cloud data of a source point cloud for a first encoding process; encode the geometric data according to the first encoding process to generate encoded geometric data and a geometric bitstream; decode the encoded geometric data to generate reconstructed geometric data of a target point cloud; perform an attribute recalculation process on attribute data of the point cloud data of the source point cloud based on the reconstructed geometric data to generate recalculated reconstructed point cloud data of the target point cloud; and encode the recalculated reconstructed point cloud data according to a second encoding process to generate an attribute bitstream.

[0009] In one example, the present disclosure describes a system for decoding point cloud data, the system comprising: one or more memories configured to store point cloud data; and a processing circuit coupled to the one or more memories and configured to: receive a geometry bitstream comprising encoded geometry data of point cloud data encoded according to a first encoding process; decode the encoded geometry data using a first decoding process opposite to the first encoding process to generate reconstructed geometry data; receive an attribute bitstream comprising encoded attribute data of the point cloud data encoded according to a second encoding process; and decode the encoded attribute data based on the reconstructed geometry data using a second decoding process opposite to the second encoding process to generate decoded attribute data and reconstructed geometry data of the point cloud data.

[0010] In one example, the present disclosure describes a computer-readable storage medium having instructions stored thereon, which, when executed, cause one or more processors to: receive geometric data of point cloud data of a source point cloud for a first encoding process; encode the geometric data according to the first encoding process to generate encoded geometric data and a geometric bit stream of a target point cloud; decode the encoded geometric data to generate reconstructed geometric data; perform an attribute recalculation process on attribute data of the point cloud data of the source point cloud based on the reconstructed geometric data to generate recalculated reconstructed point cloud data of the target point cloud; and encode the recalculated reconstructed point cloud data according to a second encoding process to generate an attribute bit stream.

[0011] In one example, the present disclosure describes a computer-readable storage medium having instructions stored thereon, which, when executed, cause one or more processors to: receive a geometry bitstream, the geometry bitstream comprising encoded geometry data of point cloud data encoded according to a first encoding process; decode the encoded geometry data using a first decoding process that is opposite to the first encoding process to generate reconstructed geometry data; receive an attribute bitstream, the attribute bitstream comprising encoded attribute data of the point cloud data encoded according to a second encoding process; and decode the encoded attribute data using a second decoding process that is opposite to the second encoding process and based on the reconstructed geometry data to generate decoded attribute data and reconstructed geometry data of the point cloud data.

[0012] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a block diagram illustrating an example encoding and decoding system that may perform the techniques of this disclosure.

[0014] Figure 2 is a block diagram illustrating an example geometric point cloud compression (G-PCC) encoder.

[0015] Figure 3 is a block diagram illustrating an example G-PCC decoder.

[0016] Figure 4 is a conceptual diagram illustrating an example coding framework.

[0017] Figure 5 is a conceptual diagram illustrating an example decoding framework.

[0018] Figure 6 is a flow chart illustrating an example method of encoding point cloud data.

[0019] Figure 7 is a flow chart illustrating an example method of decoding point cloud data.

[0020] Figure 8 is a conceptual diagram illustrating an example ranging system that may be used with one or more techniques of this disclosure.

[0021] Fig. 9 is a conceptual diagram illustrating an example vehicle-based scenario in which one or more techniques of this disclosure may be used.

[0022] Fig.10 is a conceptual diagram illustrating an example extended reality system in which one or more techniques of this disclosure may be used.

[0023] Fig.11 is a conceptual diagram illustrating an example mobile device system in which one or more techniques of this disclosure may be employed. DETAILED DESCRIPTION

[0024] Point cloud (PC) is a 3D data representation used for tasks such as virtual reality (VR) and mixed reality (MR), autonomous driving, etc. A point cloud is a collection of points in 3D space, represented by 3D coordinates (x, y, z) called geometric data. Each point can also be associated with multiple attributes such as color, normal vector, and reflectivity. PCs can be classified into point cloud scenes and point cloud objects according to the target application and PC acquisition method. Point cloud scenes are usually captured using LiDAR sensors and are usually acquired dynamically. Point cloud objects can be further subdivided into static point clouds and dynamic point clouds. Static PCs are single objects, while dynamic PCs are time-varying PCs, where each instance of a dynamic PC is a static PC. Dynamic time-varying PCs are used in AR / VR, volumetric video streaming, and telepresence, and can be generated using 3D models (i.e. CGI) or captured from real-world scenes using various methods such as multiple cameras with depth sensors surrounding the object. These PCs are dense photorealistic point clouds that can have a large number of points, especially in high-precision or large-scale captures (millions of points per frame, up to 60 frames per second (FPS)). Therefore, efficient point cloud compression (PCC) is useful for achieving practical usage in VR and MR applications.

[0025] The Moving Picture Experts Group (MPEG) has approved two PCC (point cloud compression) standards: (1) S. Schwarz, M. Preda, V. Baroncini, M. Budagavi, P. Cesar, PAChou, R. A. Cohen, M. Krivoku´ca, S. Lasserre, Z. Li et al., “Emerging MPEG standards for point cloud compression,” IEEE Journal on Emerging and Selected TopIC in Circuits and Systems, vol. 9, no. 1, pp. 133–148, 2018, and (2) D. Graziosi, O. Nakagami, S. Kuma, A. Zaghetto, T. Suzuki, and A. Tabatabai, “An overview of ongoing point cloud compression standardization activities: Video-based (v-pcc) and geometry-based (g-pcc),” APSIPA Transactions on Signal and Information Processing, vol. 9, 2020. MPEG has approved the standard for Geometry-based Point Cloud Compression (G-PCC): “MPEG-PCC-TMC13: Geometry-based Point Cloud Compression G-PCC”, 2021. [Online] Available: https: / / github.com / MPEGGroup / mpeg-pcc-tmc13. MPEG has approved the standard for Video-based Point Cloud Compression (V-PCC): “MPEG-PCC-TMC2: Video-based Point Cloud Compression VPCC”, 2022. [Online] Available: https: / / github.com / MPEGGroup / mpeg-pcc-tmc2.

[0026] G-PCC includes octree geometry coding as a general geometry coding tool and predictive geometry coding (tree-based) tools targeting LiDAR-based point clouds. G-PCC is still developing methods based on triangle meshes or triangle soups to approximate the surface of 3D models. On the other hand, V-PCC encodes dynamic point clouds by projecting 3D points onto a 2D plane and then uses a video codec (e.g., High Efficiency Video Coding (HEVC)) to encode each frame over time. MPEG also proposed Common Test Conditions (CTC) to evaluate test models: S. Schwarz, G. Martin-Cocher, D. Flynn, and M. Budagavi, "Common test conditions for point cloud compression", document ISO / IEC JTC1 / SC29 / WG11W17766, Ljubljana, Slovenia, 2018.

[0027] Effective point cloud compression is useful for applications such as virtual and mixed reality, autonomous driving, and cultural heritage. Some techniques, such as m59617: Dynamic Point Cloud Geometry Compression using Sparse Convolutions by Anique Akhtar, Zhu Li, Geert Van der Auwera, Adarsh ​​Krishnan Ramasubramonian, Luong Pham Van, Marta Karczewicz; MPEG-137 Online Document m59617, April 2022, and m60307: [AI-3DGC][EE5.3 Test 2] Results dynamic point cloud compression by Anique Akhtar, Zhu Li, Geert Van der Auwera, Adarsh ​​Krishnan Ramasubramonian, Marta Karczewicz; MPEG-139 Online Document M60307, ​​July 2022, These techniques use deep learning based point cloud compression for dense dynamic point clouds using a deep learning network consisting of encoder and decoder modules. Deep learning based point cloud compression schemes may lack in some aspects. For example, deep learning based methods perform well in compressing the geometry of point clouds, but may lack in effectively compressing the attributes associated with these point clouds. Many deep learning based point cloud compression schemes may not be suitable for attribute compression.

[0028] In one or more examples, the present disclosure describes a flexible configuration of a deep learning-based framework, where instead of having a joint geometry and attribute compression scheme, the example techniques use an attribute recalculation process (such as a recoloring scheme (as an example)) to generate attributes of the reconstructed point cloud and adopt other (e.g., traditional) attribute compression schemes. As an example, one or more example techniques described in the present disclosure can utilize the recoloring scheme of G-PCC to obtain the attributes of the target point cloud from the source point cloud. The recoloring scheme of G-PCC adopts the weighted distance-based nearest neighbors in the source point cloud to calculate the attributes of the target point cloud. Therefore, in one or more examples, the example techniques can allow the use of attribute compression from different codecs and geometry compression from different codecs.

[0029] For example, a G-PCC encoder may receive point cloud data of a source point cloud (also referred to as an original point cloud). The point cloud data includes geometric data and attribute data. The G-PCC encoder may generate a bitstream of geometric data and attribute data of a target point cloud based on the geometric data and attribute data of the source point cloud. Typically, the target point cloud and the source point cloud may be similar, but there may be some differences, as described in more detail. As an example, the position of a point in a target point cloud may generally correspond to the position of a point in a source point cloud, but may be shifted relative to the position of a point in the source point cloud. In addition, the attribute data of a point in a target point cloud may be similar to the attribute data of a point in a source point cloud, but there may be some differences.

[0030] Thus, the target point cloud may not be exactly the same as the source point cloud, but the difference between the target point cloud and the source point cloud may be small enough so that there is very little or no impact on performance. In one or more examples, the G-PCC encoder may encode the geometric data of the point cloud data of the source point cloud according to a first encoding process to generate encoded geometric data of the target point cloud and a geometry bitstream for the encoded geometric data of the target point cloud.

[0031] Because the geometric data of the target point cloud and the geometric data of the source point cloud may not be exactly the same, there may be a separation between the attribute data of the points in the source point cloud and the points in the target point cloud. For example, in the source point cloud, each point may be associated with geometric data and attribute data. Since the geometric data in the target point cloud may be different (e.g., slightly different) from the geometric data in the source point cloud, it may not be clear which point in the target point cloud is associated with the attribute data of which point in the source point cloud.

[0032] To resolve this separation, the G-PCC encoder can be configured to perform an attribute recalculation process. During the attribute recalculation process, the G-PCC encoder can be configured to determine the attribute data of the points in the target point cloud based on the attribute data of the points in the source point cloud. As an example, the G-PCC encoder can be configured to determine multiple points in the source point cloud that are close to the points in the target point cloud. The G-PCC encoder can use the attribute data of the multiple points in the source point cloud to determine the attribute data of the points in the target point cloud.

[0033] The G-PCC encoder can generate recalculated reconstructed point cloud data of the target point cloud by using the attribute recalculation process of the attribute data of the point cloud data of the source point cloud. The G-PCC encoder can encode the recalculated reconstructed point cloud data according to the second encoding process to generate an attribute bitstream.

[0034] In this manner, the example techniques allow a G-PCC encoder to encode geometry data using techniques that are well-suited for encoding geometry data (e.g., a first encoding process), and to encode attribute data using techniques that are well-suited for encoding attribute data (e.g., a second encoding process). By using the attribute recalculation process, the G-PCC encoder can resolve the disassociation problem when encoding geometry data and attribute data separately. Thus, the example techniques can provide more efficient compression (e.g., requiring fewer bits) of geometry data and attribute data, thereby producing bandwidth-efficient geometry bitstreams and attribute bitstreams.

[0035] Figure 1 1 is a block diagram illustrating an example encoding and decoding system 100 that can perform the techniques of the present disclosure. The techniques of the present disclosure generally relate to encoding (encoding and / or decoding) point cloud data, i.e., supporting point cloud compression. Generally, point cloud data includes any data used to process point clouds. Encoding can effectively compress and / or decompress point cloud data.

[0036] like Figure 1 As shown, the system 100 includes a source device 102 and a target device 116. The source device 102 provides encoded point cloud data to be decoded by the target device 116. Figure 1 In an example of , source device 102 provides point cloud data to target device 116 via computer-readable medium 110. Source device 102 and target device 116 may include any of a variety of devices, including a desktop computer, a notebook (i.e., laptop) computer, a tablet computer, a set-top box, a telephone handset such as a smart phone, a television, a camera, a display device, a digital media player, a video game console, a video streaming device, a ground or sea vehicle, a spacecraft, an aircraft, a robot, a LIDAR device, a satellite, or the like. In some cases, source device 102 and target device 116 may be configured for wireless communication.

[0037] exist Figure 1In the example of , the source device 102 includes a data source 104, a memory 106, a G-PCC encoder 200, and an output interface 108. The target device 116 includes an input interface 122, a G-PCC decoder 300, a memory 120, and a data consumer 118. According to the present disclosure, the G-PCC encoder 200 of the source device 102 and the G-PCC decoder 300 of the target device 116 can be configured to apply the technology of the present disclosure related to attribute encoding for point cloud compression. Therefore, the source device 102 represents an example of an encoding device, and the target device 116 represents an example of a decoding device. In other examples, the source device 102 and the target device 116 may include other components or arrangements. For example, the source device 102 can receive data (e.g., point cloud data) from an internal or external source. Similarly, the target device 116 can interface with an external data consumer instead of including the data consumer in the same device.

[0038] like Figure 1 The system 100 shown is only an example. In general, other digital encoding and / or decoding devices may perform the techniques of the present disclosure related to attribute encoding for point cloud compression. The source device 102 and the target device 116 are only examples of such devices, wherein the source device 102 generates encoded data for transmission to the target device 116. The present disclosure refers to a "coding" device as a device that performs data encoding (encoding and / or decoding). Therefore, the G-PCC encoder 200 and the G-PCC decoder 300 represent examples of encoding devices, in particular encoders and decoders, respectively. In some examples, the source device 102 and the target device 116 can operate in a substantially symmetrical manner, so that each of the source device 102 and the target device 116 includes encoding and decoding components. Therefore, the system 100 can support one-way or two-way transmission between the source device 102 and the target device 116, such as for streaming, playback, broadcasting, telephone, navigation and other applications.

[0039] In general, the data source 104 represents a data source (i.e., raw, unencoded point cloud data) and can provide a series of continuous "frames" of data to the G-PCC encoder 200, which encodes the data of the frames. The data source 104 of the source device 102 may include a point cloud capture device, such as any of a variety of cameras or sensors, such as a 3D scanner or a light detection and ranging (LIDAR) device, one or more cameras, an archive containing previously captured data, and / or a data feed interface for receiving data from a data content provider. Additionally or alternatively, the point cloud data may be generated by a computer from a scanner, camera, sensor, or other data. For example, the data source 104 may generate computer graphics-based data as source data, or generate a combination of real-time data, archived data, and computer-generated data. In each case, the G-PCC encoder 200 encodes captured, pre-captured, or computer-generated data. The G-PCC encoder 200 may rearrange the frames from the order in which they are received (sometimes referred to as "display order") into an encoding order for encoding. G-PCC encoder 200 may generate one or more bitstreams including encoded data. Source device 102 may then output the encoded data onto computer-readable medium 110 via output interface 108 for receipt and / or retrieval by input interface 122 of target device 116, for example.

[0040] The memory 106 of the source device 102 and the memory 120 of the target device 116 may represent a general memory. In some examples, the memory 106 and the memory 120 may store original data, for example, original data from the data source 104 and original decoded data from the G-PCC decoder 300. Additionally or alternatively, the memory 106 and the memory 120 may store software instructions that can be executed by, for example, the G-PCC encoder 200 and the G-PCC decoder 300, respectively. Although the memory 106 and the memory 120 are shown separately from the G-PCC encoder 200 and the G-PCC decoder 300 in this example, it should be understood that the G-PCC encoder 200 and the G-PCC decoder 300 may also include an internal memory for a functionally similar or equivalent purpose. In addition, the memory 106 and the memory 120 may store coded data, for example, coded data output from the G-PCC encoder 200 and input to the G-PCC decoder 300. In some examples, portions of memory 106 and memory 120 may be allocated as one or more buffers, for example, to store raw, decoded, and / or encoded data. For example, memory 106 and memory 120 may store data representing a point cloud.

[0041] The computer-readable medium 110 may represent any type of medium or device capable of transmitting encoded data from the source device 102 to the target device 116. In one example, the computer-readable medium 110 represents a communication medium that enables the source device 102 to transmit the encoded data directly to the target device 116 in real time, for example, via a radio frequency network or a computer-based network. According to a communication standard (e.g., a wireless communication protocol), the output interface 108 may modulate a transmission signal containing the encoded data, and the input interface 122 may demodulate the received transmission signal. The communication medium may include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network (e.g., a local area network, a wide area network, or a global network such as the Internet). The communication medium may include a router, a switch, a base station, or any other device that can be used to facilitate communication from the source device 102 to the target device 116.

[0042] In some examples, source device 102 may output the encoded data from output interface 108 to storage device 112. Similarly, target device 116 may access the encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded data.

[0043] In some examples, source device 102 may output the encoded data to file server 114 or another intermediate storage device that may store the encoded data generated by source device 102. Target device 116 may access the stored data from file server 114 via streaming or downloading. File server 114 may be any type of server device capable of storing encoded data and sending the encoded data to target device 116. File server 114 may represent a network server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Target device 116 may access the encoded data from file server 114 via any standard data connection (including an Internet connection). This may include a wireless channel (e.g., a Wi-Fi connection) suitable for accessing the encoded data stored on file server 114, a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination of both. File server 114 and input interface 122 may be configured to operate according to a streaming protocol, a downloading transmission protocol, or a combination thereof.

[0044] Output interface 108 and input interface 122 may represent a wireless transmitter / receiver, a modem, a wired networking component (e.g., an Ethernet card), a wireless communication component operating according to any of the various IEEE 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 include wireless components, output interface 108 and input interface 122 may be configured to transmit data (e.g., encoded data) according to a cellular communication standard (e.g., 4G, 4G-LTE (Long Term Evolution), LTE Advanced, 5G, or the like). In some examples where output interface 108 includes a wireless transmitter, output interface 108 and input interface 122 may be configured to transmit data (e.g., encoded data) according to other wireless standards (e.g., IEEE 802.11 specifications, IEEE 802.15 specifications (e.g., ZigBee™), Bluetooth™ standards, or the like). In some examples, source device 102 and / or target device 116 may include corresponding system-on-chip (SoC) devices. For example, source device 102 may include a SoC device for performing the functions attributed to G-PCC encoder 200 and / or output interface 108, and target device 116 may include a SoC device for performing the functions attributed to G-PCC decoder 300 and / or input interface 122.

[0045] The techniques of this disclosure may be applied to encoding and decoding to support any of a variety of applications, such as communications between autonomous vehicles, communications between scanners, cameras, sensors and processing devices (such as local or remote servers), geographic mapping, or other applications.

[0046] The input interface 122 of the target device 116 receives a coded bitstream from a computer-readable medium 110 (e.g., a communication medium, a storage device 112, a file server 114, or the like). The coded bitstream may include signaling information defined by the G-PCC encoder 200, which is also used by the G-PCC decoder 300, such as syntax elements with values ​​describing characteristics and / or processing of coding units (e.g., slices, pictures, groups of pictures, sequences, etc.). The data consumer 118 uses the decoded data. For example, the data consumer 118 may use the decoded data to determine the location of a physical object. In some examples, the data consumer 118 may include a display for presenting an image based on a point cloud.

[0047] The G-PCC encoder 200 and the G-PCC decoder 300 can each be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is partially implemented in software, the device may store instructions for the software in a suitable non-transitory computer-readable medium, and use one or more processors to execute the instructions in hardware to perform the technology of the present disclosure. Each of the G-PCC encoder 200 and the G-PCC decoder 300 can be included in one or more encoders or decoders, and any of the encoders or decoders can be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device. The device including the G-PCC encoder 200 and / or the G-PCC decoder 300 may include one or more integrated circuits, microprocessors, and / or other types of devices.

[0048] The G-PCC encoder 200 and the G-PCC decoder 300 may operate according to a coding standard such as the Video Point Cloud Compression (V-PCC) standard or the Geometric Point Cloud Compression (G-PCC) standard. The present disclosure may generally refer to the coding (e.g., encoding and decoding) of a picture to include the process of encoding or decoding data. The coded bitstream typically includes a series of values ​​of syntax elements that represent coding decisions (e.g., coding modes).

[0049] The present disclosure may generally refer to "signaling" certain information, such as syntax elements. The term "signaling" may generally refer to the communication of values ​​of syntax elements and / or other data used to decode encoded data. That is, the G-PCC encoder 200 may signal the values ​​of syntax elements in a bitstream. Generally speaking, signaling refers to producing values ​​in a bitstream. As mentioned above, the source device 102 may transmit the bitstream to the target device 116 substantially in real time or non-real time, such as may occur when storing syntax elements to the storage device 112 for later retrieval by the target device 116.

[0050] ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) is investigating the potential need for standardization of point cloud coding techniques whose compression capabilities significantly exceed those of current methods, with the goal of creating such a standard. The group is working on this exploratory activity in a collaborative effort called the 3-Dimensional Graphics Group (3DG) to evaluate compression technology designs proposed by their experts in the field.

[0051] Point cloud compression activities are categorized in two different approaches. The first approach is “Video Point Cloud Compression” (V-PCC), which segments 3D objects and projects the segments in multiple 2D planes (which are represented as “patches” in 2D frames), which are further encoded by a traditional 2D video codec such as the High Efficiency Video Coding (HEVC) (ITU-T H.265) codec. The second approach is “Geometry-based Point Cloud Compression” (G-PCC), which directly compresses 3D geometry data, i.e., the positions of a collection of points in 3D space, and the associated attribute values ​​(for each point associated with the 3D geometry). G-PCC addresses the compression of point clouds in category 1 (static point clouds) and category 3 (dynamically acquired point clouds). The most recent draft of the G-PCC standard is available in G-PCC DIS, ISO / IEC JTC1 / SC29 / WG11W19088, Brussels, Belgium, January 2020, and a description of the codec is available in G-PCC Codec Description v6, ISO / IEC JTC1 / SC29 / WG11W19091, Brussels, Belgium, January 2020.

[0052] A point cloud contains a collection of points in 3D space and may have attributes associated with the point. The attributes may be color information such as R, G, B or Y, Cb, Cr or reflectivity information or other attributes. Point clouds may be captured by various cameras or sensors (such as LIDAR sensors and 3D scanners) and may also be computer generated. Point cloud data is used in a variety of applications including, but not limited to, architecture (modeling), graphics (3D models for visualization and animation), and the automotive industry (LIDAR sensors to aid navigation).

[0053] The 3D space occupied by the point cloud data can be surrounded by a virtual bounding box. The position of the points in the bounding box can be represented by a specific precision; therefore, the position of one or more points can be quantified based on the precision. At the smallest level, the bounding box is divided into voxels, which are the smallest spatial units represented by unit cubes. Voxels in a bounding box can be associated with zero, one, or more than one point. The bounding box can be divided into multiple cube / cuboid areas, which can be referred to as tiles. Each tile can be encoded into one or more slices. The division of the bounding box into slices and tiles can be based on the number of points in each partition, or based on other considerations (for example, a particular area can be encoded as a tile). The slice area can be further divided using a split decision similar to the split decision in a video codec.

[0054] Figure 2 An overview of the G-PCC encoder 200 is provided. Figure 3An overview of a G-PCC decoder 300 is provided. The modules shown are logical and do not necessarily correspond one-to-one with implementation code in a reference implementation of the G-PCC codec, namely the TMC13 test model software studied by ISO / IEC MPEG (JTC 1 / SC29 / WG 11).

[0055] In both the G-PCC encoder 200 and the G-PCC decoder 300, the point cloud positions are first encoded. The attribute encoding depends on the decoded geometry data. Figure 2 and Figure 3 In the , the gray shaded modules are options that are commonly used for Category 1 data. The diagonal cross hatched modules are options that are commonly used for Category 3 data. All other modules are common between Category 1 and Category 3.

[0056] For category 3 data, the compressed geometry data is typically represented as an octree from the root down to the leaf level of a single voxel. For category 1 data, the compressed geometry data is typically represented by a pruned octree (i.e., an octree from the root down to the leaf level of blocks larger than a voxel) plus a model that approximates the surface within each leaf of the pruned octree. In this way, category 1 and category 3 data share the octree encoding mechanism, while category 1 data may additionally approximate the voxels within each leaf with a surface model. The surface model used is a triangulation consisting of 1-10 triangles per block, resulting in a triangle soup. Therefore, category 1 geometry codecs are called trisoup geometry codecs, while category 3 geometry codecs are called octree geometry codecs.

[0057] At each node of the octree, occupancy is signaled (when not inferred) for one or more of its children (up to eight nodes). Multiple neighborhoods are specified, including (a) nodes that share faces with the current octree node, (b) nodes that share faces, edges, or vertices with the current octree node, etc. Within each neighborhood, the occupancy of the node and / or its children can be used to predict the occupancy of the current node or its children. For points that are sparsely populated in some nodes of the octree, the codec also supports a direct encoding mode in which the 3D position of the point is encoded directly. A flag can be signaled to indicate that direct mode is signaled. At the lowest level, the number of points associated with the octree node / leaf node can also be encoded.

[0058] Once the geometry data is encoded, the attributes corresponding to the geometry points are encoded. When there are multiple attribute points corresponding to one reconstructed / decoded geometry point, the attribute values ​​representing the reconstructed points can be derived.

[0059] There are three attribute coding methods in G-PCC: Region Adaptive Hierarchical Transform (RAHT) coding, interpolation-based hierarchical nearest neighbor prediction (prediction transform), and interpolation-based hierarchical nearest neighbor prediction with an update / lifting step (lifting transform). RAHT and lifting are typically used for category 1 data, while prediction is typically used for category 3 data. However, either method can be used for any data, and just like the geometry codec in G-PCC, the attribute coding method used to encode the point cloud is specified in the bitstream.

[0060] The encoding of attributes can be done in levels of detail (LODs), where with each LOD a finer representation of the point cloud attributes can be obtained. Each LOD can be specified based on a distance metric to neighboring nodes or based on a sampling distance.

[0061] At the G-PCC encoder 200, the residual obtained as an output of the attribute encoding method is quantized. The residual can be obtained by subtracting the attribute value from a prediction derived based on points in the neighborhood of the current point and based on the attribute value of previously encoded points. The quantized residual can be encoded using context adaptive arithmetic coding.

[0062] exist Figure 2 In the example, the G-PCC encoder 200 may include a coordinate transformation unit 202, a color transformation unit 204, a voxelization unit 206, an attribute transfer unit 208, an octree analysis unit 210, a surface approximation analysis unit 212, an arithmetic coding unit 214, a geometric reconstruction unit 216, a RAHT unit 218, an LOD generation unit 220, a lifting unit 222, a coefficient quantization unit 224 and an arithmetic coding unit 226.

[0063] like Figure 2 As shown in the example of , the G-PCC encoder 200 can obtain a set of positions and a set of attributes of points in the point cloud. The G-PCC encoder 200 can obtain the position set and the attribute set of points in the point cloud from the data source 104 ( Figure 1 ) obtains a set of positions and a set of attributes of points in a point cloud. The positions may include the coordinates of the points in the point cloud. The attributes may include information about the points in the point cloud, such as the color associated with the points in the point cloud. The G-PCC encoder 200 may generate a geometry bitstream 203 including an encoded representation of the positions of the points in the point cloud. The G-PCC encoder 200 may also generate an attribute bitstream 205 including an encoded representation of the attribute set.

[0064] The coordinate transformation unit 202 may apply a transformation to the coordinates of the point to transform the coordinates from the initial domain to the transformed domain. The present disclosure may refer to the transformed coordinates as transformed coordinates. The color transformation unit 204 may apply a transformation to transform the color information of the attribute to a different domain. For example, the color transformation unit 204 may transform the color information from the RGB color space to the YCbCr color space.

[0065] In addition, Figure 2 In the example of , the voxelization unit 206 may voxelize the transformed coordinates. Voxelization of the transformed coordinates may include quantizing and removing some points of the point cloud. In other words, multiple points of the point cloud may be contained within a single "voxel", which may then be considered as one point in some respects. In addition, the octree analysis unit 210 may generate an octree based on the voxelized transformed coordinates. In addition, in Figure 2 In the example of FIG. 1 , the surface approximation analysis unit 212 may analyze the points to potentially determine a surface representation of the set of points. The arithmetic coding unit 214 may entropy encode syntax elements representing information of the octree and / or surface determined by the surface approximation analysis unit 212. The G-PCC encoder 200 may output these syntax elements in a geometry bitstream 203. The geometry bitstream 203 may also include other syntax elements, including syntax elements that are not arithmetically coded.

[0066] The geometric reconstruction unit 216 may reconstruct the transformed coordinates of the points in the point cloud based on the octree, the data indicating the surface determined by the surface approximation analysis unit 212, and / or other information. Due to voxelization and surface approximation, the number of transformed coordinates reconstructed by the geometric reconstruction unit 216 may be different from the original number of points of the point cloud. The present disclosure may refer to the resulting points as reconstructed points. The attribute transmission unit 208 may transfer the attributes of the original points of the point cloud to the reconstructed points of the point cloud.

[0067] In addition, the RAHT unit 218 may apply RAHT encoding to the attributes of the reconstruction points. In some examples, under RAHT, the attributes of the block of 2×2×2 point positions are taken and transformed in one direction to obtain four low (L) and four high (H) frequency nodes. Subsequently, the four low frequency nodes (L) are transformed in the second direction to obtain two low (LL) and two high (LH) frequency nodes. The two low frequency nodes (LL) are transformed along the third direction to obtain one low frequency (LLL) and one high frequency (LLH) node. The low frequency node LLL corresponds to the DC coefficient, and the high frequency nodes H, LH and LLH correspond to the AC coefficient. The transform in each direction may be a 1-D transform with two coefficient weights. The low frequency coefficients may be regarded as coefficients of the 2×2×2 block for the next higher level RAHT transform, and the AC coefficients are encoded without changes; this transform continues until the top root node. The tree traversal for encoding is used from top to bottom to calculate the weights to be used for the coefficients; the transform order is from bottom to top. The coefficients may then be quantized and encoded.

[0068] Alternatively or additionally, the LOD generation unit 220 and the lifting unit 222 may apply LOD processing and lifting to the attributes of the reconstructed points, respectively. LOD generation is used to divide the attributes into different refinement levels. Each refinement level provides a refinement of the attributes of the point cloud. The first refinement level provides a rough approximation and contains few points; subsequent refinement levels typically contain more points, and so on. The refinement level can be constructed using a distance-based metric, or one or more other classification criteria (e.g., subsampling from a specific order) can also be used. Therefore, all reconstructed points can be included in one refinement level. Each level of detail is generated by merging all points until a specific refinement level: for example, LOD1 is obtained based on refinement level RL1, LOD2 is obtained based on RL1 and RL2, ... LODN is obtained by merging RL1, RL2, ... RLN. In some cases, LOD generation may be followed by a prediction scheme (e.g., a predictive transform), in which the attributes associated with each point in the LOD are predicted based on a weighted average of previous points, and the residual is quantized and entropy encoded. The lifting scheme is built on top of the predictive transform mechanism, where an update operator is used to update the coefficients and perform adaptive quantization of the coefficients.

[0069] The RAHT unit 218 and the lifting unit 222 may generate coefficients based on the attributes. The coefficient quantization unit 224 may quantize the coefficients generated by the RAHT unit 218 or the lifting unit 222. The arithmetic coding unit 226 may apply arithmetic coding to the syntax elements representing the quantized coefficients. The G-PCC encoder 200 may output these syntax elements in the attribute bitstream 205. The attribute bitstream 205 may also include other syntax elements, including non-arithmetic coded syntax elements.

[0070] exist Figure 3 In the example, the G-PCC decoder 300 may include a geometric arithmetic decoding unit 302, an attribute arithmetic decoding unit 304, an octree synthesis unit 306, an inverse quantization unit 308, a surface approximation synthesis unit 310, a geometric reconstruction unit 312, a RAHT unit 314, an LOD generation unit 316, an inverse lifting unit 318, an inverse transform coordinate unit 320 and an inverse transform color unit 322.

[0071] The G-PCC decoder 300 may obtain the geometry bitstream 203 and the attribute bitstream 205. The geometry arithmetic decoding unit 302 of the decoder 300 may apply arithmetic decoding (e.g., context adaptive binary arithmetic coding (CABAC) or other types of arithmetic decoding) to the syntax elements in the geometry bitstream 203. Similarly, the attribute arithmetic decoding unit 304 may apply arithmetic decoding to the syntax elements in the attribute bitstream 205.

[0072] The octree synthesis unit 306 can synthesize an octree based on the syntax elements parsed from the geometry bitstream 203. Starting from the root node of the octree, the occupancy of each of the eight child nodes at each octree level is signaled in the bitstream. When the signaling indicates that a child node at a specific octree level is occupied, the occupancy of the child nodes of the child node is signaled. Before proceeding to the subsequent octree level, the signaling of the nodes at each octree level is signaled. At the final level of the octree, each node corresponds to a voxel position; when a leaf node is occupied, one or more points can be specified to be occupied at the voxel position. In some cases, due to quantization, some branches of the octree may terminate earlier than the final level. In this case, the leaf node is considered to be an occupied node without a child node. In the case of using surface approximation in the geometry bitstream 203, the surface approximation synthesis unit 310 can determine the surface model based on the syntax elements parsed from the geometry bitstream 203 and based on the octree.

[0073] In addition, the geometric reconstruction unit 312 may perform reconstruction to determine the coordinates of the points in the point cloud. For each position at a leaf node of the octree, the geometric reconstruction unit 312 may reconstruct the node position by using the binary representation of the leaf node in the octree. At each corresponding leaf node, the number of points at the corresponding leaf node is signaled; this indicates the number of repeated points at the same voxel position. When geometric quantization is used, the point position is scaled to determine the reconstructed point position value.

[0074] The inverse transform coordinate unit 320 can apply an inverse transform to the reconstructed coordinates to convert the reconstructed coordinates (positions) of the points in the point cloud from the transform domain back to the original domain. The positions of the points in the point cloud can be in the floating point domain, but the point positions in the G-PCC codec are encoded in the integer domain. The inverse transform can be used to convert the positions back to the original domain.

[0075] In addition, Figure 3 In the example of , the inverse quantization unit 308 may inverse quantize the property value. The property value may be based on syntax elements obtained from the property bitstream 205 (eg, including syntax elements decoded by the property arithmetic decoding unit 304).

[0076] Depending on how the attribute value is encoded, the RAHT unit 314 may perform RAHT encoding to determine the color value of the point of the point cloud based on the inverse quantized attribute value. RAHT decoding is performed from the top to the bottom of the tree. At each level, the low-frequency coefficients and high-frequency coefficients derived from the inverse quantization process are used to derive the component values. At the leaf node, the derived value corresponds to the attribute value of the coefficient. The weight derivation process for the point is similar to the process used at the G-PCC encoder 200. Alternatively, the LOD generation unit 316 and the inverse lifting unit 318 may use a detail level-based technique to determine the color value of the point of the point cloud. The LOD generation unit 316 decodes each LOD, giving a gradually finer representation of the attribute of the point. Using the prediction transform, the LOD generation unit 316 derives the prediction of the point from the weighted sum of the points previously reconstructed in the same LOD or in the previous LOD. The LOD generation unit 316 may add the prediction to the residual (which is obtained after inverse quantization) to obtain the reconstructed value of the attribute. When using the lifting scheme, the LOD generation unit 316 may also include an update operator to update the coefficients used to derive the attribute value. In this case, the LOD generation unit 316 may also apply inverse adaptive quantization.

[0077] In addition, Figure 3 In the example of , the inverse color transform unit 322 may apply an inverse color transform to the color value. The inverse color transform may be the inverse of the color transform applied by the color transform unit 204 of the encoder 200. For example, the color transform unit 204 may transform the color information from the RGB color space to the YCbCr color space. Thus, the inverse color transform unit 322 may transform the color information from the YCbCr color space to the RGB color space.

[0078] Figure 2 and Figure 3Various units are illustrated to help understand the operations performed by the encoder 200 and the decoder 300. The units may be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide specific functions and are preset on executable operations. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functions in the operations that can be performed. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by instructions of the software or firmware. Fixed-function circuits can execute software instructions (e.g., receive parameters or output parameters), but the type of operation performed by the fixed-function circuit is generally immutable. In some examples, one or more of the units may be different circuit blocks (fixed function or programmable), and in some examples, one or more of the units may be integrated circuits.

[0079] In one or more examples, machine learning (such as deep learning) techniques can be used for encoding and decoding of point cloud data. Some examples of such machine learning techniques are described below. A deep learning-based lossy point cloud geometry compression scheme for dynamic point cloud compression is possible. The lossy geometry scheme uses the previous frame to predict the potential representation of the current frame by employing a prediction network. The example technique performs P-frame inter-frame point cloud encoding, where the current frame is encoded with the help of the previously decoded frame. The architecture is implemented using a sparse convolutional neural network (CNN) with a sparse tensor. The example architecture employs convolution of the target coordinates to map the potential representation of the previous frame to the downsampled coordinates of the current frame to predict the feature embedding of the current frame. The encoder sends the residuals of the predicted features and the actual features by compressing the residuals of the predicted features and the actual features using a learned probabilistic decomposition entropy model. Compared with G-PCC and V-PCC, the machine learning technique shows better compression performance on dense point clouds with efficient encoding / decoding runtime.

[0080] However, using machine learning to encode / decode (e.g., compress / decompress) point cloud data can be problematic. Typically, a point cloud compression codec can perform well on geometry encoding but poorly on attribute encoding, or vice versa. This can be true for deep learning-based point cloud compression schemes, where deep learning-based point cloud compression schemes typically perform well on geometry compression but poorly or suboptimally on attribute compression.

[0081] However, in a lossy compression scheme, the geometric data of the reconstructed point cloud (also referred to as the target point cloud) is different from that of the original point cloud (also referred to as the source point cloud). This means that attribute compression is often combined with geometric compression to have a correspondence (e.g., association) between the geometric data and its corresponding attributes. That is, in order to maintain the correspondence (e.g., association) between the geometric data and its corresponding attributes, compression of the point cloud data is performed jointly with respect to the geometric data and the attribute data of the point cloud data. However, in a lossy compression scheme, the geometry encoder changes the positions of the geometric points, and thus the correspondence between the reconstructed geometric data and the reconstructed attributes is lost. Therefore, the geometry data and the attribute data are often compressed together in the same codec.

[0082] In one or more examples described in the present disclosure, in order to achieve coding efficiency (such as coding efficiency of deep learning), as a non-limiting example, it may be beneficial to be able to separate geometry compression and attribute compression from the compression framework and then be able to combine the geometry compression method from one codec with the attribute compression method of another codec to be able to obtain better compression performance. In one or more examples, the present disclosure describes a flexible configuration in a compression framework, where the G-PCC encoder 200 and the G-PCC decoder 300 employ deep learning-based geometry compression and attribute compression methods for point cloud compression.

[0083] The present disclosure describes a framework for using a geometry encoder / decoder from one codec and an attribute encoder / decoder from a separate codec to create a complete codec that will outperform each individual codec. Performing in lossy point cloud compression can be challenging because the geometry data changes after compression, and attaching attributes to their corresponding geometry data is challenging. In some examples, the G-PCC encoder 200 and the G-PCC decoder 300 can utilize a first encoding / decoding process for geometry data (e.g., deep learning-based point cloud geometry compression) and a second encoding / decoding process for attribute data (e.g., G-PCC attribute compression), respectively. Figure 4 and Figure 5 shown.

[0084] Figure 4 is a conceptual diagram illustrating an example coding framework. Figure 5is a conceptual diagram illustrating an example decoding framework. In some examples, the G-PCC encoder 200 uses a deep learning-based encoder-decoder to compress geometric data, and the G-PCC decoder 300 uses a deep learning-based encoder-decoder to decompress geometric data, and other techniques (e.g., the RAHT part of G-PCC) may be used for attributes to obtain a reconstructed point cloud, also referred to as a target point cloud. The reconstructed point cloud (i.e., the target point cloud) differs from the original point cloud (also referred to as the source point cloud) in geometric data and may not simply use the attributes of the original point cloud.

[0085] The target point cloud or the reconstructed point cloud can be a data structure in which geometric data and attribute data are associated with each other. For example, in the present disclosure, a point in the target point cloud can refer to a data structure in which geometric data and attribute data are associated with each other. As an example, the point can be an index to a data structure. The example technology may not need to generate the target point cloud completely, such as for display at the G-PCC encoder 200. However, a target point cloud for display or other purposes can be generated at the G-PCC encoder 200. The target point cloud can be regenerated at the G-PCC decoder 300 for performing operations, including possible display.

[0086] In some examples, the G-PCC encoder 200 and the G-PCC decoder 300 may perform an attribute recalculation process, such as a recoloring scheme of G-PCC (as an example), to change the attributes of the original point cloud (e.g., attribute data of the source point cloud) to create newer attributes for the reconstructed point cloud (e.g., attribute data of the points of the target point cloud). The recolored point cloud (e.g., the target point cloud with the attributes from the attribute recalculation process) has the geometric data of the reconstructed point cloud (e.g., the target point cloud) and the attributes derived from the original point cloud (e.g., the source point cloud). In some examples, the recolored point cloud may be encoded by the G-PCC 200, wherein the geometric data is encoded in a lossless manner and the attributes are encoded in a lossy manner. The reconstructed geometric data is combined with the reconstructed attributes to obtain a reconstructed point cloud.

[0087] For example, Figure 4 As shown, the processing circuit including the G-PCC encoder 200 includes a geometry encoder 404, a geometry decoder 412, an attribute recalculation unit 416, and an attribute encoder 420. One or more memories (e.g., the memory 106 or other local memory) coupled to the processing circuit including the G-PCC encoder 200 can store the source point cloud 400, also referred to as the original point cloud.

[0088] In one or more examples, the geometry encoder 404 can be part of the arithmetic coding unit 214, and the attribute encoder 420 can be part of the arithmetic coding unit 226. However, the techniques are not limited in this regard.

[0089] As an example, the geometry data encoder 404 and the attribute encoder 420 are shown as separate units, but may be combined. For example, in one or more examples, the processing circuitry including the G-PCC encoder 200 may be considered to be an encoder configured to perform a first encoding process and a second encoding process.

[0090] The geometry encoder 404 and the attribute encoder 420 may also be separate units. For example, in one or more examples, the processing circuit including the G-PCC encoder 200 may include a first encoder (e.g., geometry encoder 404) configured to perform a first encoding process and a second encoder (e.g., attribute encoder 420) configured to perform a second encoding process. The first encoder (e.g., geometry encoder 404) may be a machine learning-based encoder, and the second encoder (e.g., attribute encoder 420) may be a non-machine learning-based encoder. The first encoder (e.g., geometry encoder 404) may be a machine learning-based encoder, and the second encoder (e.g., attribute encoder 420) may be a machine learning-based encoder.

[0091] The geometry encoder 404 receives geometry data 402 of point cloud data of the source point cloud 400 for a first encoding process, and encodes the geometry data 402 according to the first encoding to generate encoded geometry data 410 of the target point cloud. In addition, the geometry encoder 404 generates a geometry bitstream 406. The geometry bitstream 406 may include syntax elements and coded values ​​that the G-PCC decoder 300 may parse to generate geometry data of the target point cloud. In some examples, the geometry bitstream 406 may include the encoded geometry data 410 of the target point cloud.

[0092] In one or more examples, the geometry encoder 404 may be a deep learning based geometry encoder (e.g., a neural network based geometry encoder). For example, the geometry encoder 404 may be a learning geometry encoder model executed by a processing circuit including the G-PCC encoder 200. As an example, the learning geometry encoder model may be generated based on a machine learning algorithm, examples of which include an artificial neural network (ANN), a deep neural network (DNN), a graph neural network (GNN), a random forest (RF), a kernel method, and the like. The NPU may sometimes be referred to alternatively as a neural signal processor (NSP), a tensor processing unit (TPU), a neural network processor (NNP), an intelligent processing unit (IPU), or a visual processing unit (VPU).

[0093] The trained geometry encoder model can be configured to generate a geometry data predictor, such as based on previous geometry data. In one or more examples, the geometry encoder model can be trained in a server and device other than the source device 102. For example, during training, the geometry encoder model can be fed with geometry data of a point cloud, and the geometry data predictor can be trained based on the fed geometry data as a predictor of future point clouds. During training, the server can adjust the weights and biases of the neural network of the geometry encoder model to train the geometry encoder model based on a loss function indicating the difference between the geometry data prediction values ​​output by the geometry encoder model and the training geometry data prediction values. The server can then output the trained geometry encoder model to the G-PCC encoder 200.

[0094] For purposes of example, the geometry encoder 404 is described as a deep learning based geometry encoder (eg, a neural network based geometry encoder). The geometry encoder 404 may be configured to encode geometry data using other techniques, including non-machine learning techniques.

[0095] The geometry decoder 412 may decode the encoded geometry data to generate geometry data 414 for reconstruction of the target point cloud. In general, the geometry data decoder 412 may be configured to perform the inverse process of the geometry data encoder 404.

[0096] In one or more examples, the reconstructed geometric data 414 of the target point cloud may be different from the geometric data 402 of the source point cloud 400. For example, assume that the geometric data 402 of the source point cloud 400 includes Cartesian coordinates of 100 points, represented as (x1, y1, z1) to (x100, y100, z100). After the encoding process of the geometric data 402 by the geometry encoder 404 and the decoding process of the encoded geometric data 410 by the geometry decoder 412, the resulting reconstructed geometric data 414 may include Cartesian coordinates of the 100 points, but the coordinates may not be (x1, y1, z1) to (x100, y100, z100). Instead, the coordinates may be (x1', y1', z1') to (x100', y100' and z100'), where x1 and x1' may not necessarily be equal, y1 and y1' may not necessarily be equal, z1 and z1' may not necessarily be equal, and so on. One or all of (x1', y1', z1') may be equal to the corresponding one of (x1, y1, z1), but (x1', y1', z1') is not necessarily equal to the corresponding one of (x1, y1, z1) in all cases. Therefore, the geometric data of the source point cloud 400 and the geometric data of the target point cloud (e.g., the reconstructed geometric data 414) may be different.

[0097] The attribute recalculation unit 416 may receive the reconstructed geometry data 414 of the target point cloud and the attribute data 408 of the source point cloud 400 . In some examples, the attribute data 408 of the source point cloud 400 may be the geometry and attribute data of the source point cloud 400 .

[0098] Attribute data 408 may be associated with (e.g., correspond to) geometric data 402. For example, a point in source point cloud 400 may have geometric data (e.g., Cartesian coordinates) and attribute data (e.g., color and reflectivity). In this example, the attribute data of a point in source point cloud 400 and the geometric data of a point in source point cloud 400 may be associated.

[0099] As described above, the reconstructed geometric data 414 of the target point cloud may be different from the geometric data 402 of the source point cloud 400. Therefore, the attribute data 408 may be disassociated from the reconstructed geometric data 414. The attribute recalculation unit 416 may be configured to resolve the disassociation.

[0100] The attribute recalculation unit 416 may be configured to perform an attribute recalculation process on the attribute data 408 of the point cloud data of the source point cloud 400 based on the reconstruction geometry data 414 to generate recalculated reconstructed point cloud data 418 of the target point cloud. As an example, the attribute recalculation unit 416 may access the attribute data of the points in the source point cloud 400 to determine the attribute data of the points in the target point cloud.

[0101] For example, the attribute recalculation unit 416 may receive the reconstructed geometry data 414 of the first point of the target point cloud. The attribute recalculation unit 416 may determine a plurality of points in the point cloud data of the source point cloud 400 that are close to the first point of the target point cloud based on the reconstructed geometry data 414. For example, the reconstructed geometry data 414 of the first point of the target point cloud may include coordinates such as (x1', y1', z1'). The attribute recalculation unit 416 may determine a plurality of points in the source point cloud 400 that are close to the first point based on the coordinates of the plurality of points. For example, the attribute recalculation unit 416 may determine a distance between the first point of the target point cloud and a point in the source point cloud 400 based on the coordinates of the first point in the target point cloud and the point in the source point cloud 400. Based on the distance, the attribute recalculation unit 416 may determine a close point in the source point cloud 400 that is close to the first point in the target point cloud.

[0102] The attribute recalculation unit 416 may determine attribute data for a plurality of points in the point cloud data of the source point cloud 400. As an example, the attribute recalculation unit 416 may determine color and reflectance values. The attribute recalculation unit 416 may determine attribute data for a first point in the target point cloud based on the attribute data for the plurality of points in the source point cloud 400. For example, the attribute recalculation unit 416 may determine a weighted average of the attribute data for the points in the source point cloud 400 (e.g., points closer to the first point are weighted more than points farther from the first point). The result may be recalculated reconstructed point cloud data 418 for the first point in the target point cloud.

[0103] The attribute recalculation unit 416 may repeat these operations for other points in the target point cloud. In some examples, if the coordinates of a first point in the target point cloud are the same as the coordinates of a point in the source point cloud 400, the attribute recalculation unit 416 may determine the attribute data of the first point in the target point cloud based on (e.g., only based on) the attribute data of the points with the same coordinates in the source point cloud 400. For example, the attribute recalculation unit 416 may assign the attribute data of the points in the source point cloud 400 with the same coordinates as the first point in the target point cloud to be equal to the attribute data of the first point in the target point cloud.

[0104] The above example techniques may be considered examples of recoloring. For example, the attribute recalculation unit 416 may be configured to perform a recoloring process on the attribute data 408. The recalculated reconstructed point cloud data 418 may include recolored reconstructed point cloud data. In some examples, the recoloring process includes a weighted distance-based nearest neighbor search-based recoloring process, similar to the above examples.

[0105] The above are some example techniques for performing the attribute recalculation process by the attribute recalculation unit 416. However, the example techniques are not limited thereto. In some examples, to perform the attribute recalculation process, the attribute recalculation unit 416 may be configured to apply a deep learning mechanism. For example, the attribute recalculation unit 416 may represent a trained attribute recalculation model that receives the attribute data 408 and the reconstructed geometry data 414 as inputs and generates the recalculated reconstructed point cloud data 418 using a machine learning technique (e.g., a deep learning mechanism).

[0106] The attribute encoder 420 may receive the recomputed reconstructed point cloud data 418 of the target point cloud. That is, the recomputed reconstructed point cloud data 418 may be similar to the point cloud data of the source point cloud 400 but not necessarily identical.

[0107] The attribute encoder 420 may be configured to encode the recomputed reconstructed point cloud data 418 according to a second encoding process to generate an attribute bitstream 422. For example, the geometry encoder 404 may be configured to encode according to a first encoding process, and the attribute encoder 420 may be configured to encode according to a second encoding process.

[0108] There may be various example ways in which the attribute encoder 420 may encode the recalculated reconstructed point cloud data 418. As an example, the attribute encoder 420 may use a conventional G-PCC encoder to perform joint encoding of the geometry data of the source point cloud 400 and the recalculated reconstructed point cloud data 418. However, in such an example, the conventional G-PCC encoder may generate both a geometry bitstream and an attribute bitstream. In this example, because the geometry encoder 404 generates the geometry bitstream, the geometry bitstream from the attribute encoder 420 may be discarded. The attribute bitstream generated by the attribute encoder 420 may become an attribute bitstream 422. Such a conventional G-PCC encoder may be lossy for the attribute encoding process. Therefore, in some examples, the attribute encoder 420 may perform a lossy attribute encoding process.

[0109] As another example, the second encoding process performed by the attribute encoder 420 can be a deep learning encoding process (e.g., trained and implemented separately from the deep learning encoding process that the geometry encoder 404 can utilize). As another example, the second encoding process performed by the attribute encoder 420 can be a video point cloud compression (V-PCC) encoding process.

[0110] In one or more examples, processing circuitry including the G-PCC encoder 200 can be configured to perform a hybrid encoding process in which the geometry data and the attribute data are encoded using different encoding techniques. For example, a first encoding process (e.g., used by the geometry encoder 404) and a second encoding process (e.g., used by the attribute encoder 420) can be different encoding processes.

[0111] Figure 5 The processing circuitry of the G-PCC decoder 300 is illustrated and includes a geometry decoder 504 and an attribute decoder and geometry combiner unit 508. As shown, the geometry decoder 504 may receive a geometry bitstream 500. The geometry bitstream 500 may be similar to the geometry bitstream 406. The geometry decoder 504 may be similar to the geometry decoder 412 and may be configured to perform the inverse processing of the geometry encoder 404. For example, in an example where the geometry encoder 404 is a deep learning based geometry encoder, the geometry decoder 504 may be a deep learning based geometry decoder.

[0112] In one or more examples, geometry decoder 504 may be part of geometry arithmetic decoding unit 302, and attribute decoder and geometry combiner unit 508 may be part of attribute arithmetic decoding unit 304. However, the techniques are not limited in this regard.

[0113] As an example, the geometry decoder 504 and the attribute decoder and geometry combiner unit 508 are shown as separate units, but can be combined. For example, in one or more examples, the processing circuit including the G-PCC decoder 300 can be considered to be a decoder configured to perform a first decoding process and a second decoding process.

[0114] The geometry decoder 504 and the attribute decoder and geometry combiner unit 508 may also be separate units. For example, in one or more examples, the processing circuit including the G-PCC decoder 300 may include a first decoder (e.g., geometry decoder 504) configured to perform a first decoding process, and a second encoder (e.g., attribute decoder and geometry combiner unit 508) configured to perform a second decoding process. The first decoder (e.g., geometry decoder 504) may be an encoder based on machine learning, and the second decoder (e.g., attribute decoder and geometry combiner unit 508) may be a decoder based on non-machine learning. The first decoder (e.g., geometry decoder 504) may be an encoder based on machine learning, and the second decoder (e.g., attribute decoder and geometry combiner unit 508) may be a decoder based on machine learning.

[0115] Thus, geometry decoder 504 may receive geometry bitstream 500, which includes encoded geometry data for point cloud data encoded according to a first encoding process (e.g., the same encoding process as geometry encoder 404). Geometry decoder 504 may decode the encoded geometry data using a first decoding process that is the inverse of the first encoding process to generate reconstructed geometry data 506. In some examples, reconstructed geometry data 506 may not include attribute data.

[0116] The attribute decoder and geometry combiner unit 508 may receive an attribute bitstream 502 that includes encoded attribute data for point cloud data encoded according to a second encoding process (e.g., the same encoding process as the attribute encoder 420). The attribute decoder and geometry combiner unit 508 may decode the encoded attribute data using a second decoding process that is opposite to the second encoding process and based on the reconstructed geometry data 506 to generate decoded attribute data and reconstructed geometry data for the point cloud data. For example, the attribute decoder and geometry combiner unit 508 may generate a recomputed reconstructed point cloud 510. In some examples, the point cloud data of the recomputed reconstructed point cloud 510 may be similar to the recomputed reconstructed point cloud data 418; however, because the attribute encoder 420 may be lossy, there may be some differences between the point cloud data of the recomputed reconstructed point cloud 510 and the recomputed reconstructed point cloud data 418.

[0117] Similar to Figure 4 In some examples, the first decoding process (e.g., performed by the geometry decoder 504) can be a geometry decoding process based on deep learning. As a few non-limiting examples, the second decoding process (e.g., performed by the attribute decoder and geometry combiner unit 508) can be a lossy attribute decoding process, a deep learning decoding process, or a V-PCC decoding process. In some examples, the first decoding process and the second decoding process are different decoding processes.

[0118] The above example techniques are provided for ease of understanding and should not be considered limiting. As an example, although the geometry encoder 404 and the geometry decoder 504 can be an encoder and decoder based on deep learning, the geometry encoder 404 or the geometry decoder 504 need not be limited to a geometry encoder or decoder based on deep learning, but any geometry encoder or decoder can be used.

[0119] For recoloring, such as by the attribute recalculation unit 416 and / or the attribute decoder and geometry combiner unit 508, the G-PCC encoder 200 and the G-PCC decoder 300 can adopt a recoloring scheme based on a weighted distance nearest neighbor search adopted in the G-PCC standard: WG 7, MPEG 3D Graphics Coding, G-PCC Codec Description, Document N00271, January 2022. As described, the recoloring scheme can change the attributes to fit the newer geometry data. However, any recoloring scheme that can change the values ​​of the attributes and / or their correspondence with the geometry data can be used as a recoloring scheme. That is, the attribute recalculation unit 416 and / or the attribute decoder and the geometry combiner unit 508 can perform various techniques to determine the attribute data of the point.

[0120] "Recoloring" algorithms need not be limited to "recoloring" of color attributes (e.g., RGB or YCbCr), but more generally can be algorithms that recalculate attribute values ​​(such as normal vectors, reflectivity, etc.) from point locations in one geometry to point locations in a second geometry. Deep learning mechanisms can also be applied to perform recoloring.

[0121] As described, in some examples, the attribute encoder 420 can be a G-PCC lossless geometry and lossy attribute encoder. The example techniques are not limited to G-PCC lossless geometry and lossy attribute coding, such as regional adaptive hierarchical transform (RAHT) of G-PCC. As described, the attribute encoder 420 can be used with any coding scheme that includes another deep learning-based encoding or uses V-PCC. In addition, the attribute encoder 420 can use any "lossy attribute encoder" instead of using "lossless geometry and lossy attribute coding". Deep learning mechanisms can also be applied to perform decoding, such as by a geometry decoder 504 and / or an attribute decoder and geometry combiner unit 508.

[0122] The following describes an example of a high-level syntax that can be signaled by the G-PCC encoder 200 and received by the G-PCC decoder 300. The type of attribute encoding method used to encode the recolored attribute can be signaled to the decoder side for the decoder to reconstruct the attribute value. Such an encoding method (e.g., Regional Adaptive Hierarchical Transform (RAHT) of G-PCC) can be signaled in a parameter set, for example, a sequence parameter or an attribute parameter set as an identifier. The bitstream portion that carries the encoded attribute bits is, for example, a NALU. A list of encoding methods can be specified, each of which provides a means for encoding the attributes of the point cloud (optionally, they can also encode geometric data in a lossless manner). An index to the list can be signaled in the bitstream to indicate the encoding method used to encode the attribute. The index can be signaled in a parameter set (e.g., APS, SPS) or otherwise. When a deep learning mechanism is used for recoloring or decoding, the parameters / coefficients corresponding to the recoloring or decoding can also be signaled in the bitstream.

[0123] For example, the G-PCC encoder 200 and the G-PCC decoder 300 can be configured to encode and decode point cloud data using a variety of different techniques, such as those described in the present disclosure, as well as other conventional encoding or decoding techniques. Thus, the G-PCC encoder 200 can signal the G-PCC decoder 300 with information for determining a manner of decoding. For example, in some cases, the geometry decoder 504 can utilize techniques based on deep learning, but for other cases, conventional decoding techniques can be utilized.

[0124] In one or more examples, the first and second decoding processes (e.g., the decoding processes employed by the geometry decoder 504 and the attribute decoder and geometry combiner unit 508, respectively) can be two of a plurality of decoding processes. In such examples, the G-PCC decoder 300 can receive information indicating that the first and second decoding processes of the plurality of decoding processes are to be used for decoding.

[0125] Figure 6 is a flowchart illustrating an example method for encoding point cloud data. For convenience, refer to Figure 1 and Figure 4 For example, one or more memories (such as memory 106 or other memories) may be configured to store point cloud data of source point cloud 400 .

[0126] exist Figure 6 In the example of , processing circuitry including G-PCC encoder 200 may receive geometric data of point cloud data of a source point cloud for a first encoding process ( 600 ). For example, geometry encoder 404 may receive geometric data 402 of point cloud data of source point cloud 400 for a first encoding process.

[0127] The processing circuitry may be configured to encode geometry data according to a first encoding process to generate encoded geometry data and a geometry bitstream (602). For example, geometry encoder 404 may be configured to encode geometry data 402 according to a first encoding process to generate encoded geometry data 410 and geometry bitstream 406. As an example, the first encoding process may be a deep learning based geometry encoding process.

[0128] The processing circuitry may be configured to decode the encoded geometry data to generate reconstructed geometry data for the target point cloud (604). For example, geometry decoder 412 may perform a reverse process of the first encoding process to decode encoded geometry data 410 to generate reconstructed geometry data 414 for the target point cloud.

[0129] The processing circuitry may perform an attribute recalculation process on the attribute data of the point cloud data of the source point cloud based on the reconstructed geometric data to generate recalculated reconstructed point cloud data of the target point cloud (606). For example, the attribute recalculation unit 416 may perform an attribute recalculation process on the attribute data 408 of the point cloud data of the source point cloud 400 based on the reconstructed geometric data 414 to generate recalculated reconstructed point cloud data 418 of the target point cloud.

[0130] As an example, the processing circuitry may perform a recoloring process on the attribute data 408. The recomputed reconstructed point cloud data 418 may be recolored reconstructed point cloud data. In some examples, the recoloring process includes a nearest neighbor search based recoloring process based on a weighted distance.

[0131] As another example that may be performed in combination or separately, in order to perform the attribute recalculation process, the attribute recalculation unit 416 may be configured to receive the reconstructed geometry data 414 of the first point of the target point cloud, and determine a plurality of points in the point cloud data of the source point cloud 400 that are close to the first point of the target point cloud based on the reconstructed geometry data 414 (e.g., for the first point of the target point cloud). The attribute recalculation unit 416 may determine the attribute data of the plurality of points in the point cloud data of the source point cloud 400, and determine the attribute data of the first point in the target point cloud based on the attribute data of the plurality of points of the source point cloud 400, to generate recalculated reconstructed point cloud data 418 of the target point cloud.

[0132] In some examples, to perform the attribute recalculation process, the attribute recalculation unit 416 may apply a deep learning mechanism to perform the attribute recalculation process. As an example, the above example techniques described for the attribute recalculation unit 416 or other attribute recalculation processes may be based on generating a learning model, and the processing circuit executes the training model to perform the attribute recalculation process.

[0133] The processing circuitry may encode the recomputed reconstructed point cloud data according to a second encoding process to generate an attribute bitstream (608). For example, the attribute encoder 420 may encode the recomputed reconstructed point cloud data 418 according to the second encoding process to generate an attribute bitstream 422. Examples of the second encoding process include a lossy attribute encoding process, a deep learning encoding process, and a video point cloud compression (V-PCC) encoding process. In some examples, the first encoding process and the second encoding process are different encoding processes.

[0134] Figure 7 is a flowchart illustrating an example method for decoding point cloud data. Figure 1 and Figure 5 For example, one or more memories (such as memory 120 or other memories) may be configured to store point cloud data.

[0135] Processing circuitry including G-PCC decoder 300 may receive a geometry bitstream 700 including encoded geometry data for point cloud data encoded according to a first encoding process. For example, geometry decoder 504 may receive geometry bitstream 500 including encoded geometry data for point cloud data encoded according to a first encoding process.

[0136] The processing circuit may decode the encoded geometry data using a first decoding process that is opposite to the first encoding process to generate reconstructed geometry data (702). For example, the geometry decoder 504 may decode the encoded geometry data using a first decoding process that is opposite to the first encoding process to generate reconstructed geometry data 506. The first decoding process may be a deep learning based geometry decoding process.

[0137] The processing circuitry may receive an attribute bitstream including encoded attribute data for point cloud data encoded according to the second encoding process (704). For example, the attribute decoder and geometry combiner unit 508 may receive the attribute bitstream 502 including encoded attribute data for point cloud data encoded according to the second encoding process.

[0138] The processing circuitry may decode the encoded attribute data using a second decoding process that is the reverse of the second encoding process and based on the reconstructed geometry data to generate decoded attribute data and reconstructed geometry data for the point cloud data (706). For example, the attribute decoder and geometry combiner unit 508 may decode the encoded attribute data using a second decoding process that is the reverse of the second encoding process and based on the reconstructed geometry data to generate decoded attribute data and reconstructed geometry data for the point cloud data.

[0139] Examples of the second decoding process include a lossy attribute decoding process, a deep learning decoding process, and a video point cloud compression (V-PCC) decoding process. In some examples, the first decoding process and the second decoding process are different decoding processes. In addition, in some examples, the first and second decoding processes are both of a plurality of decoding processes. The processing circuit may receive information indicating that the first decoding process and the second decoding process of the plurality of decoding processes will be used for decoding.

[0140] Figure 8 is a conceptual diagram illustrating an example ranging system 800 that may be used with one or more techniques of this disclosure. Figure 8 In the example of , ranging system 800 includes an illuminator 802 and a sensor 804. Illuminator 802 can emit light 806. In some examples, illuminator 802 can emit light 806 as one or more laser beams. Light 806 can be one or more wavelengths, such as infrared wavelengths or visible light wavelengths. In other examples, light 806 is not a coherent laser. When light 806 encounters an object (such as object 808), light 806 produces return light 810. Return light 810 may include backscattered light and / or reflected light. Return light 810 may pass through lens 811, which guides return light 810 to create an image 812 of object 808 on sensor 804. Sensor 804 generates signal 814 based on image 812. Image 812 may include a collection of points (e.g., such as Figure 8812).

[0141] In some examples, the illuminator 802 and the sensor 804 can be mounted on a rotating structure so that the illuminator 802 and the sensor 804 capture a 360 degree view of the environment (e.g., a rotating LIDAR sensor). In other examples, the ranging system 800 can include one or more optical components (e.g., mirrors, collimators, diffraction gratings, etc.) that enable the illuminator 802 and the sensor 804 to detect the range of objects within a certain range (e.g., up to 360 degrees). Although Figure 8 The example of shows only a single illuminator 802 and sensor 804, but the ranging system 800 may include multiple sets of illuminators and sensors.

[0142] In some examples, the illuminator 802 generates a structured light pattern. In such an example, the ranging system 800 may include a plurality of sensors 804 on which respective images of the structured light pattern are formed. The ranging system 800 may use differences between the images of the structured light pattern to determine the distance to an object 808 from which the structured light pattern is backscattered. When the object 808 is relatively close to the sensor 804 (e.g., 0.2 meters to 2 meters), the ranging system based on structured light may have a high level of accuracy (e.g., accuracy in the sub-millimeter range). This high level of accuracy may be useful in facial recognition applications such as unlocking a mobile device (e.g., a mobile phone, a tablet computer, etc.) and in security applications.

[0143] In some examples, the ranging system 800 is a system based on time of flight (ToF). In some examples where the ranging system 800 is a ToF-based system, the illuminator 802 generates light pulses. In other words, the illuminator 802 can modulate the amplitude of the emitted light 806. In such an example, the sensor 804 detects the return light 810 from the light pulse 806 generated by the illuminator 802. The ranging system 800 can then determine the distance to the object 808 from which the light 806 is backscattered based on the delay between the time when the light 806 is emitted and detected and the known speed of light in the air. In some examples, instead of modulating the amplitude of the emitted light 806 (or in addition to modulating the amplitude of the emitted light 806), the illuminator 802 can modulate the phase of the emitted light 806. In such an example, sensor 804 can detect the phase of return light 810 from object 808 and determine the distance to a point on object 808 using the speed of light and based on the time difference between the time when illuminator 802 generates light 806 at a particular phase and the time when sensor 804 detects return light 810 at the particular phase.

[0144] In other examples, a point cloud can be generated without using an illuminator 802. For example, in some examples, the sensor 804 of the ranging system 800 can include two or more optical cameras. In such an example, the ranging system 800 can use the optical cameras to capture a stereoscopic image of the environment (including the object 808). The ranging system 800 can include a point cloud generator 816 that can calculate the difference between the positions in the stereoscopic images. The ranging system 800 can then use the difference to determine the distance to the position shown in the stereoscopic image. Based on these distances, the point cloud generator 816 can generate a point cloud.

[0145] Sensor 804 may also detect other properties of object 808, such as color and reflectivity information. Figure 8 In the example of , the point cloud generator 816 can generate a point cloud based on the signal 814 generated by the sensor 804. The ranging system 800 and / or the point cloud generator 816 can form a data source 104 ( Figure 1 ). Thus, the point cloud generated by the ranging system 800 may be encoded and / or decoded according to any of the techniques of this disclosure.

[0146] Fig. 9 is a conceptual diagram illustrating an exemplary vehicle-based scenario in which one or more techniques of the present disclosure may be used. Fig. 9 In the example of FIG. 1 , vehicle 900 includes a ranging system 902. The ranging system 902 can be used to measure the distance between the vehicle and the vehicle. Figure 8 Although Fig. 9 , but the vehicle 900 may also include a data source (such as data source 104 ( Figure 1 )) and a G-PCC encoder (such as G-PCC encoder 200 ( Figure 1 )).exist Fig. 9 In the example of FIG. 1 , a range finding system 902 emits a laser beam 904 that reflects off a pedestrian 906 or other object in the road. A data source of the vehicle 900 can generate a point cloud based on the signal generated by the range finding system 902. A G-PCC encoder of the vehicle 900 can encode the point cloud to generate a bit stream 908, such as a geometry bit stream ( Figure 2 ) and the attribute bitstream ( Figure 2 ).

[0147] The output interface of the vehicle 900 (eg, the output interface 108 ( Figure 1 )) can send a bitstream 908 to one or more other devices. The bitstream 908 may include significantly fewer bits than the unencoded point cloud obtained by the G-PCC encoder. Therefore, the vehicle 900 can send the bitstream 908 to other devices faster than the unencoded point cloud data. In addition, the bitstream 908 may require less data storage capacity on the device.

[0148] exist Fig. 9 In the example of FIG. 1 , vehicle 900 may send a bitstream 908 to another vehicle 910. Vehicle 910 may include a G-PCC decoder, such as G-PCC decoder 300 ( Figure 1 ). The G-PCC decoder of vehicle 910 may decode bitstream 908 to reconstruct the point cloud. Vehicle 910 may use the reconstructed point cloud for various purposes. For example, vehicle 910 may determine that pedestrian 906 is in the road ahead of vehicle 900 based on the reconstructed point cloud, and therefore begin to slow down, for example, even before the driver of vehicle 910 realizes that pedestrian 906 is on the road. Thus, in some examples, vehicle 910 may perform autonomous navigation operations based on the reconstructed point cloud.

[0149] Additionally or alternatively, the vehicle 900 can send the bitstream 908 to the server system 912. The server system 912 can use the bitstream 908 for various purposes. For example, the server system 912 can store the bitstream 908 for subsequent reconstruction of the point cloud. In this example, the server system 912 can use the point cloud and other data (e.g., vehicle telemetry data generated by the vehicle 900) to train an autonomous driving system. In other examples, the server system 912 can store the bitstream 908 for subsequent reconstruction of a forensic collision investigation.

[0150] Fig.10 is a conceptual diagram illustrating an example extended reality system in which one or more techniques of the present disclosure may be used. Extended reality (XR) is a term used to encompass a range of technologies including augmented reality (AR), mixed reality (MR), and virtual reality (VR). Fig.10 In the example of FIG. 1 , user 1000 is located at a first location 1002. User 1000 wears an XR headset 1004. As an alternative to XR headset 1004, user 1000 may use a mobile device (e.g., a mobile phone, a tablet computer, etc.). XR headset 1004 includes a depth detection sensor, such as a range finding system, that detects the location of a point on object 1006 at location 1002. A data source for XR headset 1004 may use a signal generated by the depth detection sensor to generate a point cloud representation of object 1006 at location 1002. XR headset 1004 may include a G-PCC encoder (e.g., Figure 1 A G-PCC encoder 200 is configured to encode the point cloud to generate a bitstream 1008.

[0151] The XR headset 1004 may transmit the bitstream 1008 (e.g., via a network such as the Internet) to an XR head mounted device 1010 worn by a user 1012 at a second location 1014. The XR head mounted device 1010 may decode the bitstream 1008 to reconstruct the point cloud. The XR head mounted device 1010 may use the point cloud to generate an XR visualization (e.g., an AR, MR, VR visualization) representing an object 1006 at the location 1002. Thus, in some examples, for example, when the XR head mounted device 1010 generates a VR visualization, the user 1012 may have a 3D immersive experience of the location 1002. In some examples, the XR head mounted device 1010 may determine the location of a virtual object based on the reconstructed point cloud. For example, the XR head mounted device 1010 may determine that the environment (e.g., location 1002) includes a flat surface based on the reconstructed point cloud, and then determine that the virtual object (e.g., a cartoon character) is to be positioned on the flat surface. The XR head mounted device 1010 may generate an XR visualization in which the virtual object is at the determined position. For example, the XR head mounted device 1010 may show a cartoon character sitting on a flat surface.

[0152] Fig.11 is a conceptual diagram illustrating an example mobile device system in which one or more techniques of this disclosure may be used. Fig.11 In an example of the present invention, a mobile device 1100 (e.g., a wireless communication device) (e.g., a mobile phone or a tablet computer) includes a range finding system, such as a LIDAR system, which detects the location of points on an object 1102 in the environment of the mobile device 1100. A data source of the mobile device 1100 may use a signal generated by a depth detection sensor to generate a point cloud representation of the object 1102. The mobile device 1100 may include a G-PCC encoder (e.g., Figure 1 The G-PCC encoder 200 is configured to encode the point cloud to generate a bitstream 1104. Fig.11 In an example, the mobile device 1100 may send a bitstream to a remote device 1106, such as a server system or other mobile device. The remote device 1106 may decode the bitstream 1104 to reconstruct a point cloud. The remote device 1106 may use the point cloud for various purposes. For example, the remote device 1106 may use the point cloud to generate a map of the environment of the mobile device 1100. For example, the remote device 1106 may generate a map of the interior of a building based on the reconstructed point cloud. In another example, the remote device 1106 may generate an image (e.g., computer graphics) based on the point cloud. For example, the remote device 1106 may use the points of the point cloud as vertices of a polygon, and use the color attributes of the points as the basis for coloring the polygon. In some examples, the remote device 1106 may use the reconstructed point cloud for facial recognition or other security applications.

[0153] The examples in various aspects of this disclosure may be used alone or in any combination.

[0154] Item 1A. A method for encoding point cloud data, the method comprising: for a first encoding process, receiving geometric data of the point cloud data instead of attribute data of the point cloud data; encoding the geometric data according to the first encoding process to generate a geometric bit stream; reconstructing the geometric data to generate reconstructed geometric data; performing an attribute recalculation process on the attribute data of the point cloud data based on the reconstructed geometric data to generate recalculated reconstructed point cloud data; and encoding the recalculated reconstructed point cloud data according to a second encoding process to generate an attribute bit stream.

[0155] Clause 2A. The method of clause 1A, wherein the first encoding process comprises a deep learning based geometric encoding process.

[0156] Clause 3A. The method of any of clauses 1A and 2A, wherein the second encoding pass comprises a lossless geometry and a lossy attribute encoding pass.

[0157] Clause 4A. The method of any one of clauses 1A and 2A, wherein the second encoding process comprises a deep learning encoding process.

[0158] Clause 5A. The method of any of Clause A, wherein the second encoding process comprises a video point cloud compression (V-PCC) encoding process.

[0159] Clause 6A. The method of Clause 1A, wherein the first encoding process and the second encoding process are different encoding processes.

[0160] Clause 7A. A method according to any of clauses 1A-6A, wherein performing an attribute recalculation process includes performing a recoloring process on the attribute data, wherein the recalculated reconstructed point cloud data includes recolored reconstructed point cloud data, and wherein encoding the recalculated reconstructed point cloud data includes encoding the recolored reconstructed point cloud data to generate an attribute bitstream.

[0161] Clause 8A. The method of Clause 7A, wherein the recoloring process comprises a weighted distance-based nearest neighbor search-based recoloring process.

[0162] Clause 9A. The method of any of clauses 1A-8A, wherein performing the attribute recalculation process comprises applying a deep learning mechanism to perform the attribute recalculation process.

[0163] Clause 10A. The method of any one of clauses 1A-9A, wherein the attribute recalculation process is a first attribute recalculation process of a plurality of attribute recalculation processes, the method further comprising: signaling information indicating use of the first attribute recalculation process of the plurality of attribute recalculation processes.

[0164] Clause 11A. A method for decoding point cloud data, the method comprising: receiving a geometry bitstream comprising encoded geometry data of point cloud data encoded according to a first encoding process; decoding the encoded geometry data using a first decoding process opposite to the first encoding process to generate reconstructed geometry data; receiving an attribute bitstream comprising encoded attribute data of the point cloud data encoded according to a second encoding process; and decoding the encoded attribute data based on the reconstructed geometry data using a second decoding process opposite to the second encoding process to generate decoded attribute data and reconstructed geometry data of the point cloud data.

[0165] Clause 12A. The method of clause 11A, wherein the first decoding process comprises a deep learning based geometric decoding process.

[0166] Clause 13A. The method of any of Clauses 11A and 12A, wherein the second decoding process comprises a lossless geometry data and a lossy attribute decoding process.

[0167] Clause 14A. The method of any of Clauses 11A and 12A, wherein the second decoding process comprises a deep learning decoding process.

[0168] Clause 15A. The method of any of Clauses 11A and 12A, wherein the second decoding process comprises a Video Point Cloud Compression (V-PCC) decoding process.

[0169] Clause 16A. The method of Clause 11A, wherein the first decoding process and the second decoding process are different decoding processes.

[0170] Clause 17A. The method of any of Clauses 11A-16A, wherein the second decoding process comprises an inverse recoloring process.

[0171] Clause 18A. The method of Clause 17A, wherein the inverse recoloring process comprises a weighted distance-based nearest neighbor search-based inverse recoloring process.

[0172] Clause 19A. The method of any of clauses 11A-18A, wherein the second decoding process comprises a deep learning based decoding process.

[0173] Clause 20A. The method of any of 11A-19A, wherein the second decoding process is one of a plurality of decoding processes, the method further comprising: receiving information indicating that a second decoding process of the plurality of decoding processes is to be used for decoding.

[0174] Clause 21A. A system for encoding point cloud data, the system comprising: a memory configured to store the point cloud data; and a processing circuit coupled to the memory and configured to perform the method of any of clauses 1A-10A.

[0175] Clause 22A. The system of Clause 21A, wherein the processing circuit is configured to generate the point cloud data.

[0176] Clause 23A. A system for decoding point cloud data, the system comprising: a memory configured to store the point cloud data; and a processing circuit coupled to the memory and configured to perform the method of any of clauses 11A-20A.

[0177] Clause 24A. The system of clause 23A, wherein the processing circuit is configured to display to present an image based on the point cloud.

[0178] Clause 25A. An apparatus for encoding point cloud data, the apparatus comprising means for performing at least one of the method of any of clauses 1A-10A or the method of any of clauses 11A-20A.

[0179] Clause 26A. A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to perform the method of any of clauses 1A-10A or the method of any of clauses 11A-20A.

[0180] Item 1B. A method for encoding point cloud data, the method comprising: receiving geometric data of point cloud data of a source point cloud for a first encoding process; encoding the geometric data according to the first encoding process to generate encoded geometric data and a geometric bit stream; decoding the encoded geometric data to generate reconstructed geometric data of a target point cloud; performing an attribute recalculation process on attribute data of the point cloud data of the source point cloud based on the reconstructed geometric data to generate recalculated reconstructed point cloud data of the target point cloud; and encoding the recalculated reconstructed point cloud data according to a second encoding process to generate an attribute bit stream.

[0181] Clause 2B. The method of clause 1B, wherein the first encoding process comprises a deep learning based geometric encoding process.

[0182] Clause 3B. The method of any of clauses 1B and 2B, wherein the second encoding pass comprises a lossy attribute encoding pass.

[0183] Clause 4B. A method as described in any of clauses 1B and 2B, wherein the second encoding process comprises a deep learning encoding process.

[0184] Clause 5B. The method of any of clauses 1B and 2B, wherein the second encoding process comprises a video point cloud compression (V-PCC) encoding process.

[0185] Clause 6B. The method of any of clauses 1B-5B, wherein the first encoding pass and the second encoding pass are different encoding passes.

[0186] Clause 7B. A method as described in any of clauses 1B-6B, wherein performing an attribute recalculation process includes performing a recoloring process on the attribute data, wherein the recalculated reconstructed point cloud data includes recolored reconstructed point cloud data, and wherein encoding the recalculated reconstructed point cloud data includes encoding the recolored reconstructed point cloud data to generate an attribute bitstream.

[0187] Clause 8B. The method of Clause 7B, wherein the recoloring process comprises a weighted distance-based nearest neighbor search-based recoloring process.

[0188] Clause 9B. A method according to any one of clauses 1B-8B, wherein performing an attribute recalculation process includes: receiving reconstructed geometric data of a first point of a target point cloud; determining, based on the reconstructed geometric data, a plurality of points in the point cloud data of the source point cloud that are close to the first point of the target point cloud; determining attribute data of the plurality of points in the point cloud data of the source point cloud; and determining attribute data of the first point in the target point cloud based on the attribute data of the plurality of points of the source point cloud to generate recalculated reconstructed point cloud data of the target point cloud.

[0189] Clause 10B. The method of any of clauses 1B-8B, wherein performing the attribute recalculation process comprises applying a deep learning mechanism to perform the attribute recalculation process.

[0190] Item 11B. A method for decoding point cloud data, the method comprising: receiving a geometry bitstream comprising encoded geometry data of point cloud data encoded according to a first encoding process; decoding the encoded geometry data using a first decoding process opposite to the first encoding process to generate reconstructed geometry data; receiving an attribute bitstream comprising encoded attribute data of the point cloud data encoded according to a second encoding process; and decoding the encoded attribute data based on the reconstructed geometry data using a second decoding process opposite to the second encoding process to generate decoded attribute data and reconstructed geometry data of the point cloud data.

[0191] Clause 12B. The method of clause 11B, wherein the first decoding process comprises a deep learning based geometric decoding process.

[0192] Clause 13B. The method of any of Clauses 11B and 12B, wherein the second decoding process comprises a lossy attribute decoding process.

[0193] Clause 14B. The method of any of Clauses 11B and 12B, wherein the second decoding process comprises a deep learning decoding process.

[0194] Clause 15B. The method of any of Clauses 11B and 12B, wherein the second decoding process comprises a Video Point Cloud Compression (V-PCC) decoding process.

[0195] Clause 16B. The method of any of clauses 11B-15B, wherein the first decoding process and the second decoding process are different decoding processes.

[0196] Clause 17B. The method of any of clauses 11B-16B, wherein the first and second decoding processes are two of a plurality of decoding processes, the method further comprising: receiving information indicating that the first and second decoding processes of the plurality of decoding processes are to be used for decoding.

[0197] Item 18B. A system for encoding point cloud data, the system comprising: one or more memories configured to store point cloud data; and a processing circuit coupled to the one or more memories and configured to: receive geometric data of point cloud data of a source point cloud for a first encoding process; encode the geometric data according to the first encoding process to generate encoded geometric data and a geometric bit stream; decode the encoded geometric data to generate reconstructed geometric data of a target point cloud; perform an attribute recalculation process on attribute data of the point cloud data of the source point cloud based on the reconstructed geometric data to generate recalculated reconstructed point cloud data of the target point cloud; and encode the recalculated reconstructed point cloud data according to a second encoding process to generate an attribute bit stream.

[0198] Clause 19B. The system of clause 18B, wherein the first encoding process comprises a deep learning based geometric encoding process.

[0199] Clause 20B. The system of any of Clauses 18B and 19B, wherein the second encoding process comprises a lossy attribute encoding process.

[0200] Clause 21B. The system of any of clauses 18B and 19B, wherein the second encoding process comprises a deep learning encoding process.

[0201] Clause 22B. The system of any of Clauses 18B and 19B, wherein the second encoding process comprises a video point cloud compression (V-PCC) encoding process.

[0202] Clause 23B. The system of any of clauses 18B-22B, wherein the first encoding process and the second encoding process are different encoding processes.

[0203] Clause 24B. A system as described in any of clauses 18B-23B, wherein to perform an attribute recalculation process, the processing circuit is configured to perform a recoloring process on the attribute data, wherein the recalculated reconstructed point cloud data includes recolored reconstructed point cloud data, and wherein to encode the recalculated reconstructed point cloud data, the processing circuit is configured to encode the recolored reconstructed point cloud data to generate an attribute bitstream.

[0204] Clause 25B. The system of Clause 24B, wherein the recoloring process comprises a weighted distance based nearest neighbor search based recoloring process.

[0205] Clause 26B. A system according to any one of clauses 18B-25B, wherein, in order to perform an attribute recalculation process, the processing circuit is configured to: receive reconstructed geometric data of a first point of a target point cloud; determine, based on the reconstructed geometric data, multiple points in the point cloud data of the source point cloud that are close to the first point of the target point cloud; determine attribute data of the multiple points in the point cloud data of the source point cloud; determine attribute data of the first point in the target point cloud based on the attribute data of the multiple points of the source point cloud; and assign attribute data to the first point of the target point cloud to generate recalculated reconstructed point cloud data of the target point cloud.

[0206] Clause 27B. The system of any of clauses 18B-25B, wherein to perform the attribute recalculation process, the processing circuit is configured to apply a deep learning mechanism to perform the attribute recalculation process.

[0207] Clause 28B. The system of any of clauses 18B to 27B, wherein the processing circuit comprises an encoder configured to perform the first encoding process and the second encoding process.

[0208] Clause 29B. The system of any of clauses 18B-27B, wherein the processing circuit comprises a first encoder configured to perform a first encoding process and a second encoder configured to perform a second encoding process.

[0209] Clause 30B. The system of Clause 29B, wherein the first encoder is a machine learning based encoder and the second encoder is a non-machine learning based encoder.

[0210] Clause 31B. The system of Clause 29B, wherein the first encoder is a machine learning based encoder and the second encoder is a machine learning based encoder.

[0211] Item 32B. A system for decoding point cloud data, the system comprising: one or more memories configured to store point cloud data; and a processing circuit coupled to the one or more memories and configured to: receive a geometry bitstream comprising encoded geometry data of point cloud data encoded according to a first encoding process; decode the encoded geometry data using a first decoding process opposite to the first encoding process to generate reconstructed geometry data; receive an attribute bitstream comprising encoded attribute data of the point cloud data encoded according to a second encoding process; and decode the encoded attribute data using a second decoding process opposite to the second encoding process and based on the reconstructed geometry data to generate decoded attribute data and reconstructed geometry data of the point cloud data.

[0212] Clause 33B. The system of clause 32B, wherein the first decoding process comprises a deep learning based geometric decoding process.

[0213] Clause 34B. The system of any of clauses 32B and 33B, wherein the second decoding process comprises a lossy attribute decoding process.

[0214] Clause 35B. A system as described in any of clauses 32B and 33B, wherein the second decoding process comprises a deep learning decoding process.

[0215] Clause 36B. The system of any of clauses 32B and 33B, wherein the second decoding process comprises a video point cloud compression (V-PCC) decoding process.

[0216] Clause 37B. The system of any of clauses 32B-36B, wherein the first decoding process and the second decoding process are different decoding processes.

[0217] Clause 38B. A system as described in any of clauses 32B-37B, wherein the first decoding process and the second decoding process are two of a plurality of decoding processes, and wherein the processing circuit is configured to: receive information indicating that the first decoding process and the second decoding process of the plurality of decoding processes are to be used for decoding.

[0218] Clause 39B. The system of any of clauses 32B-38B, wherein the processing circuit comprises a decoder configured to perform the first decoding process and the second decoding process.

[0219] Clause 40B. The system of any of clauses 32B-38B, wherein the processing circuit comprises a first decoder configured to perform the first decoding process and a second decoder configured to perform the second decoding process.

[0220] Clause 41B. The system of clause 40B, wherein the first decoder is a machine learning based decoder and the second decoder is a non-machine learning based decoder.

[0221] Clause 42B. The system of clause 40B, wherein the first decoder is a machine learning based decoder and the second encoder is a machine learning based decoder.

[0222] Item 43B. A computer-readable storage medium having instructions stored thereon, which when executed cause one or more processors to: receive geometric data of point cloud data of a source point cloud for a first encoding process; encode the geometric data according to the first encoding process to generate encoded geometric data and a geometric bit stream of a target point cloud; decode the encoded geometric data to generate reconstructed geometric data; perform an attribute recalculation process on attribute data of the point cloud data of the source point cloud based on the reconstructed geometric data to generate recalculated reconstructed point cloud data of the target point cloud; and encode the recalculated reconstructed point cloud data according to a second encoding process to generate an attribute bit stream.

[0223] Item 44B. A computer-readable storage medium having instructions stored thereon, which when executed cause one or more processors to: receive a geometry bitstream comprising encoded geometry data of point cloud data encoded according to a first encoding process; decode the encoded geometry data using a first decoding process that is opposite to the first encoding process to generate reconstructed geometry data; receive an attribute bitstream comprising encoded attribute data of the point cloud data encoded according to a second encoding process; and decode the encoded attribute data using a second decoding process that is opposite to the second encoding process and based on the reconstructed geometry data to generate decoded attribute data and reconstructed geometry data for the point cloud data.

[0224] It should be appreciated that, according to examples, certain actions or events of any of the techniques described herein may be performed in a different order, may be added, combined, or omitted entirely (e.g., not all described actions or events are required to practice the techniques). Furthermore, in some examples, actions or events may be performed simultaneously rather than sequentially, for example, through multithreading, interrupt processing, or multiple processors.

[0225] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or codes and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium that corresponds to a tangible medium such as a data storage medium, or a communication medium that includes any medium that facilitates, for example, the transfer of a computer program from one place to another according to a communication protocol. In this manner, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementation of the techniques described in the present disclosure. A computer program product may include a computer-readable medium.

[0226] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures and can be accessed by a computer. In addition, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave) is used to transmit instructions from a website, server or other remote source, then the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology (such as infrared, radio and microwave) is included in the definition of the medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carriers, signals or other temporary media, but are actually directed to non-temporary tangible storage media. As used herein, disks and optical disks include compact disks (CDs), laser optical disks, optical optical disks, digital versatile disks (DVDs), floppy disks and blue-ray disks, wherein disks usually reproduce data magnetically, and optical disks reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0227] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the terms "processor" and "processing circuit" as used herein may refer to any of the aforementioned structures or any other structures suitable for implementation in the techniques described herein. In addition, in some aspects, the functionality described herein may be provided in dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. In addition, these techniques may be fully implemented in one or more circuits or logic elements.

[0228] The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including wireless handsets, integrated circuits (ICs), or sets of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily need to be implemented by different hardware units. Instead, as described above, the various units may be combined in a codec hardware unit in conjunction with appropriate software and / or firmware, or provided by a collection of interoperating hardware units (including one or more processors as described above).

[0229] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. A method for encoding point cloud data, the method comprising: For a first encoding process, receiving geometric data of point cloud data of a source point cloud; encoding the geometry data according to the first encoding process to generate encoded geometry data and a geometry bitstream; decoding the encoded geometric data to generate reconstructed geometric data of a target point cloud; performing an attribute recalculation process on attribute data of the point cloud data of the source point cloud based on the reconstructed geometric data to generate recalculated reconstructed point cloud data of the target point cloud; as well as The recomputed reconstructed point cloud data is encoded according to a second encoding process to generate an attribute bitstream.

2. The method according to claim 1, wherein: The first encoding process includes a geometric encoding process based on deep learning.

3. The method according to claim 1, wherein: The second encoding process includes a lossy attribute encoding process.

4. The method according to claim 1, wherein: The second encoding process includes a deep learning encoding process.

5. The method according to claim 1, wherein: The second encoding process includes a video point cloud compression (V-PCC) encoding process.

6. The method according to claim 1, wherein: The first encoding process and the second encoding process are different encoding processes.

7. The method according to claim 1, wherein: Performing the attribute recalculation process includes performing a recoloring process on the attribute data, wherein the recalculated reconstructed point cloud data includes recolored reconstructed point cloud data, and wherein encoding the recalculated reconstructed point cloud data includes encoding the recolored reconstructed point cloud data to generate the attribute bitstream.

8. The method according to claim 7, wherein: The recoloring process includes a weighted distance based nearest neighbor search based recoloring process.

9. The method according to claim 1, wherein: Executing the attribute recalculation process includes: Receiving reconstructed geometric data of a first point of the target point cloud; Based on the reconstructed geometric data, determining a plurality of points in the point cloud data of the source point cloud that are close to the first point of the target point cloud; determining attribute data of the plurality of points in the point cloud data of the source point cloud; and Attribute data of the first point in the target point cloud is determined based on the attribute data of the plurality of points of the source point cloud to generate recomputed reconstructed point cloud data of the target point cloud.

10. The method according to claim 1, wherein: Performing the attribute recalculation process includes applying a deep learning mechanism to perform the attribute recalculation process.

11. A method for decoding point cloud data, the method comprising: receiving a geometry bitstream comprising encoded geometry data of the point cloud data encoded according to a first encoding process; Decoding the encoded geometric data using a first decoding process that is the inverse of the first encoding process to generate reconstructed geometric data; receiving an attribute bitstream, the attribute bitstream comprising encoded attribute data of the point cloud data encoded according to a second encoding process; as well as The encoded attribute data is decoded based on the reconstructed geometric data using a second decoding process that is opposite to the second encoding process to generate decoded attribute data and reconstructed geometric data of the point cloud data.

12. The method according to claim 11, wherein: The first decoding process includes a geometric decoding process based on deep learning.

13. The method according to claim 11, wherein: The second decoding process includes a lossy attribute decoding process.

14. The method according to claim 11, wherein: The second decoding process includes a deep learning decoding process.

15. The method according to claim 11, wherein: The second decoding process includes a video point cloud compression (V-PCC) decoding process.

16. The method according to claim 11, wherein: The first decoding process and the second decoding process are different decoding processes.

17. The method according to claim 11, wherein: The first decoding process and the second decoding process are two of a plurality of decoding processes, and the method further includes: Information indicating that the first decoding process and the second decoding process of the plurality of decoding processes are to be used for decoding is received.

18. A system for encoding point cloud data, the system comprising: one or more memories configured to store point cloud data; as well as a processing circuit coupled to the one or more memories and configured to: For a first encoding process, receiving geometric data of point cloud data of a source point cloud; encoding the geometry data according to the first encoding process to generate encoded geometry data and a geometry bitstream; decoding the encoded geometric data to generate reconstructed geometric data of a target point cloud; performing an attribute recalculation process on attribute data of the point cloud data of the source point cloud based on the reconstructed geometric data to generate recalculated reconstructed point cloud data of the target point cloud; as well as The recomputed reconstructed point cloud data is encoded according to a second encoding process to generate an attribute bitstream.

19. The system of claim 18, wherein the first encoding process comprises a deep learning based geometric encoding process.

20. The system of claim 18, wherein: The second encoding process includes a lossy attribute encoding process.

21. The system of claim 18, wherein the second encoding process comprises a deep learning encoding process.

22. The system of claim 18, wherein: The second encoding process includes a video point cloud compression (V-PCC) encoding process.

23. The system of claim 18, wherein: The first encoding process and the second encoding process are different encoding processes.

24. The system of claim 18, wherein: In order to perform the attribute recalculation process, the processing circuit is configured to perform a recoloring process on the attribute data, wherein the recalculated reconstructed point cloud data includes recolored reconstructed point cloud data, and wherein, in order to encode the recalculated reconstructed point cloud data, the processing circuit is configured to encode the recolored reconstructed point cloud data to generate the attribute bitstream.

25. The system of claim 24, wherein: The recoloring process includes a weighted distance based nearest neighbor search based recoloring process.

26. The system of claim 18, wherein: To perform the attribute recalculation process, the processing circuit is configured to: Receiving reconstructed geometric data of a first point of the target point cloud; Based on the reconstructed geometric data, determining a plurality of points in the point cloud data of the source point cloud that are close to the first point of the target point cloud; determining attribute data of the plurality of points in the point cloud data of the source point cloud; determining attribute data of the first point in the target point cloud based on the attribute data of the plurality of points in the source point cloud; as well as The attribute data is assigned to the first point of the target point cloud to generate recalculated reconstructed point cloud data of the target point cloud.

27. The system of claim 18, wherein to perform the attribute recalculation process, the processing circuit is configured to apply a deep learning mechanism to perform the attribute recalculation process.

28. The system of claim 18, wherein: The processing circuit includes an encoder configured to perform the first encoding process and the second encoding process.

29. The system of claim 18, wherein the processing circuit comprises a first encoder configured to perform the first encoding process and a second encoder configured to perform the second encoding process.

30. The system of claim 29, wherein the first encoder is a machine learning based encoder and the second encoder is a non-machine learning based encoder.

31. The system of claim 29, wherein the first encoder is a machine learning based encoder and the second encoder is a machine learning based encoder.

32. A system for decoding point cloud data, the system comprising: one or more memories configured to store point cloud data; as well as a processing circuit coupled to the one or more memories and configured to: receiving a geometry bitstream comprising encoded geometry data of the point cloud data encoded according to a first encoding process; Decoding the encoded geometric data using a first decoding process that is opposite to the first encoding process to generate reconstructed geometric data; receiving an attribute bitstream, the attribute bitstream comprising encoded attribute data of the point cloud data encoded according to a second encoding process; as well as The encoded attribute data is decoded based on the reconstructed geometric data using a second decoding process that is opposite to the second encoding process to generate decoded attribute data and reconstructed geometric data of the point cloud data.

33. The system of claim 32, wherein the first decoding process comprises a deep learning based geometric decoding process.

34. The system of claim 32, wherein: The second decoding process includes a lossy attribute decoding process.

35. The system of claim 32, wherein the second decoding process comprises a deep learning decoding process.

36. The system of claim 32, wherein: The second decoding process includes a video point cloud compression (V-PCC) decoding process.

37. The system of claim 32, wherein: The first decoding process and the second decoding process are different decoding processes.

38. The system of claim 32, wherein: The first decoding process and the second decoding process are two of a plurality of decoding processes, and wherein the processing circuit is configured to: Information indicating that the first decoding process and the second decoding process of the plurality of decoding processes are to be used for decoding is received.

39. The system of claim 32, wherein: The processing circuit includes a decoder configured to perform the first decoding process and the second decoding process.

40. The system of claim 32, wherein: The processing circuit includes a first decoder configured to perform the first decoding process and a second decoder configured to perform the second decoding process.

41. The system of claim 40, wherein the first decoder is a machine learning based decoder and the second decoder is a non-machine learning based decoder.

42. The system of claim 40, wherein the first decoder is a machine learning based decoder and the second encoder is a machine learning based decoder.

43. A computer-readable storage medium having stored thereon instructions which, when executed, cause one or more processors to: For a first encoding process, receiving geometric data of point cloud data of a source point cloud; encoding the geometric data according to the first encoding process to generate encoded geometric data and a geometric bitstream of a target point cloud; decoding the encoded geometric data to generate reconstructed geometric data; performing an attribute recalculation process on attribute data of the point cloud data of the source point cloud based on the reconstructed geometric data to generate recalculated reconstructed point cloud data of the target point cloud; as well as The recomputed reconstructed point cloud data is encoded according to a second encoding process to generate an attribute bitstream.

44. A computer-readable storage medium having stored thereon instructions which, when executed, cause one or more processors to: receiving a geometry bitstream comprising encoded geometry data of the point cloud data encoded according to a first encoding process; Decoding the encoded geometric data using a first decoding process that is opposite to the first encoding process to generate reconstructed geometric data; receiving an attribute bitstream, the attribute bitstream comprising encoded attribute data of the point cloud data encoded according to a second encoding process; as well as The encoded attribute data is decoded based on the reconstructed geometric data using a second decoding process that is opposite to the second encoding process to generate decoded attribute data and reconstructed geometric data of the point cloud data.

Images