Device for decoding point cloud data, and method for decoding point cloud data

Description

1 / 83 “DEVICE FOR DECODING POINT CLOUD DATA, AND, Point Cloud Data Decoding Method

[0001] This application is a U.S. utility priority PCT application for U.S. Patent No. 18 / 632,140, ​​filed April 10, 2024, which claims the benefit of U.S. Provisional Patent Application No. 63 / 495,833, filed April 13, 2023, each of which has its entire content incorporated into the present invention by reference. TECHNICAL FIELD

[0002] This disclosure relates to point cloud encoding and decoding. BACKGROUND

[0003] A point cloud is a collection of points in three-dimensional space. The points can correspond to points on objects in three-dimensional space. In this way, a point cloud can be used to represent the physical content of three-dimensional space. Point clouds can be useful in a wide variety of situations. For example, point clouds can be used in the context of autonomous vehicles to represent the positions of objects on a highway. In another example, point clouds can be used in the context of representing the physical content of an environment for the purposes of positioning virtual objects in an augmented reality (AR) or mixed reality (MR) application. Point cloud compression is a process for encoding and decoding point clouds. Encoding point clouds can reduce the amount of data required for storing and transmitting point clouds. Petition 870250089998, dated 02 / 10 / 2025, p. 8 / 206 2 / 83 SUMMARY

[0004] In general, this disclosure describes techniques for point cloud coding (e.g., encoding or decoding), including coding geometry and attribute data from point cloud data. In particular, geometry information (e.g., point coordinates within the point cloud) can be efficiently coded using a deep learning-based coder. Attribute data for the points (e.g., color, reflectance, brightness, surface normals, or the like) generally includes a larger amount of data than geometry data. Therefore, the point cloud coder can reconstruct the geometry data, and then downscale the geometry data and also downscale the attribute data before encoding the attribute data.A point cloud decoder can then decode and reconstruct the geometry data at full scale, downscale the geometry data, and decode the attribute data using the downscaled geometry data. According to the techniques in this disclosure, the decoder can apply a residual learning network to the decoded attribute data in order to oversample the attribute data to the original full scale. In this way, the techniques in this disclosure can achieve significant bit savings when transporting encoded attribute data, while simultaneously reconstructing the full-scale point cloud with high accuracy.

[0005] In one example, a device for decoding point cloud data includes: a memory configured to store point cloud data; and a or Petition 870250089998, dated 02 / 10 / 2025, page 9 / 206 3 / 83 more processors implemented in the circuit set and configured to: decode encoded point cloud geometry data into a point cloud in order to form reconstructed point cloud geometry data for the point cloud; downscale the point cloud geometry data to form downscaled point cloud geometry data; decode attribute data into the point cloud using the downscaled point cloud geometry; apply the attribute data to the reconstructed point cloud geometry data to form intermediate point cloud data; and apply a residual learning network to the intermediate point cloud data to form a reconstructed point cloud.

[0006] In another example, a point cloud data decoding method includes: decoding encoded point cloud geometry data into a point cloud in order to form reconstructed point cloud geometry data for the point cloud; downscaling the point cloud geometry data to form downscaled point cloud geometry data; decoding attribute data into the point cloud using the downscaled point cloud geometry; applying the attribute data to the reconstructed point cloud geometry data to form intermediate point cloud data; and applying a residual learning network to the intermediate point cloud data to form a reconstructed point cloud.

[0007] In another example, a computer-readable storage medium has instructions stored within it that, when executed, perform a Petition 870250089998, dated 02 / 10 / 2025, page 10 / 206 4 / 83 Processor: Decode encoded point cloud geometry data into a point cloud in order to form reconstructed point cloud geometry data for the point cloud; downscale the point cloud geometry data to form downscaled point cloud geometry data; decode attribute data into the point cloud using the downscaled point cloud geometry; apply the attribute data to the reconstructed point cloud geometry data to form intermediate point cloud data; and apply a residual learning network to the intermediate point cloud data to form a reconstructed point cloud.

[0008] In another example, a device for decoding point cloud data includes: means for decoding encoded point cloud geometry data to a point cloud in order to form reconstructed point cloud geometry data for the point cloud; means for downscaling the point cloud geometry data to form downscaled point cloud geometry data; means for decoding attribute data to the point cloud using downscaled point cloud geometry; means for applying the attribute data to the reconstructed point cloud geometry data to form intermediate point cloud data; and means for applying a residual learning network to the intermediate point cloud data to form a reconstructed point cloud.

[0009] The details of one or more examples are set out in the attached drawings and in the description below. Petition 870250089998, dated 02 / 10 / 2025, page 11 / 206 5 / 83 Other attributes, objectives, and advantages will become evident from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a block diagram illustrating an example point cloud encoding and decoding system, which can perform the techniques of this disclosure.

[0011] Figure 2 is a block diagram illustrating an example point cloud encoder according to the techniques of this disclosure.

[0012] Figure 3 is a block diagram illustrating an example point cloud decoder according to the techniques of this disclosure.

[0013] Figure 4 is a conceptual diagram illustrating an example coding structure according to certain examples of the techniques in this disclosure.

[0014] Figure 5 is a conceptual diagram illustrating an example decoding structure according to certain examples of the techniques in this disclosure.

[0015] Figure 6 is a block diagram illustrating an example point cloud coding structure according to the techniques of this disclosure.

[0016] Figure 7 is a block diagram illustrating an example point cloud decoding structure according to the techniques of this disclosure.

[0017] Figures 8A and 8B are conceptual diagrams illustrating examples of voxel downscaling of a point cloud.

[0018] Figure 9 is a block diagram illustrating a set of example stages that can be Petition 870250089998, dated 02 / 10 / 2025, page 12 / 206 6 / 83 included in a deep learning-based attribute supersampler.

[0019] Figure 10 is a block diagram illustrating another set of example stages that can be included in a deep learning-based feature supersampler.

[0020] Figures 11A and 11B are conceptual diagrams illustrating examples of residual learning networks according to the techniques of this disclosure.

[0021] Figure 12 is a conceptual diagram illustrating an example of a residual learning network for point clouds according to the techniques of this disclosure.

[0022] Figure 13 is a conceptual diagram illustrating an example of a residual learning network employing a network as a filter to remove noise from point clouds according to the techniques of this disclosure.

[0023] Figure 14 is a conceptual diagram illustrating an example decoding structure according to the techniques of this disclosure.

[0024] Figure 15 is a flowchart illustrating a method for encoding example point cloud data according to the techniques of this disclosure.

[0025] Figure 16 is a flowchart illustrating a method for decoding example point cloud data according to the techniques of this disclosure.

[0026] Figure 17 is a conceptual diagram illustrating a laser package, such as a LIDAR sensor or other system that includes one or more lasers, scanning points in three-dimensional space. Petition 870250089998, dated 02 / 10 / 2025, page 13 / 206 7 / 83

[0027] Figure 18 is a conceptual diagram illustrating an example 900 range measurement system that can be used with one or more techniques of this disclosure.

[0028] Figure 19 is a conceptual diagram illustrating a vehicle-based example scenario in which one or more techniques from this disclosure can be used.

[0029] Figure 20 is a conceptual diagram illustrating an example extended reality system in which one or more techniques from this disclosure can be used.

[0030] Figure 21 is a conceptual diagram illustrating an example mobile device system in which one or more techniques from this disclosure can be used. DETAILED DESCRIPTION

[0031] A point cloud (PC) is a three-dimensional data representation for tasks such as virtual reality (VR) and mixed reality (MR), autonomous driving, cultural heritage, etc. Point clouds are a collection of points in three-dimensional space, represented by their three-dimensional coordinates (x, y, z) called the geometry. Each point can also be associated with multiple attributes, such as color, normal vectors, and reflectance. Depending on the target application and point cloud acquisition methods, the point cloud can be categorized into point cloud scenes and point cloud objects. Point cloud scenes can be captured using LiDAR sensors and can be dynamically acquired.

[0032] Point cloud objects can be subdivided into static point clouds and dynamic point clouds. A static point cloud is a single object. A Petition 870250089998, dated 02 / 10 / 2025, p. 14 / 206 8 / 83 A dynamic point cloud is a time-varying point cloud, including a sequence of point cloud instances. Each instance of a dynamic point cloud is a static point cloud. Time-varying dynamic point clouds can be used in AR / VR, volumetric video streaming, and telepresence, and can be generated using three-dimensional models, i.e., CGI, or captured from real-world scenarios using various methods, such as multiple cameras with depth sensors surrounding the object. These point clouds are dense, photorealistic point clouds that can have a huge number of points, especially in high-precision or large-scale captures (millions of points per frame at up to 60 frames per second (FPS)). Therefore, efficient point cloud compression (PCC) is useful to enable practical use in VR and MR applications.

[0033] The Moving Picture Experts Group (MPEG) has approved two PCC (point cloud compression) standards: (1) S. Schwarz, M. Preda, V. Baroncini, M. Budagavi, P. Cesar, PA Chou, RA Cohen, M. Krivoku'ca, S. Lasserre, Z. Li et al., Emerging MPEG standards for point cloud compression, IEEE Journal on Emerging and Selected Topics in Circuits and Systems, volume 9, no. 1, pages 133 to 148, 2018, and (2) D. Graziosi, O. Nakagami, S. Kuma, A. Zaghetto, T. Suzuki, and A. Tabatabai, An overview of ongoing point cloud compression standardization activities: Video-based (v-pcc) and geometrybased (g-pcc), APSIPA Transactions on Signal and Information Processing, volume 9, 2020. MPEG approved the geometry-based point cloud compression standard (G-PCC Petition 870250089998, dated 10 / 02 / 2025, p. 15 / 206). 9 / 83 geometry-based point cloud compression): MPEG-PCC-TMC13: Geometry Based Point Cloud Compression G-PCC, 2021, available at github.com / MPEGGroup / mpeg-pcc-tmc13. MPEG approved video-based point cloud compression (VPCC): MPEG-PCC-TMC2: Video Based Point Cloud Compression VPCC, 2022, available at github.com / MPEGGroup / mpeg-pcc-tmc2.

[0034] G-PCC includes octree geometry coding as a generic geometry coding tool and a predictive (tree-based) geometry coding tool that targets LiDAR-based point clouds. G-PCC is still developing a method based on triangle meshes or triangle soup (trisoup) to approximate the surface of the three-dimensional model. V-PCC, on the other hand, encodes dynamic point clouds by projecting three-dimensional points onto a two-dimensional plane and then using video codecs, for example, high-efficiency video coding (HEVC), to encode each frame over time. MPEG also proposed common test conditions (CTCs) for evaluating test models: S. Schwarz, G. Martin-Cocher, D. Flynn and M. Budagavi, Common test conditions for point cloud compression, ISO / IECJTC1 / SC29 / WG11 w17766 document, Ljubljana, Slovenia, 2018.

[0035] As mentioned above, efficient point cloud compression is useful for applications such as virtual and mixed reality, autonomous driving, and cultural heritage. Some techniques, such as m59617: Anique Akhtar, Zhu Li, Geert Van der Auwera, Adarsh ​​Krishnan Ramasubramonian, Luong Pham Van, Marta Karczewicz, Dynamic Point Cloud Petition 870250089998, dated 02 / 10 / 2025, page 16 / 206 10 / 83 Geometry Compression using Sparse Convolutions, MPEG-137 Online, document m59617, April 2022, and m60307: Anique Akhtar, Zhu Li, Geert Van der Auwera, Adarsh ​​Krishnan Ramasubramonian, Marta Karczewicz, [AI-3DGC][EE5.3 Test 2] Results dynamic point cloud compression, MPEG-139 Online, document M60307, ​​July 2022, use deep learning-based point cloud compression for dense dynamic point clouds using a deep learning network consisting of an encoder and decoder module.

[0036] Research on performing point cloud compression using deep learning solutions is ongoing. A point cloud typically includes a collection of points as well as attributes for the points. The points correspond to positions in a three-dimensional space, for example, having X, Y, and Z coordinates. The attributes may include, for example, color, reflectance, brightness, surface normals, or the like. Because point clouds have both geometry and attributes, solutions have been proposed for point cloud geometry compression, point cloud attribute compression, as well as joint compression of point cloud geometry and attributes.

[0037] Deep learning-based solutions typically perform well when applied to geometry compression. The techniques in this disclosure include using geometry compression from one codec and attribute compression from another codec. For example, a deep learning-based geometry compression scheme can be combined with a non-deep learning-based attribute compression scheme. These techniques can provide flexibility to the compression framework and Petition 870250089998, dated 02 / 10 / 2025, page 17 / 206 11 / 83 established a strong baseline for deep learning-based point cloud attribute compression.

[0038] These techniques may additionally include the use of multiscale attribute compression with deep learning-based postprocessing. Heuristic tests have shown that most of the compression bits are consumed by attribute encoding. Therefore, creating an effective attribute compression scheme is important to achieve good compression performance. The multiscale attribute compression scheme can improve the overall compression performance of the encoding framework. In particular, according to the techniques in this disclosure, after decoding attribute data at a reduced scale, a residual learning network can scale the attribute data up to the original full-scale resolution. In this way, significant bitrate savings can be achieved when transporting the attribute data, while achieving high accuracy when scaling the attribute data up to the original full-scale.

[0039] Some coding techniques include a lossy point cloud geometry compression scheme based on deep learning for dynamic point cloud compression. The lossy geometry scheme predicts the latent representation of a current frame using a previous frame by employing a prediction network. The framework performs inter-frame point cloud coding of frame P, where the current frame is coded with reference to the previously decoded frame. The architecture is implemented using a sparse convolutional neural network (CNN) with sparse tensors. Petition 870250089998, dated 02 / 10 / 2025, page 18 / 206 12 / 83 The architecture employs convolution on target coordinates to map the latent representation of the previous frame to the subsampled coordinates of the current frame to predict the embedding of current frame attributes. The structure transmits the residual of the predicted attributes and the actual attributes by compressing them using a learned probabilistic factored entropy model. Compared to G-PCC and V-PCC, these techniques demonstrate better geometry compression performance in dense point clouds with efficient encoding / decoding runtime.

[0040] In one or more examples, this disclosure describes a flexible configuration of the deep learning-based framework where, instead of having a joint geometry and attribute compression scheme, the example techniques use a recoloring scheme (as an example) to generate attributes for the reconstructed point cloud and employ traditional attribute compression schemes. One or more of the example techniques described in this disclosure may utilize the G-PCC recoloring scheme to obtain attributes of the target point cloud from the source point cloud. The G-PCC recoloring scheme employs nearest neighbors based on weighted distance in the source point cloud to calculate attributes of the target point cloud. Consequently, in one or more examples, the example techniques may allow the use of attribute compression from a different codec, and geometry compression from a different codec.

[0041] Figure 1 is a block diagram illustrating an example encoding and decoding system 100 that can perform the techniques of this disclosure. As Petition 870250089998, dated 02 / 10 / 2025, page 19 / 206 13 / 83 The techniques in this disclosure generally refer to the process of encoding (e.g., encoding and / or decoding) point cloud data, that is, to support point cloud compression. In general, point cloud data includes any data for processing a point cloud. The encoding process can be effective in compressing and / or decompressing point cloud data.

[0042] As shown in Figure 1, the system 100 includes a source device 102 and a destination device 116. The source device 102 provides encoded point cloud data to be decoded by a destination device 116. In particular, in the example in Figure 1, the source device 102 provides the point cloud data to the destination device 116 via a computer-readable medium 110. The source device 102 and the destination device 116 may comprise any of a wide range of devices, including desktop computers, notebook computers (i.e., laptop computers), tablet computers, set-top boxes, telephone handsets such as smartphones, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, land or sea vehicles, spacecraft, aircraft, robots, LIDAR devices, satellites, or similar devices.In some cases, source device 102 and destination device 116 can be equipped for wireless communication.

[0043] In the example in Figure 1, the source device 102 includes a data source 104, a memory 106, a point cloud encoder 200, and an output interface 108. The destination device 116 includes an interface of Petition 870250089998, dated 02 / 10 / 2025, p. 20 / 206 14 / 83 input 122, a point cloud decoder 300, a memory 120, and a data consumer 118. According to this disclosure, the point cloud encoder 200 of source device 102 and the point cloud decoder 300 of destination device 116 can be configured to apply the techniques of this disclosure related to attribute encoding for point cloud compression. Thus, source device 102 represents an example of an encoding device, while destination device 116 represents an example of a decoding device. In other examples, source device 102 and destination device 116 may include other components or arrangements. For example, source device 102 may receive data (e.g., point cloud data) from an internal or external source.Similarly, the target device 116 can interface with an external data consumer, instead of including a data consumer on the same device.

[0044] As shown in Figure 1, system 100 is merely an example. In general, other digital encoding and / or decoding devices can perform the techniques of this disclosure related to attribute encoding for point cloud compression. Source device 102 and destination device 116 are merely examples of devices in which source device 102 generates encoded data for transmission to destination device 116. This disclosure refers to an encoding device as a device that performs encoding (encoding and / or decoding) of data. Thus, point cloud encoder 200 and point cloud decoder 200 Petition 870250089998, dated 02 / 10 / 2025, page 21 / 206 15 / 83 of points 300 represent examples of encoding devices, in particular, an encoder and a decoder, respectively. In some examples, the source device 102 and the destination device 116 may operate in a substantially symmetrical manner, such that each of the source device 102 and the destination device 116 includes encoding and decoding components. In this way, the system 100 can support unidirectional or bidirectional transmission between the source device 102 and the destination device 116, for example, for streaming, playback, broadcast, telephony, navigation, and other applications.

[0045] In general, data source 104 represents a data source (i.e., raw, unencoded point cloud data) and can provide a sequential series of data frames to the point cloud encoder 200, which encodes the data into frames. Data source 104 of source device 102 may include a point cloud capture device, such as any of a variety of cameras or sensors, for example, a three-dimensional scanning device or a light detection and ranging (LIDAR) device, one or more video cameras, a file containing previously captured data, and / or a data feed interface for receiving data from a data content provider. Alternatively or additionally, point cloud data may be computer-generated from a scanner, camera, sensor, or other data type.For example, data source 104 can generate data based on computer graphics, like the source data, or produce a combination of live data, archived data, and computer-generated data. In each case... Petition 870250089998, dated 02 / 10 / 2025, page 22 / 206 16 / 83 In this case, the point cloud encoder 200 encodes captured, pre-captured, or computer-generated data. The point cloud encoder 200 can rearrange the frames from the received order (sometimes called the display order) into an encoding order for the encoding process. The point cloud encoder 200 can generate one or more bitstreams including encoded data. The source device 102 can then output the encoded data via the output interface 108 to the computer-readable medium 110 for reception and / or retrieval, for example, via the input interface 122 of the destination device 116.

[0046] Memory 106 of source device 102 and memory 120 of destination device 116 may represent general-purpose memories. In some examples, memory 106 and memory 120 may store raw data, for example, raw data from data source 104 and raw and decoded data from point cloud decoder 300. Additionally or alternatively, memory 106 and memory 120 may store executable software instructions, for example, by point cloud encoder 200 and point cloud decoder 300, respectively. Although memory 106 and memory 120 are shown separately from point cloud encoder 200 and point cloud decoder 300 in this example, it should be understood that point cloud encoder 200 and point cloud decoder 300 may also include internal memories for functionally similar or equivalent purposes.In addition, memory 106 and memory 120 can store encoded data, for example, the output of point cloud encoder 200 and the input to point cloud decoder. Petition 870250089998, dated 02 / 10 / 2025, page 23 / 206 17 / 83 300. In some examples, portions of memory 106 and memory 120 may be allocated as one or more buffers, for example, to store raw, decoded, and / or encoded data. For example, memory 106 and memory 120 may store data representing a point cloud.

[0047] The computer-readable medium 110 may represent any type of medium or device capable of carrying encoded data from the source device 102 to the destination device 116. In one example, the computer-readable medium 110 represents a communication medium to enable the source device 102 to transmit encoded data directly to the destination device 116 in real time, for example, via a radio frequency network or computer-based network. The output interface 108 may modulate a transmission signal that includes the encoded data, and the input interface 122 may demodulate the received transmission signal according to a communication standard, such as a wireless communication protocol. The communication medium may comprise any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines.The communication medium can be part of a packet-based network, such as a local area network, a wide area network, or a global network, such as the Internet. The communication medium may include routers, switches, base stations, or any other equipment that may be useful for facilitating communication between the source device 102 and the destination device 116.

[0048] In some examples, source device 102 may output encoded data from output interface 108 Petition 870250089998, dated 02 / 10 / 2025, page 24 / 206 18 / 83 for storage device 112. Similarly, destination device 116 can access encoded data from storage device 112 via input interface 122. Storage device 112 may include any of several distributed or locally accessed data storage media, such as a hard disk, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other digital storage media suitable for storing encoded data.

[0049] In some examples, source device 102 may output encoded data to file server 114 or another intermediate storage device that can store the encoded data generated by source device 102. Destination device 116 may access stored data from file server 114 via streaming or download. File server 114 may be any type of server device with the ability to store encoded data and transmit that encoded data to destination device 116. File server 114 may represent a web server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device.The target device 116 can access encrypted data from the file server 114 through any standard data connection, including an internet connection. This can include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., line). Petition 870250089998, dated 02 / 10 / 2025, page 25 / 206 19 / 83 digital subscriber (DSL - digital subscriber line), cable modem, etc.) or a combination of both that is suitable for accessing encrypted data stored on file server 114. File server 114 and input interface 122 can be configured to operate according to a streaming protocol, a download protocol, or a combination thereof.

[0050] Output interface 108 and input interface 122 may represent wireless transmitters / receivers, modems, wired network communication components (e.g., Ethernet cards), wireless communication components operating in accordance with any of a variety of Institute of Electrical and Electronics Engineers (IEEE) 802.11 standards, or other physical components. In examples where output interface 108 and input interface 122 comprise wireless components, output interface 108 and input interface 122 may be configured to transfer data, such as encoded data, in accordance with a cellular communication standard such as fourth generation (4G), 4G-LTE (long-term evolution), LTE-advanced, fifth generation (5G), or similar.In some examples where output interface 108 comprises a wireless transmitter, output interface 108 and input interface 122 can be configured to transfer data, such as encoded data, according to other wireless standards, such as an IEEE 802.11 specification, an IEEE 802.15 specification (e.g., ZigBee™), a Bluetooth™ standard, or similar. In some examples, the... Petition 870250089998, dated 02 / 10 / 2025, page 26 / 206 20 / 83 source device 102 and / or destination device 116 may include their respective system-on-a-chip (SoC) devices. For example, source device 102 may include a SoC device to perform the functionality assigned to media encoder 200 and / or output interface 108, and destination device 116 may include a SoC device to perform the functionality assigned to point cloud decoder 300 and / or input interface 122.

[0051] The techniques of this disclosure can be applied to encode and decode in support of any of a variety of applications, such as communication between autonomous vehicles, communication between scanners, cameras, sensors and processing devices, such as local or remote servers, geographic mapping or other applications.

[0052] The input interface 122 of the destination device 116 receives an encoded bitstream from the computer-readable medium 110 (e.g., a communication medium, storage device 112, file server 114, or similar). The encoded bitstream may include signaling information defined by the point cloud encoder 200, which is also used by the point cloud decoder 300, as syntax elements having values ​​that describe the characteristics and / or processing of encoded units (e.g., slices, images, image groups, sequences, or similar). The data consumer 118 uses the decoded data. For example, the data consumer 118 may use the decoded data to determine the locations of physical objects. In some examples, the Petition 870250089998, dated 02 / 10 / 2025, page 27 / 206 21 / 83 data consumer 118 can understand a display to present images based on a point cloud.

[0053] Each of the 200 point cloud encoder and 300 point cloud decoder can be implemented as any one of a variety of suitable encoder and / or decoder circuit sets, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the techniques are implemented partially in software, a device may store instructions for the software in a suitable non-transient, computer-readable medium and execute the instructions in hardware using one or more processors to perform the techniques of this disclosure.Each of the 200 point cloud encoder and 300 point cloud decoder can be included in one or more encoders or decoders, any of which can be integrated as part of a combined encoder / decoder (CODEC encoder / decoder) in a respective device. A device that includes the 200 point cloud encoder and / or the 300 point cloud decoder may comprise one or more integrated circuits, microprocessors, and / or other types of devices.

[0054] The 200 point cloud encoder and the 300 point cloud decoder can operate according to an encoding standard, such as a compression standard of Petition 870250089998, dated 02 / 10 / 2025, page 28 / 206 22 / 83 Video point cloud (V-PCC) or a geometry point cloud compression standard (G-PCC). This disclosure may refer generally to the process of encoding (e.g., encoding and decoding) images to include the process of encoding or decoding data. An encoded bitstream generally includes a series of values ​​for syntax elements representing decisions of the encoding process (e.g., encoding modes).

[0055] This disclosure may generally refer to the signaling of certain information, such as syntax elements. The term signaling may generally refer to the communication of values ​​for syntax elements and / or other data used to decode the encoded data. That is, the point cloud encoder 200 may signal values ​​for syntax elements in the bitstream. In general, signaling refers to the generation of a value in the bitstream. As mentioned above, the source device 102 may transport the bitstream to the destination device 116 substantially in real time or in non-real time, as may occur during the storage of syntax elements in the storage device 112 for later retrieval by the destination device 116.

[0056] The ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) is studying the potential need for standardization of point cloud encoding technology with a compression capability that significantly exceeds that of current approaches and aims to create the standard. The group is working together on this exploratory activity in a collaborative effort known as the 3-Dimensional Graphics Team (3DG). Petition 870250089998, dated 02 / 10 / 2025, page 29 / 206 23 / 83 evaluate the compression technology designs proposed by their experts in this area.

[0057] Point cloud compression activities are categorized into two different approaches. The first approach is video point cloud compression (V-PCC), which segments the three-dimensional object and projects the segments onto multiple two-dimensional planes (which are represented as patches in the two-dimensional frame), which are additionally encoded by a legacy two-dimensional video codec, such as a High Efficiency Video Coding (HEVC) codec (ITU-T H.265). The second approach is geometry-based point cloud compression (G-PCC), which directly compresses the three-dimensional geometry, i.e., the position of a set of points in three-dimensional space, and the associated attribute values ​​(for each point associated with the three-dimensional geometry). G-PCC addresses point cloud compression in category 1 (static point clouds) and category 3 (dynamically acquired point clouds).A recent preliminary version of the G-PCC standard is available in G-PCC DIS, ISO / IEC JTC1 / SC29 / WG11 w19088, Brussels, Belgium, January 2020, and a description of the codec is available in G-PCC Codec Description v6, ISO / IEC JTC1 / SC29 / WG11 w19091, Brussels, Belgium, January 2020.

[0058] A point cloud contains a set of points in three-dimensional space, and may have attributes associated with the point. The attributes may be color information, such as R, G, B or Y, Cb, Cr, or reflectance information, or other attributes. Point clouds can be captured by various cameras or sensors, such as sensors Petition 870250089998, dated 02 / 10 / 2025, p. 30 / 206 24 / 83 Point cloud data is generated by LIDAR and three-dimensional scanning devices, and can also be computer-generated. Point cloud data is used in various applications including, but not limited to, construction (modeling), graphics (three-dimensional models for visualization and animation), and the automotive industry (LIDAR sensors used to aid navigation).

[0059] The three-dimensional space occupied by point cloud data can be enclosed by a virtual bounding box. The position of the points in the bounding box can be represented with a certain precision; therefore, the positions of one or more points can be quantified based on precision. At the smallest level, the bounding box is divided into voxels, which are the smallest unit of space represented by a unit cube. A voxel in the bounding box can be associated with zero, one, or more than one point. The bounding box can be divided into multiple cube / cuboid regions, which can be called tiles. Each tile can be encoded into one or more slices. The partitioning of the bounding box into slices and tiles can be based on the number of points in each partition or can be based on other considerations (e.g., a particular region can be encoded as tiles).The slice regions can be further partitioned using splitting decisions similar to those in video codecs.

[0060] Figure 2 is a block diagram illustrating an example 200 point cloud encoder. The modules shown are logical and do not necessarily correspond, one-to-one, to the code implemented in the reference implementation of the G-PCC codec, i.e., the TMC13 test model software studied by ISO / IEC MPEG (JTC 1 / SC 29 / WG 11). Petition 870250089998, dated 02 / 10 / 2025, page 31 / 206 25 / 83

[0061] In both the 200 point cloud encoder and the 300 point cloud decoder, point cloud positions are encoded first. Attribute encoding depends on the decoded geometry. Compressed geometry is typically represented as an octree from the root down to a leaf level of individual voxels.

[0062] At each node of an octree, an occupancy is signaled (when not inferred) for one or more of its child nodes (up to eight nodes). Multiple neighborhoods are specified, including (a) nodes that share a face with a current octree node, (b) nodes that share a face, edge, or vertex with the current octree node, etc. In each neighborhood, the occupancy of a node and / or its children can be used to predict the occupancy of the current node or its children. For points that are sparsely populated at certain nodes in the tree, the codec also supports a direct encoding mode, where the three-dimensional position of the point is encoded directly. A flag can be signaled to indicate that a direct mode is signaled. At the lowest level, the number of points associated with the node / leaf node of the octree can also be encoded.

[0063] Once the geometry is encoded, the attributes corresponding to the geometry points are encoded. When there are multiple attribute points corresponding to a reconstructed / decoded geometry point, an attribute value can be derived, which is representative of the reconstructed point.

[0064] There are three attribute encoding methods in G-PCC: region adaptive hierarchical transform (RAHT) encoding, hierarchical nearest neighbor prediction Petition 870250089998, dated 02 / 10 / 2025, page 32 / 206 26 / 83 based on interpolation (predictive transform) and hierarchical nearest neighbor prediction based on interpolation with an update / elevation step (elevation transform). RAHT and elevation are typically used for category 1 data, while prediction is typically used for category 3 data. However, either method can be used for any data, and as with geometry codecs in G-PCC, the attribute encoding method used to encode the point cloud is specified in the bitstream.

[0065] Attribute encoding can be conducted at a level of detail (LOD), where, with each level of detail, a finer representation of the point cloud attribute can be obtained. Each level of detail can be specified based on the distance metric from neighboring nodes or based on a sampling distance.

[0066] In the 200 point cloud encoder, the residuals obtained as output from the coding methods for the attributes are quantized. The residuals can be obtained by subtracting the attribute value from a prediction that is derived based on points in the neighborhood of the current point and based on the attribute values ​​of previously coded points. The quantized residuals can be coded using context-adaptive arithmetic coding.

[0067] In the example in Figure 2, the point cloud encoder 200 includes deep learning-based geometry encoder 202, geometry reconstruction unit 206, geometry downscaling unit 210, recoloring unit 212, color transformation unit 204, adaptive hierarchical transform unit Petition 870250089998, dated 02 / 10 / 2025, page 33 / 206 27 / 83 per region (RAHT) 218, LOD generation unit 220, elevation unit 222, coefficient quantization unit 224 and arithmetic coding unit 226.

[0068] As shown in the example in Figure 2, the point cloud encoder 200 can obtain a set of point positions in the point cloud and a set of attributes. The point cloud encoder 200 can obtain the set of point positions in the point cloud and the set of attributes from data source 104 (Figure 1). Positions can include coordinates of points in a point cloud. Attributes can include information about the points in the point cloud, such as colors, reflectance, intensity, or similar, associated with the points in the point cloud. The deep learning-based geometry encoder 202 of the point cloud encoder 200 can generate a geometry bitstream 203 that includes an encoded representation of the positions of the points in the point cloud. The point cloud encoder 200 can also generate an attribute bitstream 205 that includes an encoded representation of the attribute set.

[0069] After encoding the geometry information, the geometry reconstruction unit 206 can decode and reconstruct the geometry information. The geometry downscaling unit 210 can downscale the geometry information. As mentioned above, geometry information can be represented using an octree. A root node of the octree can be partitioned into eight subnodes. For each node of the octree, the node can be further partitioned into eight subnodes, including splitting the node in half along the Petition 870250089998, dated 02 / 10 / 2025, page 34 / 206 28 / 83 dimensions X, Y, and Z. This partitioning can continue until, for example, reaching a node of the smallest size for the octree, called a leaf node of the octree. That is, a leaf node has no child nodes and is unpartitioned.

[0070] In order to downscale the octree, the geometry downscaling unit 210 can determine that a node has eight leaf subnodes, and determine a number of those that are occupied among the eight leaf subnodes. If the number is above a threshold (for example, zero), the geometry downscaling unit 210 can represent the node as an occupied leaf node in a downscaled octree. Otherwise, if the number is less than or equal to the threshold, the geometry downscaling unit 210 can represent the node as an unoccupied leaf node in the downscaled octree. For example, if the threshold is zero, all eight leaf subnodes would need to be unoccupied to represent the node as an unoccupied leaf node in the downscaled octree; otherwise, the node would be represented as an occupied leaf node in the downscaled octree.

[0071] After downscaling the geometry information, recoloring unit 212 can apply attribute information to the scaled-down octree points. For example, recoloring unit 212 can similarly downscale the original attribute information to the same degree as the geometry information. Such downscaling may include selectively removing attribute data, merging attribute data, or otherwise reducing attribute data so that scaled-down attribute data can be applied to scaled-down geometry. Petition 870250089998, dated 02 / 10 / 2025, page 35 / 206 29 / 83

[0072] The color transformation unit 204 can transform the color information of attributes to a different domain. For example, the color transformation unit 204 can transform color information from an RGB color space to a YCbCr color space.

[0073] Furthermore, the RAHT 218 unit can apply RAHT encoding to the attributes of the reconstructed points. In some examples, under RAHT, the attributes of a 2x2x2 block of point positions are taken and transformed along one direction to obtain four low (L) and four high (H) frequency nodes. Subsequently, the four low frequency (L) nodes are transformed in a second direction to obtain two low (LL) and two high (LH) frequency nodes. The two low frequency (LL) nodes are transformed along a third direction to obtain one low frequency (LLL) and one high frequency (LLH) node. The low frequency LLL node corresponds to the DC coefficients and the high frequency H, LH, and LLH nodes correspond to the AC coefficients. The transformation in each direction can be a 1-D transformation with two coefficient weights.Low-frequency coefficients can be considered as 2x2x2 block coefficients for the next higher level of the RAHT transformation, and the AC coefficients are encoded without changes; such transformations continue up to the upper root node. The tree traversal for encoding is top-down and is used to calculate the weights to be used for the coefficients; the transformation order is bottom-up. The coefficients can then be quantized and encoded. Petition 870250089998, dated 02 / 10 / 2025, page 36 / 206 30 / 83

[0074] Alternatively or additionally, the LOD generation unit 220 and the elevation unit 222 can apply LOD processing and elevation, respectively, to the attributes of the reconstructed points. LOD generation is used to divide the attributes into different levels of refinement. Each level of refinement provides a refinement to the attributes of the point cloud. The first level of refinement provides a rough approximation and contains few points; the subsequent level of refinement typically contains more points, and so on. The levels of refinement can be constructed using a distance-based metric or can also use one or more other classification criteria (e.g., subsampling from a particular order). In this way, all reconstructed points can be included in a level of refinement.Each level of detail can be produced by taking a union of all points up to a particular refinement level: for example, LOD1 is obtained based on refinement level RL1, LOD2 is obtained based on RL1 and RL2, ... LODN is obtained by the union of RL1, RL2, ... RLN. In some cases, LOD generation can be followed by a prediction scheme (e.g., prediction transformation) where the attributes associated with each point in the LOD are predicted from a weighted average of the previous points, and the residual part is quantized and entropy-encoded. The elevation scheme is based on the prediction transformation mechanism, where an update operator is used to update the coefficients and an adaptive quantization of the coefficients is performed.

[0075] The RAHT 218 unit and the elevation 222 unit can generate coefficients based on the attributes. The unit Petition 870250089998, dated 02 / 10 / 2025, page 37 / 206 The 31 / 83 coefficient quantization unit 224 can quantize the coefficients generated by the RAHT unit 218 or the elevation unit 222. The arithmetic encoding unit 226 can apply arithmetic encoding to the syntax elements representing the quantized coefficients. The point cloud encoder 200 can output these syntax elements in the attribute bitstream 205. The attribute bitstream 205 can also include other syntax elements, including non-arithmetically encoded syntax elements.

[0076] In some examples, the geometry downscaling unit 210 may downscale the geometry data by a certain amount represented by a particular value, such as a downscaling factor. The point cloud encoder 200 may encode the value as a parameter of a parameter set, such as a sequence parameter set (SPS) or an attribute parameter set (APS), a slice header, a frame header, or other high-level syntax (HLS). In some examples, the downscaling factor may also indicate an amount by which to downscale the attribute data before recoloring. In some examples, the point cloud encoder 200 may encode a second value, separate from the first value, indicating the amount by which to downscale the attribute data.

[0077] Figure 3 is a block diagram illustrating an example 300 point cloud decoder. In the example in Figure 3, the 300 point cloud decoder includes a deep learning-based geometry decoder 302, an arithmetic decoding unit. Petition 870250089998, dated 02 / 10 / 2025, page 38 / 206 32 / 83 of attribute 304, geometry scaling unit 306, inverse quantization unit 308, RAHT unit 314, LoD generation unit 316, inverse elevation unit 318, inverse color transformation unit 322, and point cloud scaling unit 324.

[0078] The point cloud decoder 300 can receive a geometry bitstream 203 and an attribute bitstream 205. The deep learning-based geometry decoder 302 generally decodes geometry data from the geometry bitstream 203. The attribute arithmetic decoding unit 304 of the decoder 300 can apply arithmetic decoding (e.g., context-adaptive binary arithmetic coding (CABAC) or other type of arithmetic decoding) to syntax elements in the attribute bitstream 205 to decode the attribute bitstream 205.

[0079] In general, attribute bitstream 250 represents a scaled-down version of attribute data relative to geometry bitstream 203 geometry data. Therefore, geometry downscaling unit 306 can downscale the reproduced geometry data from the deep learning-based geometry decoder 302, for example, according to a downscaling value. Point cloud decoder 300 can decode the downscaling value from high-level syntax (HLS) data, such as a sequence parameter set (SPS), an attribute parameter set (APS), a slice header, a frame header, or the like. Geometry downscaling unit 306 Petition 870250089998, dated 02 / 10 / 2025, page 39 / 206 33 / 83 can reduce the scale of geometry data according to the scale reduction factor.

[0080] Additionally, the inverse quantization unit 308 can inversely quantize attribute values. The attribute values ​​can be based on syntax elements obtained from the attribute bitstream 205 (for example, including syntax elements decoded by the attribute arithmetic decoding unit 304).

[0081] Depending on how the attribute values ​​are encoded, the RAHT unit 314 can perform RAHT encoding to determine, based on the inversely quantized attribute values, the color values ​​for the point cloud points. RAHT decoding is done from top to bottom of the tree. At each level, the low- and high-frequency coefficients, which are derived from the inverse quantization process, are used to derive the constituent values. At the leaf node, the derived values ​​correspond to the attribute values ​​of the coefficients. The weight derivation process for the points is similar to the process used in the point cloud encoder 200. Alternatively, the LOD generation unit 316 and the inverse elevation unit 318 can determine color values ​​for point cloud points using a detail-based technique level.The LOD 316 generation unit decodes each LOD, resulting in progressively finer representations of the point attribute. With a prediction transformation, the LOD 316 generation unit derives the point prediction from a weighted sum of points that are in previous LODs or previously reconstructed in the same LOD. The LOD 316 generation unit can add the prediction to the residual (which is obtained after the... Petition 870250089998, dated 02 / 10 / 2025, page 40 / 206 34 / 83 inverse quantization) to obtain the reconstructed attribute value. When the elevation scheme is used, the LOD 316 generation unit can also include an update operator to update the coefficients used to derive the attribute values. The LOD 316 generation unit can, in this case, also apply inverse adaptive quantization.

[0082] In addition, the inverse color transformation unit 322 can apply an inverse color transformation to color values. The inverse color transformation can be an inverse of a color transformation applied by the color transformation unit 204 of the encoder 200. For example, the color transformation unit 204 can transform color information from an RGB color space to a YCbCr color space. Consequently, the inverse color transformation unit 322 can transform color information from the YCbCr color space to the RGB color space.

[0083] After decoding both geometry and attribute information, the 324 point cloud upscaling unit can reconstruct the point cloud. In particular, the 324 point cloud upscaling unit can upscale the attribute information to the scale of the geometry information. The 324 point cloud upscaling unit can upscale the attribute information according to the downscaling factor. Alternatively, the 300 point cloud decoder can decode a separate value representing an amount of upscaling to be applied to the attribute information. Finally, the 324 point cloud upscaling unit can apply the Petition 870250089998, dated 02 / 10 / 2025, page 41 / 206 35 / 83 attribute information scaled up to the geometry information points to reconstruct the point cloud.

[0084] According to the techniques of this disclosure, the 324 point cloud upscaling unit can be a deep learning-based residual learning network for attribute oversampling. That is, the 324 point cloud upscaling unit can initially be trained on data including original full-scale attribute data and corresponding downscaled attribute data, decoded and reconstructed. The 324 point cloud upscaling unit can be trained to upscale the downscaled attribute data, decoded and reconstructed to full scale, and compare the differences between the original full-scale attribute data and the upscaled attribute data (i.e., the residual) and update the internal network model accordingly (e.g., by changing the weighting values ​​within the model).The residual learning network can be considered trained when the differences between the original attribute data and the scaled-up attribute data are below a predetermined threshold value.

[0085] The various units in Figure 2 and Figure 3 are illustrated to help understand the operations performed by the encoder 200 and the decoder 300. The units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide particular functionality and are predefined in the operations that can be performed. The circuits Petition 870250089998, dated 02 / 10 / 2025, p. 42 / 206 36 / 83 Programmable circuits refer to circuits that can be programmed to perform various tasks and to provide flexible functionality in the operations that can be performed. For example, programmable circuits can execute software or firmware that causes the programmable circuits to operate in the manner defined by the software or firmware instructions. Fixed-function circuits can execute software instructions (for example, to receive parameters or to emit parameters), but the types of operations that fixed-function circuits perform are generally immutable. In some examples, one or more of the units may be distinct circuit blocks (fixed-function or programmable), and in some examples, one or more of the units may be integrated circuits.

[0086] As described above, machine learning techniques, such as deep learning, can be used for encoding and decoding point cloud data. The techniques in this disclosure include the use of a deep learning-based lossy point cloud geometry compression scheme for dynamic point cloud compression. A lossy geometry scheme predicts the latent representation of the current frame using the previous frame by employing a prediction network. The example techniques perform P-frame inter-frame point cloud encoding where the current frame is encoded with reference to a previously decoded frame. The architecture can be implemented using a sparse convolutional neural network (CNN) with sparse tensors. The example architecture employs convolution on target coordinates to map the Petition 870250089998, dated 02 / 10 / 2025, page 43 / 206 37 / 83 latent representation of the previous frame to the subsampled coordinates of the current frame to predict the embedding of attributes of the current frame. The encoder transmits the residual of the predicted attributes and the actual attributes by compressing them using a learned probabilistic factored entropy model. Compared to G-PCC and V-PCC, machine learning techniques show better compression performance in dense point clouds with efficient encoding / decoding runtime.

[0087] By employing downscaling and subsequent upscaling for attribute information, additional flexibility can be provided. Most of the bit rate in previous encoding schemes was consumed by attribute information. Any improvement in the attribute compression scheme can greatly improve the overall encoding efficiency of the structure. Thus, according to the techniques of this disclosure, the 200 point cloud encoder can downscale the attribute information before encoding, and the 300 point cloud decoder can decode and then upscale the attribute information.

[0088] High-level syntax (HLS) data can be signaled by pointcloud encoder 200 and received by pointcloud decoder 300. The HLS data may include a type of attribute encoding employed to encode the recolored attributes for pointcloud decoder 300 in order to reconstruct the attribute values. This encoding method, for example, the region-by-region adaptive hierarchical transform (RAHT) of G Petition 870250089998, dated 02 / 10 / 2025, page 44 / 206 38 / 83 Point Cloud Coefficient (PCC) can be signaled in a parameter set, for example, the Sequence Parameter Set (SPS) or the Attribute Parameter Set (APS), as an identifier. The part of the bitstream that carries the encoded attribute bits is, for example, a Natural Number Unit (NALU). A list of encoding methods can be specified, each encoding method providing a means to encode the point cloud attributes (optionally, they can also encode the geometry in a lossless manner). An index to this list can be signaled in the bitstream to indicate the encoding method used to encode the attributes. This index can be signaled in a parameter set (e.g., APS, SPS) or other means. When deep learning mechanisms are used for recoloring or decoding, the parameters / coefficients corresponding to the recoloring or decoding can also be signaled in the bitstream.

[0089] Figure 4 is a block diagram illustrating an example 400 encoding structure. Figure 5 is a block diagram illustrating an example 500 decoding structure. In Figure 4, the 400 encoding structure includes deep learning-based geometry encoder 404, G-PCC recoloring unit 406, and lossless and lossy G-PCC-based geometry encoder 408. In general, the 400 encoding structure can correspond to the 200 point cloud encoder of Figures 1 and 2, where the deep learning-based geometry encoder 404 can correspond to the deep learning-based geometry encoder 202, the G-PCC recoloring unit 406 can correspond to the recoloring unit 212, and the lossless geometry encoder and Petition 870250089998, dated 02 / 10 / 2025, page 45 / 206 39 / 83 of the lossy attribute by G-PCC 408 can correspond to any or all of the following: color transformation unit 204, RAHT unit 218, LOD generation unit 220, elevation unit 222, coefficient quantization unit 224, and arithmetic encoding unit 226. In this example, the original point cloud 402 is provided to both the deep learning-based geometry encoder 400 and the G-PCC recoloring unit 406. The deep learning-based geometry encoder 404 encodes geometry information from the original point cloud 402 and forms the geometry bitstream 410 including encoded geometry information for the original point cloud 402.

[0090] The deep learning-based geometry encoder 400 also decodes and reconstructs geometry information, for example, an octree including nodes that indicate whether points are present at the nodes (i.e., whether the nodes are occupied). Occupied nodes can be partitioned into eight subnodes, each of which can include indications of being occupied. The G-PCC recoloring unit 406 can receive the reconstructed geometry information from the deep learning-based geometry encoder 404 and use the geometry and attribute information from the original point cloud 402 to recolor the reconstructed geometry. The lossless and lossy G-PCC-based attribute geometry encoder 408 can then encode the recolored reconstructed geometry to form the attribute bitstream 412.

[0091] The 500 decoding structure in Figure 5 includes the deep learning-based geometry decoder 502 (which may correspond to the deep learning-based geometry decoder 302) and the Petition 870250089998, dated 02 / 10 / 2025, page 46 / 206 40 / 83 decoder by G-PCC 504 (which may correspond to attribute arithmetic decoding unit 304, inverse quantization unit 308, RAHT unit 314, LOD generation unit 316, inverse elevation unit 318, and inverse color transformation unit 322). In this example, the deep learning-based geometry decoder 502 receives a geometry bitstream 508. The deep learning-based geometry decoder 502 decodes the geometry bitstream 508 and reconstructs the geometry information. The G-PCC decoder 504 receives the attribute bitstream 510 and uses the reconstructed geometry information to decode the attribute bitstream 510 and reconstruct the point cloud. For example, the G-PCC 504 decoder can apply the decoded attribute information to the reconstructed geometry information to form the recolored reconstructed point cloud 506.

[0092] In Figures 4 and 5, the 400 encoding structure compresses and the 500 decoding structure decompresses the geometry using a deep learning-based encoder-decoder to obtain a reconstructed point cloud. The reconstructed point cloud differs in geometry from the original point cloud and may not simply be used as the attributes of the original point cloud. In some examples, the 400 encoding structure compresses and the 500 decoding structure may use the G-PCC recoloring scheme to alter the attributes of the original point cloud in order to create newer attributes for the reconstructed point cloud. The recolored point cloud has the geometry of the reconstructed point cloud and attributes that were derived from the original point cloud. Petition 870250089998, dated 02 / 10 / 2025, page 47 / 206 41 / 83 The recolored point cloud is encoded using G-PCC, where the geometry was coded in a lossless manner and the attributes were coded in a lossy manner. The reconstructed geometry is combined with reconstructed attributes to obtain the reconstructed point cloud.

[0093] The following changes / flexibility can be added to the structure illustrated in Figures 4 and 5. As an example, although Figures 4 and 5 illustrate deep learning-based encoders and decoders, the geometry encoder or decoder is not necessarily limited to a deep learning-based geometry encoder or decoder, and any geometry encoder or decoder can be employed.

[0094] For recoloring, the 400 encoding structure compresses and the 500 decoding structure may employ a recoloring scheme based on weighted distance-nearest-search employed in the G-PCC standard: WG 7, MPEG 3D Graphics Coding, G-PCC codec description, document N00271, January 2022. The recoloring scheme may alter attributes to fit the most recent geometry. Any recoloring scheme that can alter attribute values ​​and / or their correspondence with geometry may be employed as a recoloring scheme. The recoloring algorithm need not be limited to recoloring color attributes, e.g., RGB or YCbCr, but may more generally be an algorithm that recomputes attribute values, such as normal vectors, reflectance, etc., from point positions in one geometry to point positions in a second geometry. Petition 870250089998, dated 02 / 10 / 2025, p. 48 / 206 42 / 83 geometry. Deep learning mechanisms can also be applied to perform recoloring.

[0095] As illustrated in Figure 4, the lossless geometry and lossy attribute encoder using G-PCC is shown. The example techniques are not limited to lossless geometry and lossy attribute encoding using G-PCC, for example, the region-by-region adaptive hierarchical transform (RAHT) of G-PCC. This method can be employed with any encoding scheme, including other deep learning-based encoding or using V-PCC. Furthermore, instead of employing lossless geometry and lossy attribute encoding, the example can use any lossy attribute encoder. Deep learning mechanisms can also be applied to perform the decoding.

[0096] The 400 encoding structure and the 500 decoding structure use a geometry encoder / decoder from one codec and an attribute encoder / decoder from a separate codec to create a complete codec structure that would outperform the two individual codecs. In lossy point cloud compression, the geometry changes after compression, and linking attributes from a separate codec to their corresponding geometry is challenging. Thus, in the 400 encoding structure and the 500 point cloud decoding structure, a recoloring scheme is used to link attributes along with their corresponding geometry.

[0097] Figure 6 is a block diagram illustrating an example 600 point cloud coding structure according to the techniques of this disclosure. Petition 870250089998, dated 02 / 10 / 2025, page 49 / 206 43 / 83 In this example, the point cloud encoding structure 600 includes deep learning-based geometry encoder 604 (which may correspond to deep learning-based geometry encoder 202), geometry downscaling unit 606 (which may correspond to geometry downscaling unit 210), recoloring unit 608 (which may correspond to recoloring unit 212), and attribute encoder 610 (which may correspond to color transformation unit 204, RAHT unit 218, LOD generation unit 220, elevation unit 222, coefficient quantization unit 224, and arithmetic encoding unit 226).The 600 point cloud encoding structure differs from the 400 encoding structure in that the 600 point cloud encoding structure includes the geometry downscaling unit 606 and the attribute encoder 610 encodes a downscaling version of the attribute information, as discussed in more detail below.

[0098] In general, the deep learning-based geometry encoder 604 receives geometry information from the original point cloud 602, while the recoloring unit 608 receives both geometry and attribute information from the original point cloud 602. The deep learning-based geometry encoder 604 encodes the geometry information to form the geometry bitstream 612. The deep learning-based geometry encoder 604 decodes and reconstructs the geometry information and provides the reconstructed geometry information to the geometry downscaling unit 606. The geometry downscaling unit 606 can then downscale Petition 870250089998, dated 02 / 10 / 2025, page 50 / 206 44 / 83 scales the geometry information and provides scaled-down geometry information to the recoloring unit 608. The recoloring unit 608 can form a scaled-down recolored point cloud from the scaled-down geometry information and the original geometry and attribute information of the original point cloud 602. The attribute encoder 610 can then encode attribute information from the scaled-down recolored point cloud to form the attribute bitstream 614.

[0099] Figure 7 is a block diagram illustrating an example 700 point cloud decoding structure according to the techniques of this disclosure. In this example, the 700 point cloud decoding structure includes a deep learning-based geometry decoder 702, a geometry downscaling unit 704, an attribute decoder 706, and a deep learning-based attribute supersampler 708.The deep learning-based geometry decoder 702 can correspond to the deep learning-based geometry decoder 302, the geometry downscaling unit 704 can correspond to the geometry downscaling unit 306, the attribute decoder 706 can correspond to the arithmetic attribute decoding unit 304, inverse quantization unit 308, RAHT unit 314, LOD generation unit 316, inverse upscaling unit 318 and inverse color transformation unit 322, and the deep learning-based attribute supersampler 708 can correspond to the point cloud upscaling unit 324. Petition 870250089998, dated 02 / 10 / 2025, page 51 / 206 45 / 83

[0100] In general, the deep learning-based geometry decoder 702 receives a geometry bitstream 710. The deep learning-based geometry decoder 702 decodes geometry information from the geometry bitstream 710 and reconstructs geometry information from the decoded geometry information. The geometry downscaling unit 704 downscales the geometry information to form scaled-down geometry information and provides the scaled-down geometry information to the attribute decoder 706. The attribute decoder 706 receives the attribute bitstream 712, including encoded attribute information, and decodes the encoded attribute information.The attribute decoder 706 can provide the decoded attribute information to the deep learning-based attribute supersampler 708, which can also receive the original reconstructed geometry information and scale the attribute information up to the scale of the reconstructed geometry information. The deep learning-based attribute supersampler 708 can also apply the scaled-up attribute information to the reconstructed geometry information to form a reconstructed point cloud 714.

[0101] According to the techniques of this disclosure, the 610 attribute encoder encodes attributes at a reduced scale to save attribute bits, which can increase encoding efficiency. Geometry information is compressed using the 604 deep learning-based geometry encoder, and then decoded and reconstructed to obtain a reconstructed point cloud. Petition 870250089998, dated 02 / 10 / 2025, page 52 / 206 46 / 83 The geometry downscaling unit 606 can downscale the reconstructed geometry information using various downscaling factors (e.g., 1, 2, 4, 8, etc.) as discussed in more detail below. The recoloring unit 608 then recolors the downscaled geometry using the original point cloud attributes according to a recoloring scheme, such as the G-PCC recoloring scheme. The attribute encoder 610 can then encode the recolored point cloud attributes with an attribute encoding scheme (lossy or lossless), such as G-PCC RAHT or G-PCC predictive / elevation transformation.

[0102] In the point cloud decoding framework 700, the geometry bitstream 710 is decoded by the deep learning-based geometry decoder 702. The reconstructed geometry is scaled down by the geometry scale-down unit 704, and then provided to the attribute decoder 706. The attribute decoder 706 can perform inverse GPCC RAHT or predictive / uplift transformation. The attribute decoder 706 can link the scaled-down decoded attributes to their corresponding scaled-down reconstructed geometry to obtain a scaled-down point cloud. As an example, the deep learning-based point cloud attribute supersampler is employed to oversample the attributes and map the oversampled attributes to the reconstructed geometry.

[0103] In some examples, explicit scaling of the reconstructed geometry may not be performed. For example, the decoding process itself Petition 870250089998, dated 02 / 10 / 2025, page 53 / 206 47 / 83 geometry may include one or more scaled-down versions of geometry that are scaled up / processed to obtain the reconstructed geometry. The one or more scaled-down versions of geometry can be passed to the attribute decoder and used instead of the scaled-down reconstructed geometry. This can avoid the need to perform the explicit scale-down operation on the reconstructed geometry, thus saving processing time and resources.

[0104] Figures 8A and 8B are conceptual diagrams illustrating examples of voxel downscaling of a point cloud. Figure 8A depicts a sample voxel 800 including subvoxels 802A, 802B, and 802C, each of which is occupied, and other subvoxels are not occupied. In this example, downscaling voxel 800 results in the downscaled voxel 804 which is occupied, as subvoxels 802A, 802B, and 802C are occupied. Figure 8B depicts a sample voxel 810 including all unoccupied subvoxels. So, in this example, downscaling voxel 810 results in the downscaled voxel 812 which is unoccupied.

[0105] In some examples, the geometry downscaling unit 306 or the geometry downscaling unit 704 may employ a voxel grid downscaling of KxKxK, where K is a value that defines a downscaling factor. If K is 2, then the point cloud is divided into a grid of 2x2x2 voxels, where each voxel is a point in three-dimensional space. Then, the 8 voxels within the 2x2x2 voxel grid are integrated into a single voxel. In some examples, the final voxel is considered occupied if any of the 8 voxels in its Petition 870250089998, dated 02 / 10 / 2025, page 54 / 206 The 48 / 83 interior is also occupied. In some examples, the final voxel is considered occupied if the number of occupied subvoxels exceeds a threshold value.

[0106] Figure 9 is a block diagram illustrating a set of example stages that can be included in a deep learning-based attribute supersampler, such as the deep learning-based attribute supersampler 708 or the point cloud upscaling unit 324. In this example, the stage set includes deep learning layers 902, supersampling layer 910, and deep learning layers 920. Deep learning layers 902 initially process the geometry and attribute information of the scaled-down point cloud, providing results to the supersampling layer 910. The supersampling layer 910 then oversamples the results from the deep learning layers 902. Deep learning layers 920 then process the oversampled geometry and attribute information to produce the reconstructed point cloud 930.

[0107] Figure 10 is a block diagram illustrating a set of example stages that can be included in a deep learning-based attribute supersampler, such as the deep learning-based attribute supersampler 708 or the point cloud upscaling unit 324. In this example, the stage set includes deep learning layers 1002 (which may correspond to deep learning layers 902), supersampling layer 1010 (which may correspond to supersampling layer 910), and deep learning layers 1020 (which may correspond to deep learning layers). Petition 870250089998, dated 02 / 10 / 2025, page 55 / 206 49 / 83 920). In particular, the deep learning layers 102 include the 3x3x3 sparse convolutional (SConv) layer 1004, the inception residual block (IRB) layer 1006, and the 3x3x3 SConv layer 1008. In this example, the supersampling layer 1010 is a 5x5x5 convolutional layer in target coordinates. In this example, the deep learning layers 1020 include the 3x3x3 SConv layer 1022, the IRB layer 1024, and the SConv layer 1026. These layers process the geometry and attributes of the scaled-down point cloud sequentially to produce the reconstructed point cloud 1030.

[0108] Sparse convolutional layers with a kernel size of 3x3x3 are used in Figure 10 for example purposes, but other deep learning-based layers can be used additionally or alternatively. Similarly, although a convolution in target coordinates with a kernel size of 5x5x5 is used as an example, other supersampling techniques can be used in place of this layer. The convolution layer in target coordinates can be employed to map the attributes of the scaled-down geometry to the scaled-up geometry. If the scale-down / oversampling factor is large, multiple successive oversampling layers can be employed, or multiple successive attribute supersamplers can be used to oversample the attributes.

[0109] The feature supersampler does not necessarily need to be based on deep learning. Where a fully convolutional network using sparse convolutions is shown in the example in Figure 10, the same could be Petition 870250089998, dated 02 / 10 / 2025, page 56 / 206 50 / 83 achieved using different layers based on deep learning. Similarly, for the supersampling layer, similar results could be achieved using other deep learning layers, such as transposed convolutions, deconvolutions, degrouping, or the like.

[0110] The attributes in the structure are not restricted to color information. For example, these techniques can be applied to encode / oversample surface normal, reflectance, intensity, or similar information. Color space conversion can be applied to individual modules or to the entire structure. For example, the YCbCr color space can be employed during recoloring by the attribute encoder, the attribute decoder, and / or the deep learning-based attribute oversampler. The 200 point cloud encoder can encode syntax elements representing color spaces associated with individual modules or the entire structure in the bitstream, and similarly, the 300 point cloud decoder can decode and use the syntax elements to determine which color spaces should be used in which modules.

[0111] The 200 point cloud encoder and the 309 point cloud decoder can encode a scaling factor so that the 300 point cloud decoder determines how much adaptive geometry scaling should be performed and how much attribute oversampling to perform. If no scaling is performed in the 200 point cloud encoder, then there will be no need for scaling in the decoder, and the architecture shown in Figures 3 and 4 can be employed. Petition 870250089998, dated 02 / 10 / 2025, page 57 / 206 51 / 83 This downscaling factor can be signaled in a parameter set, a slice, or other syntax structure in the bitstream, for example, in the sequence parameter set (SPS) or the attribute parameter set (APS). Although called a subsampling factor (from the encoder's perspective), the 300 point cloud decoder can use this factor for other operations, including oversampling. In some examples, two downscaling factors may be signaled in the bitstream: one for geometry and one for attributes. In some examples, a subsampling factor may be signaled that applies to both geometry and attributes.

[0112] The type of attribute encoding technique employed to encode the recolored attributes can be signaled to the decoder side so that the 300 point cloud decoder reconstructs the attribute values. This encoding method, for example, the GPCC's region-by-region hierarchical adaptive transform (RAHT), can be signaled in a parameter set, for example, the sequence parameter set (SPS) or the attribute parameter set (APS) as an identifier. The bitstream portion carrying the encoded attribute bits can be, for example, a network abstraction layer unit (NALU).

[0113] A list of encoding techniques can be specified. Each encoding technique can provide a way to encode the attributes of the point cloud (optionally, they can also encode the geometry in a lossless manner). An index for this list can be Petition 870250089998, dated 02 / 10 / 2025, page 58 / 206 52 / 83 is signaled in the bitstream to indicate the encoding technique used to encode the attributes. This index can be signaled in a parameter set (e.g., APS, SPS) or in another HLS.

[0114] When deep learning mechanisms are used for recoloring or decoding, the parameters / coefficients corresponding to the recoloring or decoding can also be signaled in the bitstream.

[0115] According to the techniques in this disclosure, a deep learning-based feature supersampler can employ residual learning to reconstruct a higher-quality point cloud. An effective point cloud supersampler is not limited to point cloud compression but can have a wide range of applications.

[0116] For example, there is a growing popularity of augmented reality (AR) and virtual reality (VR) experiences, and other extended reality (XR) experiences, such as mixed reality (MR). Thus, there is interest in capturing high-resolution point clouds from the real world. Loss of detail and irregularities in the point cloud geometry can occur during the capture, processing, and compression pipeline. The level of detail (LoD) in a point cloud refers to the density or resolution of points captured or represented in a three-dimensional dataset and can affect the accuracy, precision, and usability of the point cloud for various applications. Oversampling a point cloud with a low level of detail (LoD) to a point cloud with a high LoD can provide several advantages. Often, oversampling, noise removal, and filtering in the point cloud... Petition 870250089998, dated 02 / 10 / 2025, page 59 / 206 53 / 83 points can be implemented as a post-processing technique to improve the quality of the point cloud.

[0117] According to the techniques of this disclosure, a deep learning-based residual learning network can be used when post-processing a point cloud (e.g., to upscale the decoded attribute data). The deep learning-based residual learning network can be trained to oversample attribute data from the point cloud data. For example, the deep learning-based residual learning network can, during training, receive both original full-scale attribute data and downscaled attribute data, attempt to oversample the downscaled attribute data, and update the network model based on the resulting differences between the oversampled attribute data and the original full-scale attribute data.In heuristic tests, residual learning networks outperform non-residual learning networks for tasks such as noise removal, oversampling, unblocking, and similar tasks. Residual learning networks can be used for point cloud filters, post-processing, and / or pre-processing tasks. A residual learning network can act as a learnable filter / post-processing element for point clouds.

[0118] Figures 11A and 11B are conceptual diagrams illustrating examples of residual learning networks according to the techniques of this disclosure. Residual networks include a neural network layer or a block of layers with a residual link. Figure 11A, for example, depicts the residual learning network 1100 Petition 870250089998, dated 02 / 10 / 2025, page 60 / 206 54 / 83 including the 3x3x3 convolutional layer 1102, the residual layer 1104, and the combination unit 1106. Figure 11B, as another example, depicts the residual learning network 1110, which includes the 3x3x3 convolutional layer block 1112, the residual block 1114, and the combination unit 1116.

[0119] Neural networks act as stacked nonlinear layers that fit into a mapping. Instead of learning the underlying O(x) mapping, a residual network learns the mapping R(x) = O(x) — x. Equation (1) Where the original output mapping is remodeled into R(x) + x. This is shown in Figures 11A and 11B where, instead of directly learning the output mapping y, the network learns a mapping for the residual, which when added by the combination units 1106, 1116 to the input x, results in the output y. This is achieved by having a residual / shortcut link between the input and output of the layer / layers.

[0120] Figure 12 is a conceptual diagram illustrating an example of a 1200 residual learning network for point clouds according to the techniques of this disclosure. According to the techniques of this disclosure, residual networks similar to those employed in image processing tasks can be applied to form an end-to-end residual network for point cloud processing tasks. Figure 12 depicts an example 1200 residual learning network. Instead of directly learning the mapping of the output point cloud, the 1200 residual learning network learns the residual of the 1202 point clouds which, when added to the input point cloud 1204 by the combination unit 1206, produces the cloud. Petition 870250089998, dated 02 / 10 / 2025, page 61 / 206 55 / 83 of desired output points. The 1200 residual learning network can be employed as a post-processing / filter for point cloud processing.

[0121] Figure 13 is a conceptual diagram illustrating an example of a 1300 residual learning network employing a network as a filter to remove noise from point clouds according to the techniques of this disclosure. In this example, the 1300 residual learning network is applied to perform attribute noise removal. In this example, where the input point cloud is noisy and the 1300 residual learning network learns the residual noise, which when removed from the input point cloud, produces a noise-free point cloud.

[0122] Figure 14 is a conceptual diagram illustrating an example decoding structure according to the techniques of this disclosure. In this example, the point cloud decoding structure 1400 includes deep learning-based geometry decoder 1402, geometry downscaling unit 1404, attribute decoder 1406, recoloring unit 1408, and deep learning-based residual network 1416 for noise removal. The deep learning-based geometry decoder 1402 may correspond to the deep learning-based geometry decoder 302, the geometry downscaling unit 1404 may correspond to the geometry downscaling unit 306, the attribute decoder 1406 may correspond to the arithmetic attribute decoding unit 304, inverse quantization unit 308, RAHT unit 314, LOD generation unit 316, inverse uplift unit 318, and transformation unit. Petition 870250089998, dated 02 / 10 / 2025, p. 62 / 206 56 / 83 reverse color 322, recoloring unit 1408, and the deep learning-based residual network 1416 for noise removal can correspond to the point cloud upscaling unit 324.

[0123] In general, the deep learning-based geometry decoder 1402 receives a geometry bitstream 1410. The deep learning-based geometry decoder 1402 decodes geometry information from the geometry bitstream 1410 and reconstructs geometry information from the decoded geometry information. The geometry downscaling unit 1404 downscales the geometry information to form scaled-down geometry information and provides the scaled-down geometry information to the attribute decoder 1406. The attribute decoder 1406 receives the attribute bitstream 1412, including encoded attribute information, and decodes the encoded attribute information.The attribute decoder 1406 can provide the decoded attribute information to the recoloring unit 1408, which can also receive the original reconstructed geometry information and apply the attribute information to the scale of the reconstructed geometry information. The deep learning-based residual network for noise removal 1416 can remove noise from the output of the recoloring unit 1408 to ultimately form the reconstructed point cloud 1414.

[0124] In this way, the 1400 point cloud decoding structure performs a two-stage supersampling process, in contrast to the single-stage supersampling process performed by the structure Petition 870250089998, dated 02 / 10 / 2025, page 63 / 206 57 / 83 of point cloud decoding 700 from Figure 7. Even though the point cloud decoding structure 1400 does not explicitly perform subsampling, the recoloring step oversamples the attributes and maps them to the oversampled geometry. The recoloring unit 1408 passes the downscaled attributes through a recoloring scheme to recolor the reconstructed geometry with the downscaled attributes. This produces a noisy reconstructed point cloud that would have noise introduced due to the recoloring, downscaling, and encoding process. The deep learning-based residual network for noise removal 1416 post-processes this point cloud, where the deep learning-based residual network for noise removal 1416 acts as a filter to remove noise from the point cloud.

[0125] The scaling down in both the encoder and the decoder can have a certain scaling down ratio / rate (e.g., 2, 4, 8, etc.). The scaling down ratio / rate can be signaled from the encoder to the decoder, for example, in an APS or an SPS, as discussed above. The decoder can include a single deep learning-based residual network for all scaling down ratios or multiple different deep learning-based residual networks for the various scaling down ratios / rates, where each residual network can be employed for a separate scaling down ratio.

[0126] In some examples, a deep learning-based residual network can be applied to other points in the coding pipeline. For example, in addition to or Petition 870250089998, dated 02 / 10 / 2025, page 64 / 206 58 / 83 As an alternative to post-processing attribute data, residual learning networks can be applied to geometry (coordinate) data. Residual learning networks can be employed for attribute noise removal, geometry noise removal, point cloud oversampling, artifact removal, point cloud classification, segmentation and / or compression, or similar purposes. Thus, residual learning networks can be used as a filter, a post-processing unit, a pre-processing unit, or any other task in the point cloud processing pipeline, for example, point cloud capture, compression and / or transmission. Residual learning networks can be employed with any attributes (e.g., RGB and / or intensity) or even a color space conversion to YCbCr attributes.

[0127] Figure 15 is a flowchart illustrating an example point cloud data encoding method according to the techniques of this disclosure. The method in Figure 15 is explained in relation to the 200 point cloud encoder of Figure 2. Other point cloud encoding devices, such as those conforming to the 600 point cloud encoding framework of Figure 6, may perform this method or a similar method.

[0128] Initially, point cloud encoder 200 encodes geometry information from a point cloud (1500). For example, point cloud encoder 200 can use a deep learning-based point cloud encoding technique to encode the geometry information. Point cloud encoder 200 can then decode and reconstruct the geometry information (1502). Petition 870250089998, dated 02 / 10 / 2025, page 65 / 206 59 / 83 Point cloud encoder 200 can also downscale geometry information (1504). Point cloud encoder 200 can recolor downscale geometry information using point cloud attribute information (1506). Point cloud encoder 200 can then downscale attribute information (1508). In some examples, point cloud encoder 200 can additionally encode HLS information by specifying, for example, a downscaling factor for geometry information and / or an oversampling factor for attribute information.

[0129] In this way, the method in Figure 15 represents an example of a method for encoding point cloud information, including: decoding encoded point cloud geometry data to a point cloud in order to reconstruct point cloud geometry data for the point cloud; downscaling the point cloud geometry data to form downscaled point cloud geometry data; and encoding attribute data for the point cloud using the downscaled point cloud geometry.

[0130] Figure 16 is a flowchart illustrating an example point cloud data decoding method according to the techniques of this disclosure. The method in Figure 16 is explained in relation to the 300 point cloud decoder of Figure 3. Other point cloud decoding devices, such as those conforming to the 700 point cloud decoding framework of Figure 7, may perform this method or a similar method. Petition 870250089998, dated 02 / 10 / 2025, page 66 / 206 60 / 83

[0131] Initially, the point cloud decoder 300 decodes geometry information (1600). The point cloud decoder 300 can also decode a downscaling factor, for example, in high-level syntax (HLS) data. The point cloud decoder 300 can downscale the geometry information (1602), for example, according to the downscaling factor. The point cloud decoder 300 can then decode the attribute information (1604). The point cloud decoder 300 can then apply the attribute information to the decoded geometry information (1606), which can result in an intermediate (noisy) point cloud. The point cloud decoder 300 can then apply a residual learning network to the intermediate point cloud (1608) to form a reconstructed point cloud.For example, the residual learning network could correspond to, for example, the 3x3x3 layer SConv 1022, the IRB layer 1024, and the SConv layer 1026 from Figure 10; residual learning network. 1100 from Figure 11A; residual learning network 1110 from Figure 11B; residual learning network 1200 from Figure 12; residual learning network 1300 from Figure 13; or deep learning-based residual learning network 1416 from Figure 14.

[0132] In this way, the method in Figure 16 represents an example of a method for encoding (e.g., encoding or decoding) point cloud information, including: decoding encoded point cloud geometry data to a point cloud in order to form reconstructed point cloud geometry data for the point cloud; downscaling point cloud geometry data to form geometry data Petition 870250089998, dated 02 / 10 / 2025, page 67 / 206 61 / 83 of a scaled-down point cloud; decode attribute data for the point cloud using scaled-down point cloud geometry; apply the attribute data to the reconstructed point cloud geometry data to form intermediate point cloud data; and apply a residual learning network to the intermediate point cloud data to form a reconstructed point cloud.

[0133] Figure 17 is a conceptual diagram illustrating a 1700 laser package, such as a LIDAR sensor or other system that includes one or more lasers, scanning points in three-dimensional space. Data source 104 (Figure 1) may include a 1700 laser package.

[0134] As shown in Figure 17, point clouds can be captured using the 1700 laser package, that is, the sensor scans points in three-dimensional space. It should be understood, however, that some point clouds are not generated by a true LIDAR sensor, but can be encoded as if they had been. In the example in Figure 17, the 1700 laser package includes a 1702 LIDAR head that includes multiple 1704A to 1704E lasers (collectively, 1704 lasers) arranged in a vertical plane at different angles relative to an origin point. The 1700 laser package can rotate around a vertical axis 1708. The 1700 laser package can use the returned laser light to determine the distances and positions of the points in the point cloud. The laser beams 1706A to 1306E (collectively, laser beams 1706) emitted by the lasers 1704 of the laser package 1700 can be characterized by a set of parameters. The distances denoted by Petition 870250089998, dated 02 / 10 / 2025, page 68 / 206 Arrows 62 / 83, 1710, 1712 denote example laser correction values ​​for lasers 1704B and 1704A, respectively.

[0135] 1700 lasers can be used to obtain both geometry data and attribute data for points from the geometry data. The point cloud geometry data can be scaled down, and the attribute data can be scaled down and encoded. According to the techniques in this disclosure, the attribute data can be subsequently scaled up using a residual learning network.

[0136] Figure 18 is a conceptual diagram illustrating an example 1800 range measurement system that can be used with one or more techniques of this disclosure. In the example of Figure 18, the 1800 range measurement system includes an illuminator 1802 and a sensor 1804. The illuminator 1802 can emit light 1806. In some examples, the illuminator 1802 can emit light 1806 as one or more laser beams. The light 1806 can be at one or more wavelengths, such as an infrared wavelength or a visible light wavelength. In other examples, the light 1806 is not a coherent laser light. When the light 1806 encounters an object, such as object 1808, the light 1806 creates a return light 1810. The return light 1810 can include backscattered and / or reflected light. The return light 1810 can pass through a lens 1811 which directs the return light 1810 to create an image 1812 of the object 1808 on the sensor 1804. The sensor 1804 generates signals 1814 based on the image 1812.Image 1812 may comprise a set of points (for example, as represented by points in image 1812 of Figure 18). Petition 870250089998, dated 02 / 10 / 2025, page 69 / 206 63 / 83

[0137] In some examples, illuminator 1802 and sensor 1804 may be mounted on a rotating structure so that illuminator 1802 and sensor 1804 capture a 360-degree view of an environment. In other examples, the 1800 range measurement system may include one or more optical components (e.g., mirrors, collimators, diffraction gratings, etc.) that enable illuminator 1802 and sensor 1804 to detect objects within a specific range (e.g., up to 360 degrees). Although the example in Figure 18 shows only a single illuminator 1802 and sensor 1804, the 1800 range measurement system may include multiple sets of illuminators and sensors.

[0138] In some examples, the illuminator 1802 generates a structured light pattern. In such examples, the range measurement system 1800 may include multiple sensors 1804 on which the respective images of the structured light pattern are formed. The range measurement system 1800 may use disparities between the images of the structured light pattern to determine a distance to an object 1808 from which the structured light pattern backscatters. Structured light-based range measurement systems can have a high level of accuracy (e.g., submillimeter range accuracy) when the object 1808 is relatively close to the sensor 1804 (e.g., from 0.2 meters to 2 meters). This high level of accuracy can be useful in facial recognition applications, such as unlocking mobile devices (e.g., mobile phones, tablet computers, etc.) and for security applications.

[0139] In some examples, the 1800 range measurement system is a time-of-flight based system (ToF Petition 870250089998, 02 / 10 / 2025, p. 70 / 206 64 / 83 time of flight). In some examples where the 1800 range measurement system is a ToF-based system, the illuminator 1802 generates pulses of light. In other words, the illuminator 1802 can modulate the amplitude of the emitted light 1806. In these examples, the sensor 1804 detects the return light 1810 from the light pulses 1806 generated by the illuminator 1802. The range measurement system 1800 can then determine a distance to the object 1808 from which the light 1806 backscatters based on a delay between when the light 1806 was emitted and detected and the known speed of light in air. In some examples, instead of (or in addition to) modulating the amplitude of the emitted light 1806, the illuminator 1802 can modulate the phase of the emitted light 1806. In these examples, the sensor 1804 can detect the phase of the return light 1810 coming from the object 1808 and determine the distances to points on the object 1808 using the speed of light and based on the time differences between when the illuminator 1802 generated the light 1806 in a specific phase and when the sensor 1804 detected the return light 1810 in the specific phase.

[0140] In other examples, a point cloud can be generated without using the illuminator 1802. For example, in some examples, the sensor 1804 of the range measurement system 1800 may include two or more optical cameras. In these examples, the range measurement system 1800 may use the optical cameras to capture stereo images of the environment, including the object 1808. The range measurement system 1800 (e.g., the point cloud generator 1820) can then calculate the disparities between locations in the stereoscopic images. The range measurement system 1800 can then use the disparities to determine the Petition 870250089998, dated 02 / 10 / 2025, page 71 / 206 65 / 83 distances to the locations shown in the stereoscopic images. From these distances, the 1820 point cloud generator can generate a point cloud.

[0141] The 1804 sensors can also detect other object attributes 1808, such as color and reflectance information. In the example in Figure 18, a point cloud generator 1820 can generate a point cloud based on the signals 1818 generated by the 1804 sensor. The range measurement system 1800 and / or the point cloud generator 1820 can be part of the data source 104 (Figure 1).

[0142] Figure 19 is a conceptual diagram illustrating an example vehicle-based scenario in which one or more techniques of this disclosure may be used. In the example in Figure 19, a vehicle 1900 includes a laser package 1902, as a LIDAR system. The laser package 1902 can be implemented in the same way as the laser package 600 (Figure 17). Although not shown in the example in Figure 19, the vehicle 1900 may also include a data source, such as data source 104 (Figure 1), and a G-PCC encoder, such as G-PCC encoder 200 (Figure 1). In the example in Figure 19, the laser package 1902 emits laser beams 1904 that are reflected by pedestrians 1906 or other objects on a highway. The vehicle data source 1900 can generate a point cloud based on signals generated by the laser package 1902.The vehicle's G-PCC encoder 1900 can encode the point cloud to generate 1908 bitstreams, such as the geometry bitstream in Figure 2 and the attribute bitstream in Figure 2. The 1908 bitstreams can include a much smaller number of bits than the unencoded point cloud obtained by the G-PCC encoder. A. Petition 870250089998, dated 02 / 10 / 2025, page 72 / 206 The 66 / 83 output interface of the 1900 vehicle (e.g., output interface 108 (Figure 1)) can transmit 1908 bitstreams to one or more other devices. Thus, the 1900 vehicle may be able to transmit 1908 bitstreams to other devices faster than unencoded point cloud data. Furthermore, 1908 bitstreams may require less data storage capacity.

[0143] The techniques in this disclosure can further reduce the number of bits in the 1908 bitstreams. For example, by downscaling and then encoding the attribute data, the amount of attribute data to be encoded can be significantly reduced, thus reducing the number of bits in the 1908 bitstream. By subsequently upscaling the downscaled attribute data using a residual learning network, the point cloud reconstructed from the upscaled attribute data can still maintain a high degree of accuracy compared to the original point cloud.

[0144] In the example in Figure 19, vehicle 1900 can transmit bitstreams 1908 to another vehicle 1910. Vehicle 1910 may include a G-PCC decoder, such as the G-PCC decoder 300 (Figure 1). The G-PCC decoder in vehicle 1910 can decode the bitstreams 1908 to reconstruct the point cloud. Vehicle 1910 can use the reconstructed point cloud for various purposes. For example, vehicle 1910 can determine, based on the reconstructed point cloud, that there are pedestrians 1906 on the highway ahead of vehicle 1900 and therefore begin to reduce speed, for example, even before a driver of vehicle 1910 realizes that there are pedestrians 1906 on the highway. Thus, in some Petition 870250089998, dated 02 / 10 / 2025, page 73 / 206 67 / 83 examples, vehicle 1910 can perform an autonomous navigation operation, generate a notification or warning, or perform another action based on the reconstructed point cloud.

[0145] Alternatively, vehicle 1900 may transmit bitstreams 1908 to a server system 1912. Server system 1912 may use bitstreams 1908 for various purposes. For example, server system 1912 may store bitstreams 1908 for subsequent reconstruction of point clouds. In this example, server system 1912 may use the point clouds along with other data (e.g., vehicle telemetry data generated by vehicle 1900) to train an autonomous driving system. In another example, server system 1912 may store bitstreams 1908 for subsequent reconstruction for forensic collision investigations (e.g., if vehicle 1900 collides with pedestrians 1906).

[0146] Figure 20 is a conceptual diagram illustrating an example extended reality system in which one or more techniques of this disclosure may be used. Extended reality (XR) is a term used to encompass a range of technologies that includes augmented reality (AR), mixed reality (MR), and virtual reality (VR). In the example in Figure 20, a first user 2000 is located at a first location 2002. User 2000 uses an XR headset 2004. As an alternative to the XR headset 2004, user 2000 may use a mobile device (e.g., a mobile phone, a tablet computer, etc.). The XR headset 2004 includes a depth-sensing sensor, such as a LIDAR system, which detects the positions of points on objects 2006 at location 2002. A data source from the XR headset Petition 870250089998, dated 02 / 10 / 2025, page 74 / 206 68 / 83 2004 can use the signals generated by the depth-sensing sensor to generate a point cloud representation of the objects 2006 at location 2002. The XR headset 2004 can include a G-PCC encoder (for example, the G-PCC encoder 200 of Figure 1) that is configured to encode the point cloud in order to generate bitstreams 2008.

[0147] The techniques in this disclosure can further reduce the number of bits in the 2008 bitstreams. For example, by downscaling and then encoding the attribute data, the amount of attribute data to be encoded can be significantly reduced, thus reducing the number of bits in the 2008 bitstream. By subsequently upscaling the downscaled attribute data using a residual learning network, the point cloud reconstructed from the upscaled attribute data can still maintain a high degree of accuracy compared to the original point cloud.

[0148] The XR headset 2004 can transmit bitstreams 2008 (e.g., via a network such as the Internet) to an XR headset 2010 worn by a user 2012 in a second location 2014. The XR headset 2010 can decode the bitstreams 2008 to reconstruct the point cloud. The XR headset 2010 can use the point cloud to generate an XR visualization (e.g., an AR, MR, or VR visualization) representing the objects 2006 in location 2002. Thus, in some examples, such as when the XR headset 2010 generates a VR visualization, the user 2012 in location 2014 can have a three-dimensional immersive experience of location 2002. In some examples, the XR headset 2010 can determine the position of an object. Petition 870250089998, dated 02 / 10 / 2025, page 75 / 206 69 / 83 virtual based on the reconstructed point cloud. For example, the XR 2010 headset can determine, based on the reconstructed point cloud, that an environment (e.g., location 2002) includes a flat surface and then determine that a virtual object (e.g., a cartoon character) should be positioned on the flat surface. The XR 2010 headset can generate an XR visualization in which the virtual object is in the determined position. For example, the XR 2010 headset can show the cartoon character on the flat surface.

[0149] Figure 21 is a conceptual diagram illustrating an example mobile device system in which one or more techniques of this disclosure may be used. In the example in Figure 21, a mobile device 2100, such as a mobile phone or tablet computer, includes a depth-sensing sensor, such as a LIDAR system, that detects point positions on objects 2102 in a mobile device 2100 environment. A data source of the mobile device 2100 may use the signals generated by the depth-sensing sensor to generate a point cloud representation of the objects 2102. The mobile device 2100 may include a G-PCC encoder (for example, the G-PCC encoder 200 of Figure 1) that is configured to encode the point cloud in order to generate bitstreams 2104. In the example in Figure 21, the mobile device 2100 may transmit bitstreams to a remote device 2106, such as a server system or another mobile device.Remote device 2106 can decode bitstreams 2104 to reconstruct the point cloud. Remote device 2106 can use the point cloud for various purposes. For example, remote device 2106 can use... Petition 870250089998, dated 02 / 10 / 2025, page 76 / 206 70 / 83 the point cloud to generate a map of an environment from the mobile device 2100. For example, the remote device 2106 can generate a map of a building's interior based on the reconstructed point cloud. In another example, the remote device 2106 can generate images (e.g., computer graphics) based on the point cloud. For example, the remote device 2106 can use points from the point cloud as vertices of polygons and use color attributes of the points as the basis for shading the polygons. In some examples, the remote device 2106 can perform facial recognition using the point cloud.

[0150] The following clauses represent several examples of the techniques of this disclosure.

[0151] Clause 1: A device for decoding point cloud data, the device comprising: a memory configured to store point cloud data; and one or more processors implemented in the circuit set and configured to: decode encoded point cloud geometry data to a point cloud in order to form reconstructed point cloud geometry data for the point cloud; downscale point cloud geometry data to form downscaled point cloud geometry data; decode attribute data to the point cloud using the downscaled point cloud geometry; apply the attribute data to the reconstructed point cloud geometry data to form intermediate point cloud data; and apply a residual learning network to the intermediate point cloud data to form a reconstructed point cloud. Petition 870250089998, dated 02 / 10 / 2025, page 77 / 206 71 / 83

[0152] Clause 2: The device of clause 1, wherein one or more processors are additionally configured to decode a value representing a downscaling amount to be applied to point cloud geometry data, wherein to downscale point cloud geometry data, one or more processors are configured to downscale point cloud geometry data according to the value representing the downscaling amount.

[0153] Clause 3: The device of clause 2, in which one or more processors are configured to provide the value that represents the amount of scaling down to be applied to the point cloud geometry data for the residual learning network.

[0154] Clause 4: The device of clause 2, wherein one or more processors are configured to select the residual learning network from among a plurality of residual learning networks according to the value representing the amount of downscaling to be applied to the point cloud geometry data.

[0155] Clause 5: The device of clause 1, wherein the residual learning network includes one or more convolutional neural network layers and a combining unit that combines the output of the one or more convolutional neural network layers with the intermediate point cloud data to form the reconstructed point cloud.

[0156] Clause 6: The device of clause 1, wherein the residual learning network comprises a first residual learning network, and wherein the one or more processors are additionally configured to Petition 870250089998, dated 02 / 10 / 2025, page 78 / 206 72 / 83 apply a second residual learning network during preprocessing or as a filter for the point cloud.

[0157] Clause 7: The device of clause 1, wherein the attribute data includes color data in one of a red-green-blue (RGB) format or a luminance, blue hue chrominance and red hue chrominance (YCbCr) format.

[0158] Clause 8: The device of clause 1, wherein, to downscale point cloud geometry data, one or more processors are configured to: for each node of an octree that includes eight leaf subnodes where at least one of the eight leaf subnodes is occupied by a point, redefine the node as an occupied leaf node in a downscaled octree; and for each node of the octree that includes eight leaf subnodes where none of the eight leaf subnodes is occupied by a point, redefine the node as an unoccupied leaf node in the downscaled octree.

[0159] Clause 9: The device of clause 1, wherein, to downscale point cloud geometry data, one or more processors are configured to: for each node of an octree that includes eight leaf subnodes where a number of those that are occupied among the eight leaf subnodes is greater than a threshold, redefine the node as an occupied leaf node in a downscaled octree; and for each node of the octree that includes eight leaf subnodes where a number of those that are occupied among the eight leaf subnodes is less than or equal to the threshold, redefine the node as an unoccupied leaf node in the downscaled octree.

[0160] Clause 10: The device of clause 1, in which one or more processors are additionally Petition 870250089998, dated 02 / 10 / 2025, page 79 / 206 73 / 83 configured to decode a value representing a downscaling amount to be applied to point cloud geometry data, wherein to downscale point cloud geometry data, one or more processors are configured to downscale point cloud geometry data according to the value representing the downscaling amount.

[0161] Clause 11: The device of clause 10, wherein one or more processors are configured to provide the value representing the amount of scaling down to be applied to the point cloud geometry data for the residual learning network.

[0162] Clause 12: The device of clause 10, wherein one or more processors are configured to select the residual learning network from among a plurality of residual learning networks according to the value representing the amount of downscaling to be applied to the point cloud geometry data.

[0163] Clause 13: The device of any of clauses 1 and 10 to 12, wherein the residual learning network includes one or more convolutional neural network layers and a combining unit that combines the output of the one or more convolutional neural network layers with the intermediate point cloud data to form the reconstructed point cloud.

[0164] Clause 14: The device of any of clauses 1 and 10 to 13, wherein the residual learning network comprises a first residual learning network, and wherein one or more processors are additionally configured to apply a second network. Petition 870250089998, dated 02 / 10 / 2025, page 80 / 206 74 / 83 of residual learning during preprocessing or as a filter for the point cloud.

[0165] Clause 15: The device of any of clauses 1 and 10 to 14, wherein the attribute data include color data in one of a red-blue-green (RGB) format or a luminance, blue hue chrominance and red hue chrominance (YCbCr) format.

[0166] Clause 16: The device of any of clauses 1 and 10 to 15, wherein, to downscale point cloud geometry data, one or more processors are configured to: for each node of an octree that includes eight leaf subnodes where at least one of the eight leaf subnodes is occupied by a point, redefine the node as an occupied leaf node in a downscaled octree; and for each node of the octree that includes eight leaf subnodes where none of the eight leaf subnodes is occupied by a point, redefine the node as an unoccupied leaf node in the downscaled octree.

[0167] Clause 17: The device of any of clauses 1 and 10 to 15, wherein, to downscale point cloud geometry data, one or more processors are configured to: for each node of an octree that includes eight leaf subnodes where a number of those that are occupied among the eight leaf subnodes is greater than a threshold, redefine the node as an occupied leaf node in a downscaled octree; and for each node of the octree that includes eight leaf subnodes where a number of those that are occupied among the eight leaf subnodes is less than or equal to the threshold, redefine the node as an unoccupied leaf node in the downscaled octree.

[0168] Clause 18: A method for decoding point cloud data, the method comprising: decoding Petition 870250089998, dated 02 / 10 / 2025, page 81 / 206 75 / 83 point cloud geometry data encoded into a point cloud in order to form reconstructed point cloud geometry data for the point cloud; downscaling the point cloud geometry data to form downscaled point cloud geometry data; decoding attribute data for the point cloud using the downscaled point cloud geometry; applying the attribute data to the reconstructed point cloud geometry data to form intermediate point cloud data; and applying a residual learning network to the intermediate point cloud data to form a reconstructed point cloud.

[0169] Clause 19: The method of clause 18, which further comprises decoding a value representing a scaling amount to be applied to point cloud geometry data, wherein scaling of point cloud geometry data includes scaling down point cloud geometry data according to the value representing the scaling amount.

[0170] Clause 20: The method of clause 19, which further comprises providing the value representing the amount of scaling down to be applied to the point cloud geometry data for the residual learning network.

[0171] Clause 21: The method of clause 19, which further comprises selecting the residual learning network from among a plurality of residual learning networks according to the value representing the quantity Petition 870250089998, dated 02 / 10 / 2025, page 82 / 206 76 / 83 scale reduction to be applied to point cloud geometry data.

[0172] Clause 22: The method of clause 18, wherein the residual learning network includes one or more convolutional neural network layers and a combining unit that combines the output of the one or more convolutional neural network layers with the intermediate point cloud data to form the reconstructed point cloud.

[0173] Clause 23: The method of clause 18, wherein the residual learning network comprises a first residual learning network, the method further comprising applying a second residual learning network during preprocessing or as a filter for the point cloud.

[0174] Clause 24: The method of clause 18, wherein the attribute data include color data in one of a red-green-blue (RGB) format or a luminance, blue hue chrominance, and red hue chrominance (YCbCr) format.

[0175] Clause 25: The method of clause 18, wherein the downscaling of point cloud geometry data comprises: for each node of an octree that includes eight leaf subnodes where at least one of the eight leaf subnodes is occupied by a point, redefine the node as an occupied leaf node in a downscaled octree; and for each node of the octree that includes eight leaf subnodes where none of the eight leaf subnodes is occupied by a point, redefine the node as an unoccupied leaf node in the downscaled octree.

[0176] Clause 26: The method of clause 18, in which the scaling down of the cloud geometry data of Petition 870250089998, dated 02 / 10 / 2025, page 83 / 206 77 / 83 points comprises: for each node in an octree that includes eight leaf subnodes where a number of those that are occupied among the eight leaf subnodes is greater than a threshold, redefine the node as an occupied leaf node in a scaled-down octree; and for each node in the octree that includes eight leaf subnodes where a number of those that are occupied among the eight leaf subnodes is less than or equal to the threshold, redefine the node as an unoccupied leaf node in the scaled-down octree.

[0177] Clause 27: The method of clause 18, which further comprises decoding a value representing a scaling amount to be applied to point cloud geometry data, wherein scaling of point cloud geometry data includes scaling down point cloud geometry data according to the value representing the scaling amount.

[0178] Clause 28: The method of clause 27, which additionally comprises providing the value representing the amount of scaling down to be applied to the point cloud geometry data for the residual learning network.

[0179] Clause 29: The method of clause 27, which further comprises selecting the residual learning network from among a plurality of residual learning networks according to the value representing the amount of scaling down to be applied to the point cloud geometry data.

[0180] Clause 30: The method of any of clauses 18 and 27 to 29, where the residual learning network includes one or more layers of convolutional neural network and a Petition 870250089998, dated 02 / 10 / 2025, p. 84 / 206 78 / 83 combining unit that combines the output of one or more convolutional neural network layers with intermediate point cloud data to form the reconstructed point cloud.

[0181] Clause 31: The method of any of clauses 18 and 27 to 30, wherein the residual learning network comprises a first residual learning network, the method further comprising applying a second residual learning network during preprocessing or as a filter for the point cloud.

[0182] Clause 32: The method of any of clauses 18 and 27 to 31, wherein the attribute data include color data in one of a red-green-blue (RGB) format or a luminance, blue hue chrominance and red hue chrominance (YCbCr) format.

[0183] Clause 33: The method of any of clauses 18 and 27 to 32, wherein the downscaling of point cloud geometry data comprises: for each node of an octree that includes eight leaf subnodes where at least one of the eight leaf subnodes is occupied by a point, redefine the node as an occupied leaf node in a downscaled octree; and for each node of the octree that includes eight leaf subnodes where none of the eight leaf subnodes is occupied by a point, redefine the node as an unoccupied leaf node in the downscaled octree.

[0184] Clause 34: The method of any of clauses 18 and 27 to 32, wherein the downscaling of point cloud geometry data comprises: for each node of an octree that includes eight leaf subnodes where a number of those that are occupied among the eight leaf subnodes is greater than a threshold, redefine the node as an occupied leaf node in Petition 870250089998, dated 02 / 10 / 2025, page 85 / 206 79 / 83 a scaled-down octree; and for each node in the octree that includes eight leaf subnodes where a number of those that are occupied among the eight leaf subnodes is less than or equal to the threshold, redefine the node as an unoccupied leaf node in the scaled-down octree.

[0185] Clause 35: A computer-readable storage medium having stored therein instructions which, when executed, cause a processor to: decode encoded point cloud geometry data into a point cloud in order to form reconstructed point cloud geometry data for the point cloud; downscale point cloud geometry data to form downscaled point cloud geometry data; decode attribute data into the point cloud using downscaled point cloud geometry; apply the attribute data to the reconstructed point cloud geometry data to form intermediate point cloud data; and apply a residual learning network to the intermediate point cloud data to form a reconstructed point cloud.

[0186] Clause 36: A device for encoding point cloud data, wherein the device comprises: means for decoding encoded point cloud geometry data to a point cloud in order to form reconstructed point cloud geometry data for the point cloud; means for downscaling point cloud geometry data to form downscaled point cloud geometry data; means for decoding attribute data to the point cloud using the downscaled point cloud geometry; means Petition 870250089998, dated 02 / 10 / 2025, page 86 / 206 80 / 83 to apply attribute data to reconstructed point cloud geometry data to form intermediate point cloud data; and means to apply a residual learning network to intermediate point cloud data to form a reconstructed point cloud.

[0187] It should be recognized that, depending on the example, certain actions or events of any of the techniques described in the present invention may be performed in a different sequence, may be added, combined or completely omitted (for example, not all actions or events described are necessary for the practice of the techniques). Furthermore, in certain examples, the actions or events may be performed simultaneously, for example, through multi-threaded processing, interrupt processing or multiple processors, instead of sequentially.

[0188] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored or transmitted as one or more instructions or code in a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media, which correspond to a tangible medium, such as data storage media, or communication media, including any medium that facilitates the transfer of a computer program from one place to another, for example, according to a communication protocol. In this way, computer-readable media may generally correspond to (1) media Petition 870250089998, dated 02 / 10 / 2025, page 87 / 206 81 / 83 tangible computer-readable storage media that are non-transient or (2) a communication medium, such as a signal or a carrier wave. Data storage media may be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, code and / or data structures for the implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0189] By way of example, and not limitation, such computer-readable storage media may comprise random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory or any other media that can be used to store the desired program code in the form of instructions or data structures and that can be accessed by a computer. In addition, any connection is properly termed a computer-readable medium.For example, if the instructions are transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave will be... Petition 870250089998, dated 02 / 10 / 2025, page 88 / 206 82 / 83 included in the definition of media. It should be understood, however, that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but instead refer to tangible non-transient storage media. As used in the present invention, disks (disk and disc) include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically by means of lasers. Combinations of the above items should also be included within the scope of computer-readable media.

[0190] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other distinct or equivalent integrated logic circuit assemblies. Consequently, the terms processor and processing circuit assembly, as used in the present invention, may refer to any of the foregoing structures or any other structure suitable for implementing the techniques described in the present invention. Furthermore, in some respects, the functionality described in the present invention may be provided in dedicated hardware and / or software modules configured for encoding and decoding, or may be incorporated into a combined codec. Petition 870250089998, dated 02 / 10 / 2025, pp. 89 / 206 83 / 83 Furthermore, the techniques can be fully implemented in one or more circuits or logic elements.

[0191] The techniques in this disclosure can be implemented in a wide variety of devices or appliances, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Several components, modules, or units are described in this disclosure to emphasize functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require implementation by different hardware units. Instead, as described above, several units can be combined into a codec hardware unit or provided by a collection of interoperable hardware units, including one or more processors, as described above, in conjunction with suitable software and / or firmware.

[0192] Several examples have been described. These and other examples are within the scope of the following claims. Petition 870250089998, dated 02 / 10 / 2025, pp. 90 / 206

Claims

1 / 6 CLAIMS 1. Device for decoding point cloud data, characterized by comprising: a memory configured to store point cloud data; and a processing system comprising one or more processors implemented in a circuit assembly, wherein the processing system is configured to: decode encoded point cloud geometry data to a point cloud in order to form reconstructed point cloud geometry data for the point cloud; downscale the point cloud geometry data to form downscaled point cloud geometry data; decode attribute data to the point cloud using the downscaled point cloud geometry; apply the attribute data to the reconstructed point cloud geometry data to form intermediate point cloud data;and apply a residual learning network to the intermediate point cloud data to form a reconstructed point cloud.

2. Device, according to claim 1, characterized in that the processing system is additionally configured to decode a value representing a downscaling amount to be applied to point cloud geometry data, wherein to downscale the point cloud geometry data, one or more processors are configured to downscale the point cloud geometry data according to the value representing the downscaling amount.

3. Device, according to claim 2, characterized in that one or more processors are configured to provide the value representing the amount of downscaling to be applied to the point cloud geometry data for the residual learning network.

4. Device, according to claim 2, characterized in that one or more processors are configured to select the residual learning network from among a plurality of residual learning networks according to the value representing the amount of downscaling to be applied to the point cloud geometry data.

5. Device, according to claim 1, characterized in that the residual learning network includes one or more convolutional neural network layers and a combining unit that combines the output of the one or more convolutional neural network layers with the intermediate point cloud data to form the reconstructed point cloud.

6. Device according to claim 1, characterized in that the residual learning network comprises a first residual learning network, and wherein the processing system is additionally configured to apply a second residual learning network during preprocessing or as a filter for the point cloud.

7. Device according to claim 1, characterized in that the attribute data includes color data in one of a red-green-blue (RGB) format or a luminance, blue hue chrominance and red hue chrominance (YCbCr) format.

8. Device according to claim 1, characterized in that, to reduce the scale of point cloud geometry data, the processing system is configured to: for each node of an octree that includes eight leaf subnodes where at least one of the eight leaf subnodes is occupied by a point, redefine the node as an occupied leaf node in a reduced-scale octree; and for each node of the octree that includes eight leaf subnodes where none of the eight leaf subnodes is occupied by a point, redefine the node as an unoccupied leaf node in the reduced-scale octree.

9. Device according to claim 1, characterized in that, to reduce the scale of point cloud geometry data, the processing system is configured to: for each node of an octree that includes eight leaf subnodes where a number of those that are occupied among the eight leaf subnodes is greater than a threshold, redefine the node as an occupied leaf node in a scaled-down octree; and for each node of the octree that includes eight leaf subnodes where a number of those that are occupied among the eight leaf subnodes is less than or equal to the threshold, redefine the node as an unoccupied leaf node in the scaled-down octree.

10. Device according to claim 1, characterized by further comprising a display configured to visually depict the reconstructed point cloud. Petition 870250089998, dated 10 / 02 / 2025, pp. 177 / 206 4 / 6 11. Point cloud data decoding method, characterized by comprising: decoding encoded point cloud geometry data into a point cloud in order to form reconstructed point cloud geometry data for the point cloud; downscaling the point cloud geometry data to form downscaled point cloud geometry data; decoding attribute data into the point cloud using the downscaled point cloud geometry; applying the attribute data to the reconstructed point cloud geometry data to form intermediate point cloud data; and applying a residual learning network to the intermediate point cloud data to form a reconstructed point cloud.

12. A method according to claim 11, characterized by further comprising decoding a value representing a scaling amount to be applied to point cloud geometry data, wherein the scaling of point cloud geometry data includes scaling down the point cloud geometry data according to the value representing the scaling amount.

13. Method according to claim 12, characterized by further comprising providing the value representing the amount of scaling down to be applied to the point cloud geometry data for the residual learning network. Petition 870250089998, dated 10 / 02 / 2025, pp. 178 / 206 5 / 6 14. Method, according to claim 12, characterized by further comprising selecting the residual learning network from among a plurality of residual learning networks according to the value representing the amount of downscaling to be applied to the point cloud geometry data.

15. Method, according to claim 11, characterized in that the residual learning network includes one or more convolutional neural network layers and a combining unit that combines the output of the one or more convolutional neural network layers with the intermediate point cloud data to form the reconstructed point cloud.

16. Method, according to claim 11, characterized in that the residual learning network comprises a first residual learning network, wherein the method further comprises applying a second residual learning network during preprocessing or as a filter for the point cloud.

17. Method, according to claim 11, characterized in that the attribute data include color data in one of a red-green-blue (RGB) format or a luminance, blue hue chrominance and red hue chrominance (YCbCr) format.

18. Method according to claim 11, characterized by scaling down point cloud geometry data comprising: for each node of an octree that includes eight leaf subnodes where at least one of the eight leaf subnodes is occupied by a point, redefining the node as an occupied leaf node in a scaled-down octree; and Petition 870250089998, dated 10 / 02 / 2025, p. 179 / 206 6 / 6 for each node of the octree that includes eight leaf subnodes where none of the eight leaf subnodes is occupied by a point, redefining the node as an unoccupied leaf node in the scaled-down octree.

19. A method according to claim 11, characterized by the downscaling of point cloud geometry data comprising: for each node of an octree that includes eight leaf subnodes where a number of those that are occupied among the eight leaf subnodes is greater than a threshold, redefining the node as an occupied leaf node in a downscaled octree; and for each node of the octree that includes eight leaf subnodes where a number of those that are occupied among the eight leaf subnodes is less than or equal to the threshold, redefining the node as an unoccupied leaf node in the downscaled octree.

20. Method according to claim 10, characterized by further comprising: decoding a value representing a scaling amount to be applied to point cloud geometry data, wherein the scaling of point cloud geometry data comprises scaling down the point cloud geometry data according to the value representing the scaling amount; and providing the value representing the scaling amount to be applied to point cloud geometry data for the residual learning network. Petition 870250089998, dated 10 / 02 / 2025, pp. 180 / 206

Images