Detection and indication of geometric reconstruction artifacts in point clouds based on local density

Abstract

An electronic device and method for detecting and indicating geometry reconstruction artifacts are provided. The electronic device acquires a reference point cloud, encodes the reference point cloud to generate coded point cloud data, and decodes the coded point cloud data to generate a test point cloud. The electronic device further generates a first local density map and a second local density map for points of the reference point cloud and the test point cloud, respectively. The electronic device generates a final density map based on the first local density map and the second local density map, and further generates supplemental information based on the final density map. The supplemental information includes missing point data corresponding to regions of the test point cloud that contain artifacts, such as holes, or includes a descriptor of the region that contains the artifact. The electronic device signals the supplemental information to a decoder.

Description

[Technical Field]

[0001] Cross-reference to related applications / incorporation by reference This application claims the benefit of priority to U.S. Patent Application No. 18 / 147,944, filed with the United States Patent and Trademark Office on December 29, 2022. The above application is incorporated herein by reference in its entirety.

[0002] Various embodiments of the present disclosure relate to three-dimensional (3D) point cloud encoding and reconstruction, and more particularly, to the detection and indication of geometry reconstruction artifacts in point clouds based on local density estimation. [Background technology]

[0003] Advances in the field of point cloud compression (PCC) have led to the development of PCC techniques that enable efficient representation of data related to 3D points in a point cloud. Typically, a point cloud contains many unstructured 3D points. Each 3D point can contain geometric information (e.g., the coordinates (i.e., x, y, and z coordinates) of the corresponding 3D point) and attribute information (e.g., color information, reflectance information, or opacity information) associated with the corresponding 3D point. Thus, each 3D point in a point cloud can contain a significant amount of data. Point cloud data may require compression via PCC techniques for storage, processing, or transmission of the point cloud. However, reconstructing the point cloud in a PCC decoder (through decoding) may introduce artifacts in the reconstructed point cloud (such as loss of 3D points in the original point cloud). To have high-fidelity 3D geometry data for various applications, such as game development or 3D video, it may be necessary to detect and then mitigate such artifacts in the reconstructed point cloud geometry. Summary of the Invention

[0004] The limitations and disadvantages of conventional approaches will become apparent to those skilled in the art by comparing the described system with certain aspects of the present disclosure illustrated in the remainder of this application and with reference to the drawings.

[0005] Provided are electronic devices and methods for detection and indication of geometry reconstruction artifacts in point clouds based on local density, substantially as shown and / or described in connection with at least one of the figures and more fully set forth in the claims.

[0006] These and other features and advantages of the present disclosure will become apparent from a consideration of the following detailed description of the disclosure when taken in conjunction with the accompanying drawings, in which like reference characters refer to like elements throughout. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 illustrates an exemplary network environment for detection and indication of geometry reconstruction artifacts in point clouds based on local density, in accordance with an embodiment of the present disclosure. [Figure 2] FIG. 2 is a block diagram illustrating the example electronic device of FIG. 1 in accordance with an embodiment of the present disclosure. [Figure 3] FIG. 1 illustrates an exemplary processing pipeline for estimation of local density maps of three-dimensional (3D) points of reference and test point clouds, according to embodiments of the present disclosure. [Figure 4A-4B] FIG. 4A illustrates an exemplary scenario for generating a first local density map of 3D points of a reference point cloud according to an embodiment of the present disclosure, and FIG. 4B illustrates an exemplary scenario for generating a second local density map of 3D points of a test point cloud according to an embodiment of the present disclosure. [Figure 5] FIG. 1 illustrates an exemplary scenario for generating a final density map based on local density maps associated with 3D points of a reference point cloud and a test point cloud, according to an embodiment of the present disclosure. [Figure 6]FIG. 1 illustrates an exemplary processing pipeline for generating supplemental information and signaling it to a point cloud compression (PCC) decoder, according to an embodiment of the present disclosure. [Figure 7] FIG. 1 illustrates an exemplary processing pipeline for generating supplemental information and signaling it to a PCC decoder, according to an embodiment of the present disclosure. [Figure 8] FIG. 1 illustrates an exemplary processing pipeline for mitigating geometric reconstruction artifacts in a reconstructed point cloud based on received supplemental information, in accordance with an embodiment of the present disclosure. [Figure 9] FIG. 1 illustrates an exemplary processing pipeline for mitigating geometric reconstruction artifacts in a reconstructed point cloud based on received supplemental information, in accordance with an embodiment of the present disclosure. [Figure 10] 1 is a flowchart illustrating the operation of an exemplary method for detection and indication of geometric reconstruction artifacts in point clouds based on local density, in accordance with an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0008] The disclosed electronic device method for detecting and indicating geometry reconstruction artifacts in point clouds based on local density can find embodiments described below. Exemplary aspects of the present disclosure provide an electronic device (e.g., a computing device, a gaming console, or a smart wearable device) that can indicate regions of a point cloud that contain geometry reconstruction artifacts to facilitate mitigation of geometry reconstruction artifacts appearing in the point cloud during reconstruction.

[0009] The electronic device can acquire a reference point cloud of an object and encode the reference point cloud to generate encoded point cloud data. At the encoder side, the encoded point cloud data can be decoded to generate a test point cloud. The electronic device can generate a first local density map of three-dimensional (3D) points of the reference point cloud and a second local density map of the 3D points of the test point cloud. The electronic device can generate a final density map based on a comparison between the first local density map and the second local density map. The electronic device can then generate supplementary information based on the final density map. The supplementary information can include missing point data corresponding to regions of the test point cloud that include geometry reconstruction artifacts. Alternatively, the supplementary information can include one or more descriptors of regions that include geometry reconstruction artifacts. The electronic device can signal the supplementary information to a point cloud compression (PCC) decoder.

[0010] An original point cloud generated based on a 3D scan of an object (using image and depth information) can be compressed through encoding the 3D points of the original point cloud using a suitable PCC method. Because each 3D point of the original point cloud contains a significant amount of data, the point cloud can be compressed for storage, processing, and transmission. The need for compressing the original point cloud arises from memory and computational constraints of electronic devices and bandwidth constraints of communication channels. Once the original point cloud is compressed at the encoder side, encoded point cloud data (representing the original point cloud or the geometry of the object) can be transmitted to the decoder side. The original point cloud can be reconstructed based on decoding the received encoded point cloud data. The decoding operation may result in artifacts such as holes being added to the reconstructed point cloud. When a neural network-based codec is used to realize the encoding and decoding operations, the size, quantity, or number of such artifacts in the reconstructed point cloud may become more noticeable. In some cases, such artifacts may degrade the quality of the reconstructed point cloud and affect the usability of such point clouds in 3D rendering applications such as augmented reality or 3D computer games.

[0011] To address such issues, the proposed electronic device can detect geometry reconstruction artifacts in test point clouds generated by the electronic device. The geometry reconstruction artifacts can be detected due to the presence of holes that appear in the test point cloud after decoding of the encoded point cloud data. The electronic device can encode a reference point cloud to generate coded point cloud data and generate the test point cloud to determine artifacts, such as holes, that appear after reconstruction of the reference point cloud at the decoder side. The electronic device can indicate one or more regions of the test point cloud that contain such artifacts. The electronic device can signal supplemental information to a PCC decoder along with the coded point cloud data (related to the reference point cloud). The signaling of the supplemental information can enable the PCC decoder to predict one or more regions of the point cloud that may contain geometry reconstruction artifacts. The PCC decoder can use the supplemental information to perform appropriate actions to mitigate geometry reconstruction artifacts that may appear in one or more regions of the point cloud after decoding of the coded point cloud data. For example, missing point data (extracted from the supplemental information) can be used to fill regions (e.g., hole regions) in the point cloud to mitigate geometry reconstruction artifacts. Thus, the supplemental information can facilitate a PCC decoder to efficiently post-process the reconstructed point cloud and remove maximum reconstruction artifacts, such as holes, from the reconstructed point cloud. This can result in improved quality and fidelity of the post-processed point cloud, ultimately making such point clouds suitable for applications such as 3D game development or 3D animation.

[0012] FIG. 1 illustrates an exemplary network environment for detecting and indicating geometry reconstruction artifacts in point clouds based on local density, in accordance with an embodiment of the present disclosure. FIG. 1 illustrates a network environment 100. The network environment 100 includes an electronic device 102, an external device 104, and a server 106. The electronic device 102 can communicate with the external device 104 and the server 106 over one or more networks (e.g., a communication network 108). The electronic device 102 can include a codec 110. The codec 110 can receive a reference point cloud 112 as input and generate coded point cloud data 114 as output based on the reference point cloud 112. Furthermore, the codec 110 can also generate a test point cloud 116 as output based on the coded point cloud data 114. The external device 104 can include a PCC decoder 118.

[0013] The electronic device 102 may include suitable logic, circuitry, interfaces, and / or code that can be configured to detect geometry reconstruction artifacts, such as holes, in one or more regions of a test point cloud (e.g., test point cloud 116) and generate supplemental information including volumetric missing point data or descriptors corresponding to such regions. The electronic device 102 may transmit the encoded point cloud data 114 and supplemental information to the external device 104 (i.e., PCC decoder 118) via a communications network 108. Examples of the electronic device 102 may include, but are not limited to, a computing device, a video conferencing system, an augmented reality (AR) device, a virtual reality (VR) device, a mixed reality (MR) device, a gaming console, a smart wearable device, a mainframe machine, a server, a computer workstation, and / or a consumer electronics (CE) device.

[0014] The external device 104 may include suitable logic, circuitry, interfaces, and / or code that may be configured to communicate with the electronic device 102 via the communications network 108. According to an embodiment, the external device 104 may be configured to receive the coded point cloud data 114 and the supplemental information from the electronic device 102. The external device 104 may be configured to reconstruct a point cloud based on the coded point cloud data 114 via a PCC decoder 118. The external device 104 may be further configured to apply post-processing operations to the reconstructed point cloud using the supplemental information to generate a final point cloud that is substantially free of geometric reconstruction artifacts (e.g., holes). Examples of the external device 104 may include, but are not limited to, a display system, a computing device, a gaming device, a mobile phone, a television, or any electronic device capable of storing or rendering multimedia content.

[0015] The server 106 may include suitable logic, circuitry, interfaces, and / or code that can be configured to generate a reference point cloud (e.g., reference point cloud 112) for a 3D object(s) in 3D space. The server 106 may be configured to generate the reference point cloud using image and depth information for the object(s). The server 106 may store the reference point cloud and information related to the reference point cloud for future requirements. The server 106 may receive a request for the reference point cloud 112 from the electronic device 102. The server 106 may send the reference point cloud 112 to the electronic device 102 based on the received request. The server 106 may perform operations through web applications, cloud applications, HTTP requests, repository operations, file transfers, and the like. Example implementations of the server 106 may include, but are not limited to, a database server, a file server, a web server, an application server, a mainframe server, a cloud computing server, or combinations thereof. In at least one embodiment, the server 106 may be implemented as multiple distributed cloud-based resources using multiple technologies known to those skilled in the art. Those skilled in the art will appreciate that the scope of the present disclosure may not be limited to implementing server 106 and electronic device 102 as two separate entities. In some embodiments, the functionality of server 106 may be incorporated, in whole or at least in part, into electronic device 102 without departing from the scope of the present disclosure.

[0016] The communication network 108 may include a communication medium that enables the electronic device 102, the external device 104, and the server 106 to communicate with each other. The communication network 108 may be a wired or wireless communication network. Examples of the communication network 108 may include, but are not limited to, the Internet, a Wireless Fidelity (Wi-Fi) network, a personal area network (PAN), a local area network (LAN), or a metropolitan area network (MAN). The electronic device 102 may be configured to connect to the communication network 108 according to a variety of wired and wireless communication protocols. Examples of such wired and wireless communication protocols include, but are not limited to, at least one of Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE802.11, Light Fidelity (Li-Fi), 802.16, IEEE802.11s, IEEE802.11g, multi-hop communication, wireless access point (AP), device-to-device communication, cellular communication protocols, and Bluetooth (BT) communication protocols.

[0017] The codec 110 may include suitable logic, circuitry, code, and / or interfaces that may be configured to encode the reference point cloud 112 of an object and decode the encoded point cloud data 114. According to an embodiment, the codec 110 may be configured to divide the reference point cloud 112 into multiple 3D blocks and encode each 3D block of the reference point cloud 112 to generate an encoded bitstream. In some embodiments, the codec 110 may also be configured to generate supplemental information for the PCC decoder 118.

[0018] According to an embodiment, the codec 110 can be configured to reconstruct the reference point cloud 112 based on the coded point cloud data 114. The reconstruction can generate a test point cloud 116. By way of example and not limitation, the codec 110 can be implemented as a deep neural network (in the form of trained model data including computer-executable instructions) that can run on a graphics processing unit (GPU), a central processing unit (CPU), a tensor processing unit (TPU), a reduced instruction set computing (RISC) processor, an application specific integrated circuit (ASIC) processor, or a complex instruction set computing (CISC) processor, coprocessor, and / or combinations thereof.

[0019] The codec 110 may include encoder-decoder pairs, such as encoder and decoder-1, ..., encoder and decoder-N. Each such encoder-decoder pair may include an associated neural network model. For example, encoder and decoder-1 1104A may include DNN model-1.

[0020] In some other embodiments, the codec 110 can be implemented as a DNN on dedicated hardware in conjunction with other computational circuitry of the electronic device 102. Examples of dedicated hardware can include, but are not limited to, field programmable gate arrays (FPGAs), programmable logic devices (PLDs), ASICs, programmable ASICs (PL-ASICs), application specific integrated components (ASSPs), and systems-on-chips (SOCs) based on standard microprocessors (MPUs) or digital signal processors (DSPs). According to some embodiments, the codec 110 can also interface with a GPU to parallelize the operation of the codec 110.

[0021] The reference point cloud 112 corresponds to a geometric representation of an object in 3D space and may include a series of 3D points. Each 3D point in the reference point cloud 112 may be a voxel. According to one embodiment, the electronic device 102 may obtain the reference point cloud 112 from the server 106. According to another embodiment, the electronic device 102 may generate the reference point cloud 112 based on a 3D scan of an object. The 3D points in the reference point cloud 112 may include geometric information (i.e., the location or coordinates of the 3D point in the reference point cloud 112) and attribute information associated with the 3D point. The attribute information may include color information, reflectance information, opacity information, normal vector information, material identifier information, texture information, etc.

[0022] The encoded point cloud data 114 may be generated based on the encoding of the reference point cloud 112. The encoded point cloud data 114 may include encoded geometric information (i.e., encoding of 3D point cloud geometry) related to the reference point cloud 112. In some embodiments, the encoded point cloud data 114 may include encoded attribute information related to the reference point cloud 112. The electronic device 102 may transmit the encoded point cloud data 114 as an encoded bitstream to the external device 104 (i.e., PCC decoder 118).

[0023] The test point cloud 116 may be a reconstructed point cloud corresponding to the reference point cloud 112. The test point cloud 116 may also include geometric information (e.g., coordinates of 3D points in the test point cloud 116) and attribute information associated with the 3D points in the test point cloud 116. The electronic device 102 may reconstruct the test point cloud 116 based on the encoded point cloud data 114. After reconstruction, one or more regions of the test point cloud 116 may contain geometric reconstruction artifacts, such as holes. A hole in the reconstructed point cloud (i.e., the test point cloud 116) may correspond to a 3D volume where 3D points corresponding to the original point cloud (i.e., the reference point cloud 112) were lost as a result of the reconstruction.

[0024] The PCC decoder 118 may include suitable logic, circuitry, and / or interfaces that can be configured to receive the encoded point cloud data 114 and the supplemental information as an encoded bitstream from the electronic device 102 via the communications network 108. The supplemental information can facilitate mitigating geometric reconstruction artifacts that may be detected in the reconstructed point cloud. The PCC decoder 118 may further be configured to reconstruct a point cloud based on the encoded point cloud data 114. The encoded point cloud data 114 (or the encoded bitstream) may include encoded point cloud data associated with multiple blocks. The PCC decoder 118 may individually decode each encoded block of the multiple encoded blocks to generate multiple decoded blocks. The PCC decoder 118 may further binarize and merge the multiple decoded blocks to reconstruct the point cloud. The PCC decoder 118 may use the received supplemental information to predict regions of the reconstructed point cloud that may contain geometric reconstruction artifacts. The PCC decoder 118 may use the received supplemental information to post-process the point cloud to generate a final point cloud. The final point cloud may have minimal reconstruction artifacts (eg, holes) or may be free of geometric reconstruction artifacts.

[0025] The PCC decoder 118 can be implemented as a deep neural network (in the form of computer-executable code) on a GPU, CPU, TPU, RISC processor, ASIC processor, CISC processor, coprocessor, and / or combinations thereof. In some other embodiments, the PCC decoder 118 can be implemented as a deep neural network on dedicated hardware in conjunction with other computational circuitry of the electronic device 102. In such implementations, the PCC decoder 118 can be associated with a particular form factor on a particular computational circuit. Examples of particular computational circuits include, but are not limited to, FPGAs, PLDs, ASICs, PL-ASICs, ASSPs, and standard MPU- or DSP-based SOCs. In some embodiments, the PCC decoder 118 can also interface with a GPU to parallelize the operation of the PCC decoder 118.

[0026] During operation, the electronic device 102 can be configured to acquire a reference point cloud 112 of an object. According to some embodiments, the reference point cloud can be an uncompressed point cloud of the object. The reference point cloud 112 can represent the shape of the object using a plurality of 3D points. In some embodiments, the electronic device 102 can generate the reference point cloud 112 based on multiple images of the object captured from different viewpoints and depth information associated with the object. The multiple images and depth information can be acquired via one or more image capture devices and depth sensors associated with the electronic device 102.

[0027] The electronic device 102 may be further configured to encode the reference point cloud 112 to generate coded point cloud data 114 (i.e., a coded bitstream). The electronic device 102 may locally decode the coded point cloud data 114 to generate test point cloud data 116. The test point cloud 116 may be generated to determine whether reconstruction of the reference point cloud 112 (or decoding of the coded point cloud data 114) by a PCC decoder (e.g., PCC decoder 118) may result in the generation of geometry reconstruction artifacts in the reconstructed point cloud. The electronic device 102 may predict that the PCC decoder 118 may detect geometry reconstruction artifacts in one or more regions in the reconstructed point cloud. The prediction may be based on detection of geometry reconstruction artifacts in the test point cloud 116 in one or more regions of the test point cloud 116.

[0028] The electronic device 102 may be further configured to generate a first local density map of the 3D points of the reference point cloud 112. The first local density map may include an estimate of the local density of each 3D point of the reference point cloud 112. The local density at a 3D point may indicate the number of 3D points in a neighborhood of the 3D point of the reference point cloud 112. The local density at a 3D point may be estimated further based on the volume of a region of the reference point cloud 112 that forms the neighborhood of the 3D point. According to an embodiment, the electronic device 102 may determine the count (or number) of 3D points in the reference point cloud 112 and the dimensions of a structure (e.g., a cubic region) that can contain (or surround) all the 3D points of the reference point cloud 112. The electronic device 102 may further determine a radius (Rref) based on the number of 3D points of the reference point cloud 112 and the dimensions of the structure. The electronic device 102 may also determine the volume (Vref) of a sphere of the determined radius (i.e., Rref).

[0029] Once the volume of the sphere (Vref) is determined, the electronic device 102 can determine, for each 3D point in the reference point cloud 112, the number of 3D points in the neighborhood of the corresponding 3D point. The neighborhood of the 3D point in the reference point cloud 112 can include one or more points included in a sphere of radius Rref. The center (i.e., the origin) of the sphere can be the position of the 3D point in the reference point cloud 112. The number of 3D points in the neighborhood of each 3D point in the reference point cloud 112 can be determined based on the reference point cloud 112, Rref, and the position of the corresponding 3D point in the reference point cloud 112. The electronic device 102 can further estimate the local density of each 3D point in the reference point cloud 112 based on the number and volume Vref of 3D points in the neighborhood of the corresponding 3D point. The electronic device 102 can estimate a first local density map based on the estimated local density of each 3D point in the reference point cloud 112. Similarly, the electronic device 102 can generate a second local density map of the 3D points of the test point cloud 116. The second local density map can include an estimate of the local density at each location of the test point cloud 116 that corresponds to the location of the 3D points of the reference point cloud 112. According to an embodiment, the electronic device 102 can determine a count (or number) of 3D points in the test point cloud 116 and dimensions of a structure (e.g., a cubic region) that can contain (enclose) all of the 3D points of the test point cloud 116. The electronic device 102 can further determine a radius (Rtest) based on the number of 3D points and the dimensions of the structure. The electronic device 102 can also determine a volume (Vtest) of a sphere of the determined radius (i.e., Rtest).

[0030] For each 3D point in the reference point cloud 112, the electronic device 102 can determine the number of 3D points in a neighborhood of the location in the test point cloud 116 that corresponds to the location of the 3D point in the reference point cloud 112. The neighborhood of the location in the test point cloud 116 that corresponds to the location of the 3D point in the reference point cloud 112 can include one or more points in the test point cloud 116. Such points can be included in a sphere of radius Rtest. The center (i.e., the origin) of the sphere can be the location of the test point cloud 116. The number of 3D points in a neighborhood of the location in the test point cloud 116 (that corresponds to the location of the 3D point in the reference point cloud 112) can be determined based on the test point cloud 116, Rtest, and the location (i.e., coordinates) of the 3D point in the reference point cloud 112.

[0031] The electronic device 102 may further estimate a local density at each location in the test point cloud 116 (corresponding to a location of a 3D point in the reference point cloud 112) based on Vtest and the number of 3D points in the vicinity of the corresponding location in the test point cloud 116. The electronic device 102 may further estimate a second local density map based on the estimated local density at each location in the test point cloud 116 that corresponds to the location of the 3D point in the reference point cloud 112.

[0032] According to one embodiment, the first local density map may represent a uniform local density of 3D points of the reference point cloud 112. The first local density map is uniform because the estimated local densities of the 3D points of the reference point cloud are similar. Similarly, the second local density map may represent a non-uniform local density of 3D points of the test point cloud 116. The second local density at each location of the test point cloud 116 may vary based on the corresponding location of the 3D point of the reference point cloud 112 and the number of 3D points of the test point cloud 116 that are near the location of the test point cloud 116. Details of generating an exemplary first local density map and an exemplary second local density map are described in FIGS. 3, 4A, and 4B.

[0033] The electronic device 102 may be further configured to generate a final density map based on a comparison between the first local density map and the second local density map. By way of example and not limitation, the final density map may be referred to as a difference density map, which may be generated based on local density differences between the first local density map and the second local density map. Specifically, for each 3D point in the reference point cloud 112, a local density difference value may be determined based on the difference between the local density at the corresponding 3D point in the reference point cloud 112 and the local density at the location of the corresponding 3D point in the test point cloud 116. The final density map may be composed of local density difference values ​​associated with each 3D point in the reference point cloud 112.

[0034] According to an embodiment, the electronic device 102 can detect geometry reconstruction artifacts in the test point cloud 116 based on local density difference values ​​in the final density map. According to an embodiment, the geometry reconstruction artifacts can correspond to holes in the test point cloud 116. The final density map can indicate, for 3D points corresponding to a first region of the final density map, that the local density difference value of the first region is greater than a density threshold. The final density map can indicate, for 3D points corresponding to a second region of the final density map, that the local density difference value of the second region is less than a density threshold.

[0035] The electronic device 102 can detect the presence of a geometry reconstruction artifact (e.g., a hole) in a region of the test point cloud 116 that corresponds to a first region of the final density map because the local density difference value in the first region is higher than other regions of the final density map. A higher local density difference value can indicate a larger local density difference between the corresponding regions of the reference point cloud 112 and the test point cloud 116. A high local density difference value can also indicate the loss of 3D points in the test point cloud 116 in generating the test point cloud 116.

[0036] The electronic device 102 may be further configured to generate supplemental information based on the final density map. The supplemental information may include missing point data corresponding to regions of the test point cloud 116 that contain geometry reconstruction artifacts (e.g., holes). The missing point data may include 3D points of the reference point cloud 112 that may be missing from the test point cloud 116 due to losses introduced during decoding and / or reconstruction operations (e.g., 3D points lost due to holes in the point cloud). The electronic device 102 may determine such points as missing point data from the test point cloud 116 based on the reference point cloud 112 and the final density map.

[0037] According to an embodiment, the electronic device 102 can obtain a 3D mask from the final density map based on a threshold value. The 3D mask can be used to determine missing point data. The threshold value can correspond to a percentage (e.g., 5%) of voxels in the reference point cloud 112 that can be selected as representing geometry reconstruction artifacts in the reference point cloud 112.

[0038] According to another embodiment, the electronic device 102 can obtain a 3D mask from the final density map based on a threshold value (e.g., a density threshold) for determining missing point data. The threshold value can correspond to a local density difference value. The local density difference value associated with a certain percentage (e.g., 5%) of the 3D points of the reference point cloud 112 (e.g., a first set of 3D points) can be greater than the threshold value. The local density difference value associated with the first set of 3D points can constitute a region (e.g., a first region) of the final density map. Similarly, the local density difference value associated with a certain percentage (e.g., 95%) of the 3D points of the reference point cloud 112 (i.e., a second set of 3D points) can be less than the threshold value. The local density difference value associated with the second set of 3D points can constitute another region (e.g., a second region) of the final density map.

[0039] According to an embodiment, a 3D mask can be obtained by converting local density difference values ​​associated with a first 3D point set (i.e., a first region of the final density map) to "1" and local density difference values ​​associated with a second 3D point set (i.e., a second region of the final density map) to "0." The electronic device 102 can be further configured to generate missing point data based on applying the 3D mask to the reference point cloud 112. As a result of applying the 3D mask, the position (i.e., coordinates) of the first 3D point set within the reference point cloud 112 can be determined. The position of the first 3D point set and other point cloud data (i.e., geometric information and attribute information) associated with each of the first 3D point sets can constitute missing point data or supplemental information.

[0040] According to an embodiment, the supplemental information may include one or more descriptors of regions of the test point cloud 116 that contain geometry reconstruction artifacts. By way of example and not limitation, each such descriptor may be a volume descriptor indicating one or more parameters of a polyhedral structure. For example, the polyhedral structure may be an ellipsoid, and the volume descriptor may indicate the volume of the ellipsoid. The polyhedral structure may correspond to one of the regions of the test point cloud 116 that contain geometry reconstruction artifacts (i.e., holes). For example, a first region of the final density map may correspond to a region of the test point cloud 116 that contains a hole. Thus, the polyhedral structure (i.e., an ellipsoid) may indicate the first region of the final density map (where the local density difference between the first and second local density maps is greater than a threshold density). The supplemental information may include the volume of the ellipsoid and the region of the test point cloud 116 to which the ellipsoid corresponds.

[0041] The electronic device 102 may be further configured to calculate a set of quality metrics that define the quality of the test point cloud 116 (i.e., the quality of the reconstruction of the reference point cloud 112). The set of quality metrics may quantify the presence of geometric reconstruction artifacts (e.g., holes) in the test point cloud 116. The quality of the generated test point cloud 116 may be defined based on one or more of a first metric, a second metric, and a third metric. Each of the three metrics may be calculated based on the final density map. The supplemental information may include the first metric, the second metric, and the third metric.

[0042] According to an embodiment, the electronic device 102 can be configured to calculate a first quality metric as a ratio between the number of points in a region of the test point cloud 116 that includes a geometry reconstruction artifact and the total number of 3D points in the reference point cloud 112. The number of points in a region of the test point cloud 116 can be determined based on the final density map. A region (e.g., a first region) in the final density map in which the local density difference value is greater than a threshold value (i.e., a density threshold) may represent a hole. Note that one or more regions (other than the first region) in the test point cloud 116 may contain holes. However, the presence of a hole in such a region of the test point cloud 116 may be due to the presence of a hole in the corresponding region of the reference point cloud 112. Such a hole in the reference point cloud 112 may be a natural hole in the shape or geometry features of the 3D object (e.g., a hole in the mouth in a 3D head scan) or an artifact of the 3D scan (e.g., a hole related to a photogrammetric reconstruction). Holes in such regions of the test point cloud 116 may not be used in calculating the first quality metric.

[0043] According to an embodiment, the electronic device 102 can determine the number (and location) of the first 3D point set based on the missing point data (i.e., data generated based on applying a 3D mask to the reference point cloud 112). The location of the first 3D point set can correspond to an area of ​​the test point cloud 116 that contains a geometry reconstruction artifact (i.e., a hole). Once the number of 3D points in the first 3D point set is determined, a first metric can be calculated as the ratio of the number of 3D points in the first 3D point set to the total number of 3D points in the reference point cloud 112.

[0044] According to an embodiment, the electronic device 102 may be configured to calculate a second quality metric proportional to the inverse of the number of one or more descriptors whose volumetric parameters are above a volume threshold. The second quality metric may be expressed using the following equation (1): TIFF2026502252000001.tif13150(1) Here, "Q" represents a second quality metric, "f" is a function based on "1" which can be in its simplest form (i.e., f(p,q,r,Z)=1), the global density of the reference point cloud 112, or the dimensions of the bounding box of the reference point cloud 112 (containing all 3D points of the reference point cloud 112), and "N" specifies the number of descriptors whose volume parameter is above a volume threshold, one or more.

[0045] If the bounding box of the reference point cloud 112 is a cube and the dimensions of the bounding box are expressed using "p", "q", and "r", then the count of 3D points in the reference point cloud 112 can be expressed as "Z". The function "f" can be expressed using the following equation (2): TIFF2026502252000002.tif13150(2)

[0046] According to an embodiment, the global density of the reference point cloud 112 may be directly proportional to "Q" and the hole count may be inversely proportional to "Q". In such a case, the global density may correspond to the median density distribution or the median local density of the 3D points of the reference point cloud 112.

[0047] If there are no holes in the test point cloud 116, "N" equals "0" and "Q" equals "1" (i.e., highest quality). The descriptor may be a volumetric descriptor indicating a parameter (e.g., volume) of a polyhedral structure such as an ellipsoid. For example, three regions of the test point cloud 116 may contain holes. Such regions may correspond to three ellipsoids (i.e., a first ellipsoid, a second ellipsoid, and a third ellipsoid). The final density map may include a second region and a third region (in addition to the first region) where the local density difference between the first and second local density maps exceeds a threshold density. The three regions of the test point cloud 116 may correspond to the three regions of the final density map, and three ellipsoids may be used to represent the three regions of the final density map. The volume of the first region of the final density map may correspond to the volume of the first ellipsoid. Similarly, the volumes of the second and third regions of the final density map may correspond to the volumes of the second and third ellipsoids, respectively.

[0048] According to an embodiment, the electronic device 102 may be configured to determine the volume of each of the three ellipsoids and compare the volumes to a volume threshold. Based on the comparison, a second quality metric may be calculated. For example, if the volume of one of the three ellipsoids exceeds the volume threshold (i.e., N=1), the second quality metric may be calculated as 1 / 2. If the volumes of two of the three ellipsoids exceed the volume threshold, the second quality metric may be calculated as 1 / 3 (i.e., N=2). If the volumes of all the ellipsoids exceed the volume threshold, the second quality metric may be calculated as 1 / 4 (i.e., N=3).

[0049] According to an embodiment, the electronic device 102 can be configured to calculate the third quality metric as a weighted sum of the first quality metric and the second quality metric.

[0050] The electronic device 102 may be further configured to signal the supplemental information to the PCC decoder 118. According to an embodiment, the electronic device 102 may signal the encoded point cloud data 114 along with the supplemental information as an encoded bitstream. The PCC decoder 118 may receive the encoded point cloud data 114. The encoded point cloud data 114 may be decoded to generate a reconstructed point cloud. The reconstructed point cloud may include geometry reconstruction artifacts in one or more regions corresponding to one or more regions of the test point cloud 116 (i.e., region(s) containing geometry reconstruction artifacts). The supplemental information may be used to process the reconstructed point cloud to generate a final point cloud that is free of or contains a minimal number of geometry reconstruction artifacts (e.g., holes).

[0051] According to an embodiment, the electronic device 102 can be configured to determine the number of holes in the test point cloud 116 based on the final density map. The number of holes in the test point cloud 116 can be determined based on the number of 3D points whose local density difference value (determined based on the final density map) is greater than a threshold value (i.e., a density threshold). During the encoding stage, the electronic device 102 can adjust the number of bits per point (i.e., a 3D point) until the number of holes in the test point cloud 116 is minimized. The number of bits per point can be used to encode the reference point cloud 112, and adjusting the number of bits per point can enable an appropriate selection of the rate-distortion cost. The number of holes in the test point cloud 116 can be minimized when the number of bits used to encode the 3D points or blocks of the reference point cloud 112 results in the lowest rate-distortion cost. An optimal selection of the number of bits per point to use can significantly reduce the difference density value in certain regions of the final density map (regions where the local density difference between the first and second local density maps is greater than the threshold density).

[0052] FIG. 2 is a block diagram illustrating the example electronic device of FIG. 1 , in accordance with an embodiment of the present disclosure. The description of FIG. 2 is provided with reference to elements of FIG. 1 . FIG. 2 illustrates a block diagram 200 of electronic device 102. Electronic device 102 may include circuitry 202, memory 204, input / output (I / O) devices 206, and a network interface 208. In at least one embodiment, memory 204 may include codec 110. In at least one embodiment, I / O devices 206 may also include a display device 210. Circuitry 202 may be communicatively coupled to memory 204, I / O devices 206, and network interface 208 via wired or wireless communication of electronic device 102.

[0053] Circuitry 202 may include suitable logic, circuits, and interfaces that can be configured to execute program instructions associated with different operations performed by electronic device 102. Circuitry 202 may include one or more specialized processing units, which may be implemented as an integrated processor or a collection of processors that collectively perform the functions of one or more specialized processing units. Circuitry 202 may be implemented based on multiple processor technologies known in the art. Example implementations of circuitry 202 may be an X86-based processor, a graphics processing unit (GPU), a reduced instruction set computing (RISC) processor, an application-specific integrated circuit (ASIC) processor, a complex instruction set computing (CISC) processor, a microcontroller, a central processing unit (CPU), and / or other computing circuitry.

[0054] The memory 204 may include suitable logic, circuitry, and / or interfaces that may be configured to store instructions executable by the circuitry 202. The memory 204 may be configured to store an operating system and associated applications. The memory 204 may be further configured to store the reference point cloud 112, the encoded point cloud data 114, and the test point cloud 116 of the acquired object. The memory 204 may be further configured to store the first local density map, the second local density map, the final density map, and supplemental information. In at least one embodiment, the memory 204 may include a codec 110 that may be implemented as a combination of programmable instructions stored in the memory 204 and a logic unit (or programmable logic unit) on a hardware circuit of the electronic device 102. Examples of implementations of memory 204 include, but are not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), hard disk drive (HDD), solid state drive (SSD), CPU cache, and / or secure digital (SD) cards.

[0055] The I / O devices 206 may include suitable logic, circuitry, interfaces, and / or code that may be configured to receive user input. The I / O devices 206 may be further configured to provide output in response to the user input. The I / O devices 206 may include a variety of input and output devices that may be configured to communicate with the circuit 202. Examples of input devices may include, but are not limited to, a touch screen, a keyboard, a mouse, a joystick, and / or a microphone. Examples of output devices may include, but are not limited to, a display device 210.

[0056] The network interface 208 may include suitable logic, circuitry, interfaces, and / or code that may be configured to establish communications between the electronic device 102, the external device 104, and the server 104 over the communications network 108. The network interface 208 may be implemented using various known technologies to support wired or wireless communications between the electronic device 102 and the communications network 108. The network interface 208 may include, but is not limited to, an antenna, a radio frequency (RF) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a coder-decoder (CODEC) chipset, a subscriber identity module (SIM) card, and / or a local buffer.

[0057] The network interface 208 can communicate via wireless communication with networks such as the Internet, an intranet, and / or wireless networks such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN). The wireless communication can use any of a number of communication standards, protocols, and technologies, such as Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Long Term Evolution (LTE), Fifth Generation (5G) New Radio (NR), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (WiFi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Light Fidelity (Li-Fi), Wi-MAX, protocols for email, instant messaging, and / or short message service (SMS).

[0058] The display device 210 may include suitable logic, circuitry, interfaces, and / or code that can be configured to render the reference point cloud 112 and the test point cloud 116 on a display screen of the display device 210. According to some embodiments, the display device 210 may include a touch screen for receiving user input. The display device 210 may be implemented through a number of known technologies, such as, but not limited to, liquid crystal display (LCD) display, light emitting diode (LED) display, plasma display, or organic LED (OLED) display technology, and / or other display technologies. According to some embodiments, the display device 210 may represent a display screen of a smart glasses device, a 3D display, a see-through display, a projection display, an electrochromic display, and / or a transparent display.

[0059] The functions or operations performed by electronic device 102, such as those described in Figure 1, may be performed by circuitry 202. The operations performed by circuitry 202 are described in detail, for example, in Figures 3, 4, 5, 6, 7, 8, and 9.

[0060] Figure 3 illustrates an exemplary processing pipeline for estimating local density maps of 3D points of reference and test point clouds, according to an embodiment of the present disclosure. Figure 3 is described with reference to elements of Figures 1 and 2. Figure 3 illustrates an exemplary processing pipeline 300 for estimating local density maps of reference and test point clouds 302 and 304. The processing pipeline 300 illustrates a series of operations starting at 306 and ending at 328. The series of operations may be performed by the circuitry 202 of the electronic device 102.

[0061] The reference point cloud 302 can be an exemplary point cloud that is the same as or similar to the reference point cloud 112 of Figure 1. Similarly, the test point cloud 304 can be an exemplary point cloud that is the same as or similar to the test point cloud 116 of Figure 1. The reference point cloud 302 can be an uncompressed point cloud of an object, such as a human head. The circuit 202 can generate the test point cloud 304 based on the encoded point cloud data (obtained after encoding the reference point cloud 302).

[0062] In 306, the control point group (PC ref ) 302. In at least one embodiment, the circuit 202 can be configured to determine a first bounding box for the reference point cloud 302. The first bounding box can include all 3D points of the reference point cloud 302. By way of example and not limitation, the first bounding box can correspond to a cubic region. The circuit 202 can be configured to determine the dimensions (i.e., length, width, and height) of the cubic region such that the cubic region includes (or surrounds) all 3D points of the reference point cloud 302.

[0063] 308, the control point group (PC ref) 302. In at least one embodiment, the circuit 202 can be configured to determine the first number of 3D points in the reference point cloud 302.

[0064] At 310, a first radius may be calculated. In at least one embodiment, the circuit 202 may be configured to calculate a first radius to be used to sample the 3D points of the reference point cloud 302. The first radius may be calculated based on a first bounding box and a first number of 3D points of the reference point cloud 302. The circuit 202 may determine a surface of the reference point cloud 302 based on the dimensions of the first bounding box. For example, the dimensions of the first bounding box may be a length "a", a width "b", and a height "c". The surface "S" of the reference point cloud 302 may be calculated based on the first bounding box. R " can be determined as the square of the cube root of the volume of the first bounding box. The first bounding box can include multiple surfaces of equal dimensions, which can be determined based on the dimensions of the first bounding box (i.e., cubic region). For example, the volume of the first bounding box can be determined as (a × b × c). Therefore, "SR" can be determined using equation (3) as follows: TIFF2026502252000003.tif13150(3) Once multiple surfaces have been determined, the circuit 202 determines the number of 3D points in the reference point cloud 302 (i.e., "N points-reference ") and the number of 3D points in each surface of the reference point cloud 302 ("N R "). Thus, R1 can be determined using equation (4) as follows: TIFF2026502252000004.tif13150(4)

[0065] At 312, a first spherical volume may be determined. In at least one embodiment, the circuit 202 may be configured to determine the first spherical volume based on the first radius. The first volume may be expressed using equation (5) as follows: TIFF2026502252000005.tif13150(5)

[0066] At 314, a first count (i.e., a first number) of 3D points in a neighborhood of each 3D point of reference point cloud 302 can be determined. In at least one embodiment, circuit 202 can be configured to determine the first number of 3D points in a neighborhood of each 3D point of reference point cloud 302 from the 3D points of reference point cloud 302. The neighborhood of each 3D point of reference point cloud 302 can include a plurality of points contained within a first sphere of a first radius and a first sphere volume. The first number of 3D points (i.e., a plurality of points) in a neighborhood of each 3D point of reference point cloud 302 can be determined by counting the number of 3D points in a neighborhood of a corresponding 3D point (p ref ) (i.e., coordinates x, y, and z), the reference point cloud 302, and the first radius. The center (or origin) of the first sphere can be determined based on the position of the corresponding 3D point (i.e., p ref It can be a position (x, y, z).

[0067] At 316, a first local density of each 3D point of the reference point cloud 302 can be estimated. In at least one embodiment, the circuit 202 is configured to estimate a first local density of each 3D point of the reference point cloud 302 by computing a first local density of the first sphere and a first number of 3D points in a neighborhood of the corresponding 3D point (e.g., p ref ) can be configured to estimate a first local density of each 3D point of the reference point cloud 302. Specifically, ref The first local density of the reference points 302 may be estimated as a ratio of a first number of 3D points in a neighborhood of the 3D point pref to the first spherical product. The circuit 202 may be further configured to generate a first local density map based on the first local density of each 3D point of the reference points 302.

[0068] In 318, the test point cloud (PCtest ) 304. In at least one embodiment, the circuit 202 can be configured to determine a second bounding box for the test point cloud 304. By way of example and not limitation, the second bounding box can be a cubic region that includes (or surrounds) all 3D points of the test point cloud 304.

[0069] In 320, the test point cloud (PC test ) 304. In at least one embodiment, the circuit 202 can be configured to determine the second number of 3D points in the test point cloud 304.

[0070] At 322, a second radius may be calculated. In at least one embodiment, the circuit 202 may be configured to calculate a second radius to be used to sample the 3D points of the test point cloud 304. The second radius may be calculated based on a second bounding box and a second number of 3D points of the test point cloud 304. The calculation of the second radius is similar to the calculation of the first radius. For example, the dimensions of the second bounding box may be a length "x", a width "y", and a height "z". The surface "S" of the test point cloud 304 may be a surface "S". T " can be determined as the square of the cube root of the volume of the second bounding box. The second bounding box can include multiple surfaces of equal dimensions that can be determined based on the dimensions of the second bounding box (i.e., cubic region). For example, the volume of the second bounding box can be determined as (x x y x z). Therefore, "S T can be determined using equation (6) as follows: TIFF2026502252000006.tif13150(6) Once multiple surfaces have been determined, the circuit 202 determines the number of 3D points in the test point cloud 304 (i.e., "N points-test ”) and the number of 3D points in each surface of the test point cloud 304 (“N T"). Therefore, R2 can be determined using equation (7) as follows: TIFF2026502252000007.tif13150(7)

[0071] A second spherical volume may be calculated at 324. In at least one embodiment, the circuit 202 may be configured to calculate the second spherical volume based on the second radius. The second volume may be expressed using equation (8) as follows: TIFF2026502252000008.tif13150(8)

[0072] At 326, a second count (i.e., a second number) of 3D points in a neighborhood of each location in the test point cloud 304 (corresponding to a location of a 3D point in the reference point cloud 302) can be determined. In at least one embodiment, the circuit 202 can be configured to determine the second number of 3D points in a neighborhood of each location in the test point cloud 304 from the 3D points of the test point cloud 304. The neighborhood of each location in the test point cloud 304 (corresponding to a location of a 3D point in the reference point cloud 302) can include a plurality of points of the test point cloud 304. Such points of the test point cloud 304 can be located within a second sphere of a second radius and a second sphere volume. The second number of 3D points (i.e., a plurality of points) in a neighborhood of each location in the test point cloud 304 can be determined based on the coordinates (i.e., x, y, and z coordinate values) of the corresponding location in the reference point cloud 302, including the 3D point, the test point cloud 304, and the second radius. The center (or origin) of the second sphere can be a location within the test point cloud 304 .

[0073] At 328, a second local density may be estimated at each location of the test point cloud 304 that corresponds to the location of a 3D point in the reference point cloud 302. In at least one embodiment, the circuit 202 may be configured to estimate the second local density at each location of the test point cloud 304 based on the second spherical volume and a second number of 3D points in a vicinity of the corresponding location in the test point cloud 304. The second local density at a location in the test point cloud 304 may be estimated as a ratio of the second number of 3D points in a vicinity of the location to the second spherical volume. The circuit 202 may be further configured to generate a second local density map based on the second local density at the location in the test point cloud 304.

[0074] Figure 4A illustrates an exemplary scenario for generating a first local density map of 3D points of a reference point cloud, according to an embodiment of the present disclosure. Figure 4A is described with reference to elements of Figures 1, 2, and 3. Figure 4A illustrates an exemplary scenario 400A. The exemplary scenario 400A illustrates the reference point cloud 302 of Figure 3 and a set of 3D points of a region 402 of the reference point cloud 302. Additionally, an exemplary first local density map 404 is also illustrated.

[0075] The circuit 202 can be configured to generate a first local density map 404 of the 3D points of the reference point cloud 302. The first local density map 404 can be generated based on a first local density of each 3D point of the reference point cloud 302. According to an embodiment, the circuit 202 can be configured to calculate a first radius of a first sphere 406 based on a first number of 3D points in the reference point cloud 302. The circuit 202 can also be configured to calculate a first volume of the first sphere 406 based on the first radius of the first sphere 406.

[0076] For the 3D point 408, a first local density may be estimated based on a first volume of the first sphere 406 and a first number of 3D points in a neighborhood of the 3D point 408. The center of the first sphere 406 may correspond to a position (i.e., coordinates) of the 3D point 408, and the neighborhood of the 3D point 408 may correspond to an area encompassed by the first sphere 406. The first number of 3D points in the neighborhood of the 3D point 408 may be determined based on the position (i.e., coordinates) of the 3D point 408, the reference point cloud 302, and a first radius of the first sphere 406.

[0077] FIG. 4B illustrates an exemplary scenario for generating a second local density map of 3D points of a test point cloud, according to an embodiment of the present disclosure. The description of FIG. 4B is provided in conjunction with elements of FIGS. 1, 2, 3, and 4A. FIG. 4B illustrates an exemplary scenario 400B. The exemplary scenario 400B illustrates the test point cloud 304 of FIG. 3 and a set of 3D points in a region 410 of the test point cloud 304. The region 410 of the test point cloud 304 may correspond to the region 402 of the reference point cloud 302. The region 410 of the test point cloud 304 may include geometry reconstruction artifacts (e.g., holes). Therefore, the number of 3D points in the region 410 of the test point cloud 304 may be less than the number of 3D points in the region 402 of the reference point cloud 302. Also illustrated is an exemplary second local density map 412 that can be generated for the 3D points of the test point cloud 304.

[0078] The circuit 202 can be configured to generate a second local density map 412 based on a second local density at each location of the test point cloud 304 that corresponds to the location of the 3D points of the reference point cloud 302. The circuit 202 can obtain the locations of the 3D points of the reference point cloud 302 and generate the second local density map 412 based on the obtained locations. According to an embodiment, the circuit 202 can be configured to calculate a second radius of the second sphere 414 based on a second number of 3D points in the test point cloud 304. The circuit 202 can further calculate a second volume of the second sphere 414 based on the second radius of the second sphere 414.

[0079] According to an embodiment, the circuit 202 may be configured to estimate a second local density at a location 416 in the test point cloud 304 (corresponding to a location of the 3D point 408 in the reference point cloud 302) based on a second volume of the second sphere 414 and a second number of 3D points in a vicinity of the location 416 in the test point cloud 304. A center of the second sphere 414 may correspond to the location 416, and a vicinity of the location 416 may correspond to an area encompassed by the second sphere 414. The second number of 3D points in the vicinity of the location 416 may be determined based on the coordinates of the location of the 3D point 408 (in the reference point cloud 302), the test point cloud 304, and a second radius of the second sphere 414.

[0080] Figure 5 illustrates an exemplary scenario for generating a final density map based on local density maps associated with 3D points of a reference point cloud and a test point cloud, according to an embodiment of the present disclosure. Figure 5 is described with reference to elements of Figures 1, 2, 3, 4A, and 4B. Figure 5 illustrates an exemplary scenario 500. The exemplary scenario 500 shows an exemplary final density map 502 that can be generated based on the first local density map 404 and the second local density map 412. The final density map 502 can be referred to as a difference density map because it is generated based on the difference between the first local density map 404 and the second local density map 412.

[0081] The circuit 202 can be configured to generate a final density map 502 based on a comparison between the first local density map 404 and the second local density map 412. The comparison can include determining a difference between the first local density map 404 and the second local density map 412 to generate the final density map 502. According to an embodiment, the final density map 502 can include a first region (e.g., region 504 and region 506) in which a local density difference between the first local density map 404 and the second local density map 412 is greater than that of a second region (e.g., region 508) of the final density map 502. The local density difference between the first local density map 404 and the second local density map 412 can correspond to a difference between a first local density at a location of each 3D point of the reference point cloud 302 and a second local density at a location in the test point cloud 304 that corresponds to the location of each 3D point of the reference point cloud 302. The difference between the first local density and the second local density at the location associated with each 3D point may be referred to as a local density difference value. Thus, the final density map 502 may include a local density difference value associated with the location of each 3D point in the reference point cloud 302. The local density difference value associated with the location of the first set of 3D points constituting the first region (i.e., regions 504 and 506) may be greater than a density threshold, and the local density difference value associated with the location of the second set of 3D points constituting the second region 508 may be less than the density threshold. A high local density difference value in a particular region of the final density map 502 may indicate poor reconstruction quality in the corresponding region of the test point cloud 304 and the presence of reconstruction artifacts, such as holes, in such region. Similarly, a low local density difference value in a particular region of the final density map 502 may indicate good reconstruction quality in the corresponding region of the test point cloud 304 and the absence of reconstruction artifacts, such as holes, in such region. The circuit 202 can be configured to detect geometry reconstruction artifacts in a first region (i.e., regions 504 and 506) of the final density map 502 based on variations in local density difference values ​​and comparison of such values ​​with a density threshold.

[0082] Figure 6 illustrates an exemplary processing pipeline for generating supplemental information and signaling it to a point cloud compression (PCC) decoder, according to an embodiment of the present disclosure. Figure 6 is described with reference to elements in Figures 1, 2, 3, 4A, 4B, and 5. Figure 6 illustrates an exemplary processing pipeline 600 for generating supplemental information and signaling it. The processing pipeline 600 illustrates a sequence of operations that may begin at 602 and end at 608. The sequence of operations may be performed by the circuitry 202 of the electronic device 102. The supplemental information may be generated based on the reference point cloud 302, the test point cloud 304 of Figure 3, and the final density map 502 of Figure 5. The supplemental information may include missing point data corresponding to regions of the test point cloud 304 that contain geometry reconstruction artifacts, such as holes.

[0083] At 602, missing point data 610 can be generated. In at least one embodiment, the circuit 202 can be configured to generate the missing point data 610 based on the reference point cloud 302 and the final density map 502. The missing point data 610 can include 3D points of the reference point cloud 302 that may be missing from the test point cloud 304 due to decoding or reconstruction issues. The missing point data 610 can correspond to regions of the test point cloud 304 that contain holes (i.e., holes that only appear after reconstruction).

[0084] According to an embodiment, the circuit 202 can be configured to obtain a 3D mask from the final density map 502 based on a threshold value (e.g., a density threshold value) for determining missing point data 610. The threshold value can correspond to a percentage (e.g., 5%) of the voxels (or 3D points) of the reference point cloud 302 that can be selected as representing geometry reconstruction artifacts in the reference point cloud 302. In some embodiments, the threshold value can correspond to a density threshold value used to detect geometry reconstruction artifacts. The threshold value (i.e., the density threshold value) can correspond to a local density difference value and can enable selection of a percentage of the 3D points of the reference point cloud 302 (e.g., a first 3D point set of a first number of 3D points). The circuit 202 can select 3D points from the first number of 3D points of the reference point cloud 302 based on a comparison of the local density difference value (associated with the position of each 3D point in the reference point cloud 302) to the threshold value. The local density difference value associated with the selected 3D points (i.e., the first set of 3D points) may be greater than a threshold (i.e., the density threshold) and may be greater than the local density difference values ​​of the other 3D points (i.e., the second set of 3D points).

[0085] The circuit 202 may be further configured to convert local density difference values ​​associated with the selected 3D points to “1” and convert local density difference values ​​associated with other 3D points of the reference point cloud 302 to “0.” Thus, a 3D mask may be obtained based on converting the first region of the final density map 502 (i.e., regions 504 and 506) to “1” and converting the second region 508 of the final density map 502 to “0.” Once the 3D mask is obtained, the circuit 202 may generate missing point data 610 based on applying the 3D mask to the reference point cloud 302. The missing point data 610 may include positions (or coordinates) of the selected 3D points (i.e., the first set of 3D points) within the reference point cloud 302, as well as other point cloud data (i.e., geometric information and attribute information) associated with the first set of 3D points.

[0086] At 604, supplemental information can be generated based on the missing point data 610 and the test point cloud 304. In at least one embodiment, the circuit 202 can be configured to generate the supplemental information based on the missing point data 610 and the test point cloud 304. Generating the supplemental information can involve filtering one or more 3D points from the selected first 3D point set (i.e., missing points) that may not represent holes in the test point cloud 304. The filtering can be necessary to recover from errors that may occur during the selection of the first 3D point set. An error can occur when a local density difference value associated with a 3D point's location in the first 3D point set is close to a density threshold. This error can cause the 3D point to be selected as a missing point. Without filtering, the reconstructed point cloud can contain two 3D points at the same location (one point reconstructed by the PCC decoder 118 without relying on supplemental information, and the other point included in the reconstructed point cloud based on the missing point data 610).

[0087] The circuit 202 can determine one or more first 3D points of a first set of 3D points included in the test point cloud 304 from the missing point data 610. The positions of the first 3D points in the test point cloud 304 can correspond to the positions of second 3D points of the reference point cloud 302. The one or more first 3D points of the first set of 3D points can serve as replicas of one or more second 3D points of the reference point cloud 302. Thus, the circuit 202 can remove data associated with such first 3D points from the missing point data 610. The circuit 202 can obtain filtered missing points data 612 based on the removal. The filtered missing points data 612 can be included in the supplemental information.

[0088] At 606, an encoded bitstream can be generated based on the reference point cloud 302. In at least one embodiment, the circuit 202 can be configured to generate an encoded bitstream (including encoded point cloud data).

[0089] At 608, the coded bitstream and the supplemental information may be transmitted. In at least one embodiment, the circuit 202 may be configured to transmit (or signal) the coded bitstream and the supplemental information (i.e., filtered missing point data 612) to the PCC decoder 118. The external device 104 may receive the coded bitstream and the supplemental information. The coded point cloud data associated with the reference point cloud 302 may be decoded to generate a reconstructed point cloud. The reconstructed point cloud may be processed using the supplemental information (i.e., filtered missing point data 612) to generate a final point cloud (with zero or a minimal number of reconstruction artifacts).

[0090] Figure 7 illustrates an exemplary processing pipeline for generating supplemental information and signaling it to a PCC decoder, according to an embodiment of the present disclosure. Figure 7 is described with reference to elements in Figures 1, 2, 3, 4A, 4B, 5, and 6. Figure 7 illustrates an exemplary processing pipeline 700 for generating supplemental information and signaling it. The processing pipeline 700 illustrates a sequence of operations that may begin at 702 and end at 706. The sequence of operations may be performed by the circuitry 202 of the electronic device 102. The supplemental information may be generated based on the final density map 502 of Figure 5. The generated supplemental information may include descriptors of regions of the test point cloud 304 that contain geometry reconstruction artifacts, such as holes.

[0091] At 702, supplemental information may be generated based on the final density map 502. In at least one embodiment, the circuit 202 may be configured to generate the supplemental information based on the final density map 502. Specifically, the circuit 202 may determine regions of the final density map 502 where a local density difference between the first local density map 404 and the second local density map 412 is greater than a threshold value (i.e., a density threshold). Such regions of the final density map 502 may correspond to regions of the test point cloud 304 that include geometry reconstruction artifacts.

[0092] The supplemental information may include descriptors of regions of the test point cloud 304 that contain geometry reconstruction artifacts (e.g., holes). The descriptors may indicate determined regions of the final density map 502 (because the determined regions of the final density map 502 correspond to regions of the test point cloud 304 that contain holes). For example, three descriptors may indicate three determined regions of the final density map 502. In each of the three regions, the local density difference between the first local density map 404 and the second local density map 412 may be greater than a threshold. The three determined regions of the final density map 502 may correspond to three regions of the test point cloud 304 that contain holes.

[0093] According to an embodiment, each descriptor may be a volumetric descriptor that indicates a polyhedral structure. The polyhedral structure may correspond to a region of the test point cloud 304 that contains a hole. For example, the polyhedral structure may be an ellipsoid. Thus, three ellipsoids (e.g., a first ellipsoid 708A, a second ellipsoid 708B, and a third ellipsoid 708C) may correspond to three regions of the test point cloud 304 that contain a hole. The three ellipsoids may also indicate three determined regions of the final density map 502 that represent geometry reconstruction artifacts. According to an embodiment, supplemental information may be generated based on the volume of each of the three ellipsoids and the region of the test point cloud 304 that each of the three ellipsoids can correspond to. The generated supplemental information may include the volumes of the series of ellipsoids 710 (including the first ellipsoid 708A, the second ellipsoid 708B, and the third ellipsoid 708C) and the region of the test point cloud 304 that contains a hole.

[0094] At 704, an encoded bitstream can be generated based on the reference point cloud 302. In at least one embodiment, the circuit 202 can be configured to generate an encoded bitstream (including encoded point cloud data).

[0095] At 706, the coded bitstream and the supplemental information may be transmitted. In at least one embodiment, the circuit 202 may be configured to transmit (or signal) the coded bitstream and the supplemental information (i.e., the set of ellipsoids 710) to the PCC decoder 118. The external device 104 may receive the coded bitstream and the supplemental information. The coded point cloud data associated with the reference point cloud 302 may be decoded to generate a reconstructed point cloud. The reconstructed point cloud may be processed using the supplemental information (i.e., the set of ellipsoids 710) to generate a final point cloud (with zero or a minimal number of reconstruction artifacts).

[0096] FIG. 8 illustrates an example processing pipeline for mitigating geometric reconstruction artifacts in a reconstructed point cloud based on received supplemental information, in accordance with an embodiment of the present disclosure. The description of FIG. 8 is provided with reference to elements in FIGS. 1, 2, 3, 4A, 4B, 5, 6, and 7. FIG. 8 illustrates an example processing pipeline 800 for detecting and mitigating geometric reconstruction artifacts in a reconstructed point cloud. The processing pipeline 800 illustrates a sequence of operations that may begin at 802 and end at 808. The sequence of operations may be performed by the external device 104. Also illustrated are a reconstructed point cloud 810 and a final point cloud 812. The reconstructed point cloud 810 may include geometric reconstruction artifacts, such as holes. The external device 104 may use the supplemental information to mitigate the geometric reconstruction artifacts in the reconstructed point cloud 810 and generate the final point cloud 812.

[0097] At 802, the encoded bitstream and the supplemental information may be received. In at least one embodiment, the external device 104 may be configured to receive the encoded bitstream and the supplemental information from the electronic device 102. The external device 104 or the PCC decoder 118 may extract coded point cloud data associated with the reference point cloud 302 from the received coded bitstream. According to an embodiment, the supplemental information may include filtered missing point data 612 (described in FIG. 6).

[0098] The PCC decoder 118 can predict locations in the point cloud (to be reconstructed) that may contain geometry reconstruction artifacts. The predictions can be based on the filtered missing point data 612. Such locations can correspond to locations of 3D points of the reference point cloud 302 that are included in the filtered missing point data 612.

[0099] At 804, the received encoded bitstream can be decoded. In at least one embodiment, the external device 104 can be configured to decode the encoded bitstream using a PCC decoder 118. The PCC decoder 118 can be configured to decode the encoded point cloud data of the bitstream to generate a reconstructed point cloud 810. The PCC decoder 118 can detect multiple regions (e.g., region 810A or region 810B) in the reconstructed point cloud 810 as regions containing geometry reconstruction artifacts (e.g., hole 810C in region 810A and hole 810D in region 810B). The detected multiple regions (i.e., regions 810A and 810B) can contain 3D points that are identical to the 3D points included in the filtered missing point data 612. In some embodiments, the PCC decoder 118 can determine a local density of 3D points in the detected multiple regions. The PCC decoder 118 can further verify that the local density in the detected regions (i.e., regions 810A and 810B) is less than the local density in regions of the reconstructed point cloud 810 outside the detected regions (e.g., region 810E).

[0100] At 806, the supplemental information can be applied to the reconstructed point cloud 810. In at least one embodiment, the external device 104 can be configured to apply the received supplemental information (i.e., the filtered missing point data 612) to the reconstructed point cloud 810. The external device 104 can determine the location (i.e., coordinates) of each 3D point included in the filtered missing point data 612 based on the supplemental information. The external device 104 can combine the filtered missing point data 612 with the reconstructed point cloud 810.

[0101] At 808, a final point cloud 812 can be generated. In at least one embodiment, the external device 104 can be configured to generate the final point cloud 812. The generation of the final point cloud 812 can be based on a combination of the filtered missing point data 612 and the reconstructed point cloud 810 that includes holes (i.e., holes 810C and 810D). For example, this combination can transform regions 810A and 810B of the reconstructed point cloud 810 into regions 812A and 812B, respectively, of the final point cloud 812. Thus, holes in multiple regions of the reconstructed point cloud 810 can be mitigated by applying supplemental information.

[0102] FIG. 9 illustrates an example processing pipeline for mitigating geometric reconstruction artifacts in a reconstructed point cloud based on received supplemental information, in accordance with an embodiment of the present disclosure. The description of FIG. 9 is provided in conjunction with elements of FIGS. 1, 2, 3, 4A, 4B, 5, 6, 7, and 8. FIG. 9 illustrates an example processing pipeline 900 for detecting and mitigating geometric reconstruction artifacts in a reconstructed point cloud. The processing pipeline 800 illustrates a sequence of operations that may begin at 902 and end at 908. The sequence of operations may be performed by the external device 104. Also illustrated are a reconstructed point cloud 910 and a final point cloud 912. The reconstructed point cloud 910 may include geometric reconstruction artifacts. The external device 104 may use the supplemental information to mitigate the geometric reconstruction artifacts in the reconstructed point cloud 910 and generate the final point cloud 912.

[0103] At 902, an encoded bitstream and supplemental information may be received. In at least one embodiment, the external device 104 may be configured to receive the encoded bitstream and supplemental information from the electronic device 102. The external device 104 or the PCC decoder 118 may extract encoded point cloud data associated with the reference point cloud 302 from the received encoded bitstream. According to an embodiment, the supplemental information may correspond to a series of ellipsoids 710 (i.e., a first ellipsoid 708A, a second ellipsoid 708B, and a third ellipsoid 708C).

[0104] The PCC decoder 118 can predict regions of the point cloud (to be reconstructed) that may contain geometry reconstruction artifacts (e.g., holes). The prediction can be based on a series of ellipsoids, as the series of ellipsoids corresponds to regions of the test point cloud 304 that contain holes. Such regions of the test point cloud 304 can correspond to predicted locations of the point cloud (to be reconstructed). The PCC decoder 118 can further predict the volume of each region of the point cloud (to be reconstructed) that may contain holes. The prediction can be based on the volume of each ellipsoid in the series of ellipsoids 710. For example, the PCC decoder 118 can predict that a first region of the point cloud (to be reconstructed) is likely to contain a hole. The volume of the first region can correspond to the volume of a first ellipsoid in the series of ellipsoids.

[0105] The encoded bitstream may be decoded at 904. In at least one embodiment, the external device 104 may be configured to decode the encoded bitstream via a PCC decoder 118 to generate a reconstructed point cloud 910.

[0106] At 906, the supplemental information can be applied to the reconstructed point cloud 910. In at least one embodiment, the external device 104 can be configured to apply the received supplemental information (i.e., the set of ellipsoids 710) to the reconstructed point cloud 910. The PCC decoder 118 can detect that multiple regions of the reconstructed point cloud 910 may contain holes. The external device 104 can determine locations within such regions of the reconstructed point cloud 910 where point cloud data needs to be inserted to remove holes from the reconstructed point cloud 910.

[0107] The external device 104 may further request the electronic device 102 to transmit encoded point cloud data and further supplemental information associated with the 3D points of the reference point cloud 302. The locations of the 3D points of the reference point cloud 302 for which associated encoded point cloud data is requested may correspond to determined locations within the multiple regions of the reconstructed point cloud 910. Holes may be alleviated based on the insertion of point cloud data (obtained from the reference point cloud 302) at the determined locations within the multiple regions of the reconstructed point cloud 910. The external device 104 may receive the point cloud data as further supplemental information from the electronic device 102. In another embodiment, an interpolation operation may be applied to fill the locations within the multiple regions of the reconstructed point cloud 910 with 3D points.

[0108] At 908, a final point cloud 912 may be generated. In at least one embodiment, the external device 104 may be configured to generate the final point cloud 912 based on the insertion of point cloud data at determined locations within the multiple regions of the reconstructed point cloud 910. The insertion may allow for the mitigation of holes in the reconstructed point cloud 910 that may otherwise appear after the generation of the reconstructed point cloud 910.

[0109] Figure 10 is a flowchart illustrating operations of an exemplary method for detection and indication of geometry reconstruction artifacts in point clouds based on local density, according to an embodiment of the present disclosure. Figure 10 is described with reference to elements in Figures 1, 2, 3, 4A, 4B, 5, 6, 7, 8, and 9. Figure 10 shows a flowchart 1000. Operations 1002-1018 can be performed by any computer system, such as electronic device 102 or circuitry 202 of electronic device 102 of Figure 1. Operations can begin at 1002 and proceed to 1004.

[0110] At 1004, a cloud of reference points 112 for the object may be obtained. In at least one embodiment, the circuit 202 may be configured to obtain the cloud of reference points 112 for the object. Details of obtaining the cloud of reference points 112 for the object are described, for example, in FIG. 1.

[0111] At 1006, the reference point cloud 112 may be encoded to generate encoded point cloud data 114. In at least one embodiment, the circuit 202 may be configured to encode the reference point cloud 112 to generate encoded point cloud data 114. Details of encoding the reference point cloud 112 are described, for example, in FIG.

[0112] At 1008, the encoded point cloud data 114 may be decoded to generate the test point cloud 116. In at least one embodiment, the circuit 202 may be configured to decode the encoded point cloud data 114 to generate the test point cloud 116. Details of decoding the encoded point cloud data 114 are described, for example, in FIG.

[0113] At 1010, a first local density map may be generated for the 3D points of the reference point cloud 112. In at least one embodiment, the circuit 202 may be configured to generate the first local density map for the 3D points of the reference point cloud 112. Details of generating the first local density map are described, for example, in Figures 1, 3, and 4A.

[0114] At 1012, a second local density map may be generated for the 3D points of the test point cloud 116. In at least one embodiment, the circuit 202 may be configured to generate the second local density map for the 3D points of the test point cloud 116. Details of generating the second local density map are described, for example, in Figures 1, 3, and 4B.

[0115] At 1014, a final density map may be generated based on a comparison between the first local density map and the second local density map. In at least one embodiment, the circuit 202 may be configured to generate the final density map based on a comparison between the first local density map and the second local density map. Details of generating the final density map are described, for example, in FIGS. 1 and 5.

[0116] At 1016, supplemental information can be generated based on the final density map. In at least one embodiment, the circuit 202 can be configured to generate the supplemental information based on the final density map. The supplemental information can include missing point data corresponding to regions of the test point cloud 116 that include geometry reconstruction artifacts. The supplemental information can further include one or more descriptors of regions of the test point cloud 116 that include geometry reconstruction artifacts. Details of generating the supplemental information are described, for example, in FIGS. 1, 6, and 7.

[0117] At 1018, the supplemental information can be signaled to the PCC decoder 118. In at least one embodiment, the circuit 202 can be configured to signal the supplemental information to the PCC decoder 118. Details of signaling the supplemental information are described, for example, in Figures 1, 6, and 7. Control can proceed to an end.

[0118] Although flowchart 1000 is depicted as discrete operations such as 1004, 1006, 1008, 1010, 1012, 1014, 1016, and 1018, the disclosure is not so limited. Thus, in some embodiments, such discrete operations may be further divided into additional operations, combined into fewer operations, or eliminated, depending on the implementation, without departing from the essence of the disclosed embodiments.

[0119] Various embodiments of the present disclosure may provide a non-transitory computer-readable medium and / or storage medium having stored thereon computer-executable instructions executable by a machine and / or computer to operate an electronic device (such as the electronic device 102). The computer-executable instructions may cause the machine and / or computer to perform operations including obtaining a reference point cloud 112 for the object. The operations may further include encoding the reference point cloud 112 to generate coded point cloud data 114. The operations may further include decoding the coded point cloud data 114 to generate a test point cloud 116. The operations may further include generating a first local density map for the 3D points of the reference point cloud 112. The operations may further include generating a second local density map for the 3D points of the test point cloud 116. The operations may further include generating a final density map based on a comparison between the first local density map and the second local density map. The operations may further include generating supplemental information based on the final density map. The supplemental information may include missing point data corresponding to regions of the test point cloud 116 that contain geometry reconstruction artifacts. The supplemental information may also include one or more descriptors of regions of the test point cloud 116 that contain geometry reconstruction artifacts. The operations may further include signaling the supplemental information to the PCC decoder 118.

[0120] An example embodiment of the present disclosure may include an electronic device (such as the electronic device 102 of FIG. 1 ) that may include a circuit (such as the circuit 202) that may be communicatively coupled to an external device (such as the external device 104 of FIG. 1 ). The electronic device 102 may further include a memory (such as the memory 204 of FIG. 2 ). The circuit 202 may be configured to acquire a reference point cloud 112 for an object. The reference point cloud 112 may be an uncompressed point cloud for the object. The circuit 202 may be further configured to encode the reference point cloud 112 to generate encoded point cloud data 114. The circuit 202 may be further configured to decode the encoded point cloud data 114 to generate a test point cloud 116. The circuit 202 may be further configured to generate a first local density map of 3D points of the reference point cloud 112. The circuit 202 may be further configured to generate a second local density map of 3D points of the test point cloud 116. The circuit 202 may be further configured to generate a final density map based on a comparison between the first local density map and the second local density map. The final density map may include a first region where a local density difference between the first local density map and the second local density map is greater than that of a second region of the final density map. The circuit 202 may be further configured to generate supplemental information based on the final density map. The supplemental information may include missing point data corresponding to regions of the test point cloud 116 that include geometry reconstruction artifacts. The supplemental information may also include one or more descriptors of regions of the test point cloud 116 that include geometry reconstruction artifacts. Each of the one or more descriptors may correspond to a volumetric descriptor indicating one or more parameters of a polyhedral structure. The polyhedral structure may correspond to one of the regions of the test point cloud 116 that include geometry reconstruction artifacts. The geometry reconstruction artifact may correspond to a hole in the test point cloud 116. The circuit 202 may be further configured to signal the supplemental information to the PCC decoder 118. The encoded point cloud data can be transmitted to the PCC decoder 118 as an encoded bitstream along with the supplemental information.

[0121] According to an embodiment, the circuit 202 may be further configured to determine a first bounding box for the reference point cloud 112. The circuit 202 may be further configured to determine a first number of 3D points for the reference point cloud 112. The circuit 202 may be further configured to calculate a first radius to use for sampling the 3D points for the reference point cloud 112. The first radius may be calculated based on the first bounding box and the first number of 3D points for the reference point cloud 112. The circuit 202 may be further configured to calculate a first sphere based on the first radius.

[0122] According to an embodiment, circuitry 202 may be further configured to determine, from the 3D points of reference point cloud 112, a first number of points in a neighborhood of each 3D point of reference point cloud 112. The first number of points in the neighborhood may be determined based on a position of the corresponding 3D point of reference point cloud 112, reference point cloud 112, and the first radius. Circuitry 202 may be further configured to determine a first local density at each 3D point of reference point cloud 112 based on the first spherical volume and the first number of points in the neighborhood of the corresponding 3D point. A first local density map may be generated based on the first local density at each 3D point of reference point cloud 112.

[0123] According to an embodiment, the circuit 202 may be further configured to determine a second bounding box for the test point cloud 116. The circuit 202 may be further configured to determine a second number of 3D points for the test point cloud 116. The circuit 202 may be further configured to calculate a second radius to use for sampling the 3D points of the test point cloud 116. The second radius may be calculated based on the second bounding box and the second number of 3D points for the test point cloud 116. The circuit 202 may be further configured to calculate a second sphere product based on the second radius.

[0124] According to an embodiment, the circuitry 202 may be further configured to determine, from the 3D points of the test point cloud 116, a second number of points in a neighborhood of each 3D point of the reference points 112 in the test point cloud 116. The second number of points in the neighborhood may be determined based on the location of the corresponding 3D point of the reference points 112 in the test point cloud 116, the test point cloud 116, and the second sphere radius. The circuitry 202 may be further configured to determine a second local density at each 3D point of the reference points 112 in the test point cloud 116 based on the second sphere product and the second number of points in the neighborhood of the corresponding 3D point. A second local density map may be generated based on the second local density at each 3D point of the reference points 112 in the test point cloud 116.

[0125] According to an embodiment, the circuit 202 may be further configured to obtain a 3D mask from the final density map based on a threshold. The circuit 202 may be further configured to generate missing point data based on applying the 3D mask to the reference point cloud 112. The threshold may correspond to a percentage of voxels selectable from the reference point cloud 112 or a percentage of local density differences between the first and second local density maps that exceed a difference threshold.

[0126] According to an embodiment, the circuit 202 may be further configured to determine the number of holes in the test point cloud 116 based on the final density map. The holes may correspond to geometry reconstruction artifacts. The circuit 202 may be further configured to adjust the number of bits per point until the number of holes is minimized. The number of bits per point may be used to encode each block of the reference point cloud 112.

[0127] According to an embodiment, the circuit 202 may be further configured to calculate, based on the final density map, a first quality metric as a ratio between the number of points in the region containing the geometry reconstruction artifact and the total number of 3D points in the reference point cloud. The supplemental information may further include the first quality metric.

[0128] According to an embodiment, the circuit 202 may be further configured to calculate a second quality metric based on one or more descriptors whose volumetric parameters exceed a volume threshold. The circuit 202 may use the final density map 502 for the calculation. The supplemental information may further include the second quality metric. According to an embodiment, the circuit 202 may be further configured to calculate a third quality metric as a weighted sum of the first quality metric and the second quality metric. The supplemental information may further include the third quality metric.

[0129] The present disclosure can be implemented in hardware or a combination of hardware and software. The present disclosure can be implemented in a centralized manner in at least one computer system, or in a distributed manner where different elements can be distributed across several interconnected computer systems. Any computer system or other device adapted to perform the methods described herein can be suitable. The combination of hardware and software can be a general-purpose computer system that includes a computer program that, when loaded and executed, can control the computer system to perform the methods described herein. The present disclosure can be implemented in hardware, including portions of integrated circuits that also perform other functions.

[0130] The present disclosure may also be embodied in a computer program product, which includes all features that enable the implementation of the methods described herein and which is capable of executing these methods when loaded into a computer system. A computer program in this context means any expression, in any language, code or notation, of a set of instructions intended to cause a system having information processing capabilities to perform a particular function, either directly, or after a) conversion into another language, code or notation, or b) reproduction in a different content form, or both.

[0131] While the present disclosure has been described with reference to several embodiments, those skilled in the art will recognize that various modifications may be made and equivalents may be substituted without departing from the scope of the disclosure. Additionally, many modifications may be made to adapt a particular situation or material to the teachings of the disclosure without departing from the scope of the disclosure. Therefore, it is not intended that the disclosure be limited to the particular embodiments disclosed, but rather, it is intended to include all embodiments falling within the scope of the appended claims.

Claims

1. 1. An electronic device, comprising: Obtain the reference points of the object, encoding the reference point group to generate encoded point group data; decoding the encoded point cloud data to generate a test point cloud; generating a first local density map of three-dimensional (3D) points of the reference point cloud; generating a second local density map of 3D points of the test point cloud; generating a final density map based on a comparison of the first local density map and the second local density map; Based on the final density map, Missing point data corresponding to regions of the test point cloud that contain geometry reconstruction artifacts; or one or more descriptors of the region containing the geometric reconstruction artifact; generating supplemental information including at least one of transmitting the supplemental information to a point cloud compression (PCC) decoder; An electronic device comprising a circuit configured to:

2. the reference point cloud is an uncompressed point cloud of the object; The electronic device of claim 1 .

3. the geometry reconstruction artifact corresponds to a hole in the test point cloud; The electronic device of claim 1 .

4. The circuit comprises: determining a first bounding box for the reference points; determining a first number of the 3D points of the reference point cloud; calculating a first radius to use for sampling the 3D points of the reference point cloud based on the first bounding box and the first number of the 3D points of the reference point cloud; calculating a first spherical volume based on the first radius; The electronic device of claim 1 further configured to:

5. The circuit comprises: determining, from the 3D points of the cloud of reference points, a first number of points in a neighborhood of each 3D point of the cloud of reference points based on a position of a corresponding 3D point of the cloud of reference points, the cloud of reference points, and the first radius; determining a first local density at each 3D point of the reference point cloud based on the first spherical product and the first number of points in the neighborhood of the corresponding 3D point; wherein the first local density map is generated based on the first local density at each 3D point of the reference point cloud.

5. The electronic device of claim 4.

6. The circuit comprises: determining a second bounding box for the test point cloud; determining a second number of the 3D points of the test point cloud; calculating a second radius to use for sampling the 3D points of the test point cloud based on the second bounding box and the second number of the 3D points of the test point cloud; calculating a second spherical volume based on the second radius; The electronic device of claim 1 further configured to:

7. The circuit comprises: determining a second number of points from the 3D points of the test point cloud in a neighborhood of each location of the test point cloud that corresponds to a location of a 3D point of the control point cloud based on the location of the 3D point of the control point cloud, the test point cloud, and the second sphere radius; determining a second local density at each location in the test point cloud corresponding to a location of a 3D point of the reference point cloud based on the second spherical product and the second number of points in the neighborhood of the corresponding location; wherein the second local density map is generated based on the second local density at each location.

7. The electronic device of claim 6.

8. the final density map includes a first region in which a local density difference between the first local density map and the second local density map is greater than that of a second region of the final density map; The electronic device of claim 1 .

9. The circuit comprises: obtaining a 3D mask from the final density map based on a threshold; generating the missing point data based on applying the 3D mask to the reference point cloud; The electronic device of claim 1 further configured to:

10. the threshold corresponds to a percentage of voxels selectable from the reference point cloud or a percentage of local density differences between the first local density map and the second local density map that exceed a difference threshold.

10. The electronic device of claim 9.

11. each of the one or more descriptors corresponds to a volume descriptor indicating one or more parameters of a polyhedral structure, the polyhedral structure corresponding to one of the regions containing the geometric reconstruction artifact; The electronic device of claim 1 .

12. The circuit comprises: determining a number of holes corresponding to the geometry reconstruction artifacts in the test point cloud based on the final density map; adjusting the number of bits per point to be used to encode each block of the reference points until the number of holes is minimized; The electronic device of claim 1 further configured to:

13. the circuitry is further configured to calculate, based on the final density map, a first quality metric as a ratio between a number of points in the region containing the geometry reconstruction artifact and a total number of the 3D points in the reference point cloud, and the supplemental information further comprises the first quality metric. The electronic device of claim 1 .

14. the circuitry is further configured to calculate a second quality metric based on the final density map as the inverse of a number of the one or more descriptors whose volumetric parameters exceed a volume threshold, and the supplemental information further comprises the second quality metric.

14. The electronic device of claim 13.

15. the circuitry is further configured to calculate a third quality metric as a weighted sum of the first quality metric and the second quality metric, and the supplemental information further includes the third quality metric.

15. The electronic device of claim 14.

16. the circuitry is further configured to transmit the encoded point cloud data together with the supplemental information as an encoded bitstream to the PCC decoder. The electronic device of claim 1 .

17. In an electronic device, obtaining a reference point cloud of an object; encoding the reference point group to generate encoded point group data; decoding the encoded point cloud data to generate a test point cloud; generating a first local density map of three-dimensional (3D) points of the reference point cloud; generating a second local density map of 3D points of the test point cloud; generating a final density map based on a comparison of the first local density map and the second local density map; Based on the final density map, Missing point data corresponding to regions of the test point cloud that contain geometry reconstruction artifacts; or one or more descriptors of the region containing the geometric reconstruction artifact; generating supplemental information including at least one of transmitting the supplemental information to a point cloud compression (PCC) decoder; A method comprising:

18. the reference point cloud is an uncompressed point cloud of the object; 18. The method of claim 17.

19. the geometry reconstruction artifact corresponds to a hole in the test point cloud; 18. The method of claim 17.

20. A non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed by a computer, obtaining a reference point cloud of an object; encoding the reference point group to generate encoded point group data; decoding the encoded point cloud data to generate a test point cloud; generating a first local density map of three-dimensional (3D) points of the reference point cloud; generating a second local density map of 3D points of the test point cloud; generating a final density map based on a comparison of the first local density map and the second local density map; Based on the final density map, Missing point data corresponding to regions of the test point cloud that contain geometry reconstruction artifacts; or one or more descriptors of the region containing the geometric reconstruction artifact; generating supplemental information including at least one of transmitting the supplemental information to a point cloud compression (PCC) decoder; A non-transitory computer-readable medium that causes the computer to perform operations including:

Images