Estimating density distortion metrics for processing point cloud geometries

Abstract

An electronic device and method for estimating a density distortion metric for processing point cloud geometry are provided. The electronic device acquires a reference point cloud, encodes the reference point cloud to generate encoded point cloud data, and decodes the encoded point cloud data to generate a test point cloud. The electronic device further generates a first local density map representing local density values ​​at points of the reference point cloud. The electronic device determines locations in the test point cloud corresponding to the locations of the points. The electronic device generates a second local density map representing the local density values ​​at the determined locations. The electronic device calculates a value of the density distortion metric based on the first local density map and the second local density map. The electronic device controls a display device to render a reconstruction quality of the test point cloud based on the calculated value.

Description

[Technical Field]

[0001] [CROSS-REFERENCE / INCORPORATION BY REFERENCE TO RELATED APPLICATIONS]

[0001] This application claims the benefit of priority to U.S. Patent Application No. 18 / 306,935, filed with the U.S. Patent and Trademark Office on April 25, 2023, which claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 368,266, filed on July 13, 2022, the entire contents of which are incorporated herein by reference.

[0002] Various embodiments of the present disclosure relate to encoding and reconstruction of three-dimensional (3D) point clouds. More particularly, various embodiments of the present disclosure relate to estimating density distortion metrics for processing of point cloud geometries. [Background technology]

[0003]

[0003] Advances in the field of point cloud compression (PCC) have led to the development of PCC techniques (e.g., geometry-based PCC, video-based PCC, or machine learning-based PCC) that enable efficient representation of data associated with three-dimensional (3D) points in a point cloud. Typically, a high-fidelity representation of the surface of a 3D object can be obtained using a 3D point cloud geometry that includes millions or billions of unstructured 3D points. Such geometries typically contain a large amount of point data that must be compressed using a PCC technique suitable for storing, processing, or transmitting the point cloud geometry. However, reconstruction of the point cloud geometry (by decoding) in a PCC decoder may result in the appearance of surface irregularities (e.g., deformation, erosion, or dilation) or artifacts (e.g., holes) in the reconstructed point cloud geometry. The appearance of artifacts or surface irregularities in the reconstructed point cloud geometry may degrade the reconstruction quality. The appearance of artifacts and surface irregularities may be independent of the number of bits that may be required to encode each 3D point of the reference point cloud geometry. Conventional methods for assessing the reconstruction quality of a reconstructed point cloud may be based on a metric whose value may be unaffected by the appearance of artifacts and irregularities in the reconstructed point cloud geometry.

[0004]

[0004] The limitations and disadvantages of conventional methods will become apparent to those skilled in the art by comparing the described system with certain aspects of the present disclosure illustrated in the remainder of this application with reference to the drawings. Summary of the Invention [Problem to be solved by the invention]

[0005]

[0005] An electronic device and method for estimating a density distortion metric for processing point cloud geometry is provided, as substantially shown in and / or described in connection with at least one figure and more fully set forth in the claims.

[0006]

[0006] These and other features and advantages of the present disclosure can be understood by considering the following detailed description of the disclosure in conjunction with the accompanying drawings in which like elements are designated by like reference numerals throughout. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 illustrates an exemplary network environment for estimating density distortion metrics for processing point cloud geometry, according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a block diagram illustrating an exemplary electronic device for estimating a density distortion metric for processing point cloud geometry, according to an embodiment of the present disclosure. [Figure 3] FIG. 1 illustrates an exemplary processing pipeline for estimating local density at 3D points of a reference point cloud and associated 3D locations in a test point cloud, according to an embodiment of the present disclosure. [Figure 4] FIG. 10 illustrates a comparison of a region of a reference point cloud with a corresponding region of a test point cloud, according to an embodiment of the present disclosure. [Figure 5] FIG. 1 illustrates an example scenario for generating a difference density map based on local density maps associated with a reference point cloud and a test point cloud, according to an embodiment of the present disclosure. [Figure 6] FIG. 1 illustrates an example scenario for computing a density distortion metric based on local density maps of a reference point cloud and a test point cloud, according to an embodiment of the present disclosure. [Figure 7A]10 is an exemplary graph illustrating the variation of density distortion metric values ​​with respect to the number of bits used to encode 3D points of a reference point cloud, according to an embodiment of the present disclosure. [Figure 7B] 10 is an exemplary graph illustrating the variation of density distortion metric values ​​with respect to the number of bits used to encode 3D points of a reference point cloud, according to an embodiment of the present disclosure. [Figure 8] FIG. 1 illustrates an exemplary processing pipeline for estimating local density at 3D points of reference data and associated 3D locations in test data, according to an embodiment of the present disclosure. [Figure 9] FIG. 10 illustrates a selection of regions of a reference point cloud and regions of a test point cloud for generating a local density map, according to an embodiment of the present disclosure. [Figure 10] FIG. 1 illustrates an example scenario for computing a density distortion metric based on local density maps of reference data and test data, according to an embodiment of the present disclosure. [Figure 11] 1 is a flowchart illustrating the operation of an exemplary method for estimating a density distortion metric for processing point cloud geometry, in accordance with an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0008]

[0018] Implementations described below can be found in disclosed electronic devices and methods for estimating a density distortion metric for processing (e.g., compressing, sampling, smoothing, sharpening, denoising, outlier removal, restoration, etc.) point cloud geometry. An exemplary aspect of the present disclosure provides an electronic device (e.g., a computing device, mainframe machine, or computer workstation) that can indicate the reconstruction quality of a point cloud based on the value of the density distortion metric. The value of the density distortion metric can indicate the appearance of geometry reconstruction artifacts (e.g., surface deformation, surface erosion, or surface dilation) in the reconstructed point cloud. The electronic device can obtain a reference point cloud of an object and encode the reference point cloud (using a point cloud codec within the electronic device) to generate encoded point cloud data. The electronic device can decode the encoded point cloud data (using the point cloud codec) to generate a test point cloud. The electronic device can then generate a first local density map of the reference point cloud. The first local density map can represent local density values ​​around each three-dimensional (3D) point of the reference point cloud. The electronic device can determine 3D positions on the test point cloud corresponding to positions of the 3D points of the reference point cloud and can further generate a second local density map of the test point cloud. The second local density map can represent local density values ​​corresponding to the determined 3D positions. The electronic device can then calculate values ​​of a density distortion metric for the test point cloud based on the first local density map and the second local density map. The electronic device can control a display device (e.g., a computer monitor, a tablet, or a smartphone) to render information associated with the reconstruction quality of the test point cloud based on the calculated values ​​of the density distortion metric.

[0009]

[0019] Different techniques are available for compressing the original point cloud. For example, these techniques can include geometry-based point cloud compression (PCC), video-based PCC (V-PCC), and machine learning-based multi-scale point cloud geometry compression (PCGC). The point cloud can be compressed to generate encoded point cloud data, which can be decoded to reconstruct the point cloud. The quality of the reconstructed point cloud can be evaluated using a quality assessment method (such as a full-reference method or a reduced-reference method) that utilizes a reference point cloud (i.e., the original point cloud). The full-reference quality assessment can evaluate the quality of the reconstructed point cloud using a point-to-point metric or a point-to-plane metric.

[0010]

[0020] Based on the value of the point-to-point metric or point-to-plane metric, it can be observed that the reconstruction quality of the reconstructed point cloud increases monotonically with increasing number of bits used to encode the original point cloud. However, the value of the point-to-point metric or point-to-plane metric may not take into account density-based distortions that may occur in the reconstructed point cloud due to inaccurate estimation of voxel occupancy. The density-based distortions may correspond to geometric reconstruction artifacts (e.g., holes) or irregularities (e.g., surface deformation, surface erosion, surface dilation) on the surface of the reconstructed point cloud. The geometric reconstruction artifacts or surface irregularities may appear in the reconstructed point cloud during reconstruction of the original point cloud (from the encoded point cloud data). The appearance of density-based distortions may lead to a decrease in the reconstruction quality of the point cloud because the surface contours of the original point cloud may not be preserved in the reconstructed point cloud (e.g., due to the presence of irregularities on the surface of the reconstructed point cloud). Furthermore, the appearance of density-based distortions may be independent of the number of bits used to encode the 3D points of the original point cloud. Therefore, to obtain a reliable measure of the reconstruction quality of a point cloud, the appearance of geometric reconstruction artifacts or surface irregularities in the reconstructed point cloud needs to be taken into account in estimating the value of the reconstruction quality assessment metric.

[0011]

[0021] To address these issues, the electronic device can calculate a density distortion metric. The density distortion metric can be used to determine whether the density of the original point cloud is preserved in the reconstructed point cloud. To determine the density distortion metric, the original point cloud and the reconstructed point cloud can be transformed from the geometry domain to the density domain. The transformation can include determining local density maps associated with the original point cloud and the reconstructed point cloud. Based on the local density maps, a change in density between the original point cloud and the reconstructed point cloud can be determined. The value of the density distortion metric can be inversely proportional to the degree of density change that can be determined between the original point cloud and the reconstructed point cloud. A higher degree of density change can indicate the appearance of surface irregularities or geometry reconstruction artifacts in the reconstructed point cloud. Thus, the value of the density distortion metric can indicate whether one or more density-based distortions (i.e., surface irregularities or geometry reconstruction artifacts) occurred in the reconstructed point cloud during reconstruction of the original point cloud. The density distortion metric can also be used to determine whether one or more machine learning models of a machine learning-based PCGC encoder need to be retrained. For example, a decrease in the value of the density distortion metric at a particular encoding bit rate (compared to the value of the density distortion metric at a lower encoding bit rate) may indicate that the machine learning model of the PCGC encoder needs to be retrained. Based on this indication, the weights of one or more layers of the machine learning model (used to encode the original point cloud at the particular encoding bit rate) may be adjusted to efficiently encode the original point cloud at the particular encoding bit rate.

[0012]

[0022] The original point cloud may be referred to as a reference point cloud, and the reconstructed point cloud may be referred to as a test point cloud. At each 3D point in the reference point cloud, a local density value may be determined based on the number of 3D points in the reference point cloud. A local density map associated with the reference point cloud may represent the local density of points surrounding each 3D point in the reference point cloud. At each location on the test point cloud (i.e., a location corresponding to the location of a 3D point in the reference point cloud), a local density value may be determined based on the number of 3D points in the test point cloud. A local density map associated with the test point cloud may represent the local density surrounding each location. A density distortion metric may be determined based on the difference between the local density map associated with the reference point cloud and the local density map associated with the test point cloud.

[0013]

[0023] FIG. 1 illustrates an exemplary network environment for estimating a density distortion metric for processing point cloud geometry, according to an embodiment of the present disclosure. Referring to FIG. 1, a network environment 100 is shown. The network environment 100 includes an electronic device 102, a display device 104, and a server 106. The electronic device 102 can communicate with the display device 104 and the server 106 via one or more networks (e.g., a communication network 108). The electronic device 102 can include a point cloud codec 110. Further shown are a reference point cloud 112, encoded point cloud data 114, and a test point cloud 116. The point cloud codec 110 can receive the reference point cloud 112 as input and can generate output based on the reference point cloud 112. The output can include the encoded point cloud data 114. Further, the point cloud codec 110 can generate the test point cloud 116 based on decoding the encoded point cloud data 114.

[0014]

[0024] The electronic device 102 may include suitable logic, circuitry, interfaces, and / or code that may be configured to calculate a value of a density distortion metric for the test point cloud 116. The value of the density distortion metric may be used to determine whether density distortion occurred in the test point cloud 116 during the reconstruction of the reference point cloud 112 or during the generation of the test point cloud 116. The electronic device 102 may further control the display device 104 to render the reconstruction quality (i.e., the calculated value) of the test point cloud 116. Examples of the electronic device 102 may include, but are not limited to, a computing device, a mainframe machine, a video conferencing system, an augmented reality (AR) device, a virtual reality (VR) device, a mixed reality (MR) device, a game console, a server, a computer workstation, and / or a consumer electronics (CE) device.

[0015]

[0025] The display device 104 may include suitable logic, circuitry, interfaces, and / or code that may be configured to communicate with the electronic device 102 via the communications network 108. The display device 104 may be configured to receive control instructions from the electronic device 102 via a suitable network interface. According to an embodiment, the display device 104 may receive control instructions for rendering information associated with the reconstruction quality of the test point cloud 116 on an electronic user interface. The display device 104 may further be configured to receive user input associated with a selection of rate-distortion points or number of bits that the point cloud codec 110 may use to encode the reference point cloud 112. The display device 104 may receive user input for initiating training of a machine-learning-based model of the point cloud codec 110. Examples of the display device 104 may include, but are not limited to, a display system, a computing device, a gaming device, a mobile phone, a television, or an electronic device capable of storing or rendering multimedia content.

[0016]

[0026] The server 106 may include suitable logic, circuitry, interfaces, and / or code that can be configured to generate a reference point cloud (e.g., reference point cloud 112) for a 3D object(s) in 3D space. The server 106 may process image and depth information for the object(s) to generate the reference point cloud. The server 106 may store the reference point cloud and information associated with the reference point cloud for future requirements. The server 106 may receive requests from the electronic device 102 and, based on the received request, may send the reference point cloud 112 to the electronic device 102. The server 106 may perform operations through a web application, a cloud application, an HTTP request, a repository operation, a file transfer, etc. Example implementations of the server 106 may include, but are not limited to, a database server, a file server, a web server, an application server, a mainframe server, a cloud computing server, or a combination thereof.

[0017]

[0027] In at least one embodiment, the server 106 may be implemented as multiple distributed cloud-based resources using a number of techniques known to those skilled in the art. Those skilled in the art will appreciate that the scope of the present disclosure is not limited to implementing the server 106 and the electronic device 102 as two separate entities. In some embodiments, the functionality of the server 106 may be incorporated in whole or at least in part into the electronic device 102 without departing from the scope of the present disclosure.

[0018]

[0028] The communication network 108 may include a communication medium through which the electronic device 102, the display device 104, and the server 106 can communicate with each other. The communication network 108 may be a wired or wireless communication network. Examples of the communication network 108 may include, but are not limited to, the Internet, a Wireless Fidelity (Wi-Fi) network, a personal area network (PAN), a local area network (LAN), or a metropolitan area network (MAN). The electronic device 102 may be configured to connect to the communication network 108 according to a variety of wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), Zig Bee, EDGE, IEEE 802.11, Light Fidelity (Li-Fi), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access point (AP), device-to-device communication, cellular communication protocols, and Bluetooth (BT) communication protocols.

[0019]

[0029] The point cloud codec 110 may include suitable logic, circuitry, code, and / or interfaces that can be configured to encode the reference point cloud 112 of an object and decode the encoded point cloud data 114. For example, the point cloud codec 110 may be configured to divide the reference point cloud 112 into multiple 3D blocks and encode each 3D block of the reference point cloud 112 to generate the encoded point cloud data 114. Each encoder may encode a block of the reference point cloud 112 at a particular bit rate. In some embodiments, the entire reference point cloud 112 may be encoded at a particular bit rate. After encoding, the point cloud codec 110 may be configured to reconstruct the reference point cloud 112 from the encoded point cloud data 114. The reconstruction may generate the test point cloud 116.

[0020]

[0030] According to one embodiment, the point cloud codec 110 may include a set of machine learning-based encoders and a set of machine learning-based decoders. By way of example and not limitation, the point cloud codec 110 may be implemented as a deep neural network (in the form of trained model data including computer-executable instructions) that may run on a graphics processing unit (GPU), a central processing unit (CPU), a tensor processing unit (TPU), a reduced instruction set computing (RISC), an application specific integrated circuit (ASIC), or a complex instruction set computing (CISC) processor, coprocessor, and / or combinations thereof. In another embodiment, the point cloud codec 110 may be implemented as a deep neural network on dedicated hardware interfaced with other computational circuitry of the electronic device 102. Examples of dedicated hardware may include, but are not limited to, field programmable gate arrays (FPGAs), programmable logic devices (PLDs), application specific integrated components (ASSPs), ASICs, programmable ASICs (PL-ASICs), and systems-on-chips (SOCs) based on standard microprocessors (MPUs) or digital signal processors (DSPs). According to one embodiment, the point cloud codec 110 may also be interfaced with a GPU to parallelize the operation of the point cloud codec 110 .

[0021]

[0031] The reference point cloud 112 may correspond to a geometric representation of an object in 3D space and may include a set of 3D points. Each 3D point in the reference point cloud 112 may be a voxel. According to one embodiment, the electronic device 102 may obtain the reference point cloud 112 from the server 106. According to another embodiment, the electronic device 102 may generate the reference point cloud 112 based on a 3D scan (high resolution) of the object. The 3D points in the reference point cloud 112 may include geometric information (i.e., the location or coordinates of the 3D point within the reference point cloud 112) and attribute information associated with the 3D point. The attribute information may include color information, reflectance information, opacity information, normal vector information, material identifier information, texture information, etc.

[0022]

[0032] Based on the encoding of the reference point cloud 112, encoded point cloud data 114 may be generated. The encoded point cloud data 114 may include encoded geometric information associated with the 3D points of the reference point cloud 112 (i.e., encoding of the 3D point cloud geometry). In particular embodiments, the encoded point cloud data 114 may include encoded attribute information associated with the 3D points of the reference point cloud 112.

[0023]

[0033] The test point cloud 116 may be a reconstructed point cloud corresponding to the reference point cloud 112. The test point cloud 116 may include geometric information (e.g., coordinates of 3D points in the test point cloud 116) and attribute information associated with the 3D points in the test point cloud 116. The electronic device 102 may reconstruct the test point cloud 116 based on the encoded point cloud data 114. After reconstruction, density distortions may appear in the test point cloud 116. Such distortions may indicate a change between the number of 3D points in the reference point cloud 112 and the number of 3D points in the test point cloud 116. The density distortions may include geometry reconstruction artifacts (e.g., holes) in one or more regions of the test point cloud 116 or irregularities (e.g., erosion, deformation, or dilation) in the surface of one or more regions of the test point cloud 116.

[0024]

[0034] In operation, the electronic device 102 can be configured to acquire a reference point cloud 112 of an object. According to an embodiment, the reference point cloud can be an uncompressed point cloud of the object. The reference point cloud 112 can represent the geometry of the object using multiple 3D points. In some embodiments, the electronic device 102 can generate the reference point cloud 112 based on multiple images of the object (captured from different viewpoints) and depth information associated with the object. The multiple images and depth information can be acquired via one or more image capture devices and depth sensors associated with the electronic device 102.

[0025]

[0035] The electronic device 102 may be further configured to encode the reference point cloud 112 to generate encoded point cloud data 114. The reference point cloud 112 (the 3D points of the reference point cloud 112) may be encoded at a particular bit rate. The electronic device 102 may locally decode the encoded point cloud data 114 to generate a test point cloud 116. The electronic device 102 may be configured to determine a reconstruction quality of the test point cloud 116 (reconstructed by the point cloud codec 110). The reconstruction quality of the test point cloud 116 may depend on the number of bits that may be used to encode the 3D points of the reference point cloud 112 and density distortions that may appear during reconstruction of the reference point cloud 112 (or generation of the test point cloud 116 based on the encoded point cloud data 114). The reconstruction quality of the test point cloud 116 may be improved based on an increase in the encoding bit rate and training of at least one machine learning model of the point cloud codec 110. The training can prevent density distortions from appearing in the test point cloud 116 that can be generated based on the point cloud data. The point cloud data can be encoded by at least one machine learning model in the point cloud codec 110.

[0026]

[0036] The electronic device 102 may be further configured to generate a first local density map of the reference point cloud 112. The first local density map may represent an estimate of the local density value at each 3D point of the reference point cloud 112. The local density value at a 3D point may be estimated based on the volume of a region of the reference point cloud 112 that forms a neighborhood of the 3D point and the number of 3D points in the neighborhood of the 3D point. According to an embodiment, the electronic device 102 may determine the count (or number) of 3D points in the reference point cloud 112 and the dimensions of a structure (e.g., a cubic region) that can contain (or surround) all the 3D points of the reference point cloud 112. The electronic device 102 may generate a radius (R ref ) can further be determined.

[0027]

[0037] The electronic device 102 is then ref ) to find the volume of the sphere (V ref ) can also be determined. The volume of the sphere (V ref Once R ) is determined, the electronic device 102 can determine, for each 3D point in the reference point cloud 112, the number of 3D points in the neighborhood of the corresponding 3D point. The neighborhood of a 3D point in the reference point cloud 112 is defined as a radius R ref The center (i.e., origin) of the sphere can be the location of the 3D point in the reference point cloud 112. The number of 3D points in the neighborhood of each 3D point in the reference point cloud 112 can be calculated by dividing the reference point cloud 112 by the number of 3D points in the neighborhood of R ref and the location of the corresponding 3D point in the reference point cloud 112. The electronic device 102 determines the number of 3D points in the neighborhood of the corresponding 3D point and the volume V refThe local density of each 3D point of the reference point cloud 112 may be further estimated based on the local density values ​​at each 3D point of the reference point cloud 112. The first local density map may be estimated based on the local density values ​​at each 3D point of the reference point cloud 112.

[0028]

[0038] The electronic device 102 may be further configured to determine 3D positions within the test point cloud 116 that correspond to the positions of the 3D points of the reference point cloud 112. For example, if the reference point cloud 112 includes 3D points of a human face, the 3D positions determined around the nose region of the test point cloud 116 may correspond to 3D points within the nose region of the reference point cloud 112.

[0029]

[0039] The electronic device 102 may be further configured to generate a second local density map of the test point cloud 116. The second local density map may represent an estimate of a local density value at each of the determined 3D locations in the test point cloud 116. According to an embodiment, the electronic device 102 may determine a number of 3D points within a neighborhood of each of the determined 3D locations in the test point cloud 116. The neighborhood of the 3D location in the test point cloud 116 may be a neighborhood of a radius R ref The sphere may include one or more points of the test point cloud 116 that can be contained in the sphere of R. The center (i.e., origin) of the sphere may be the 3D location of the test point cloud 116. The number of 3D points within a neighborhood of the 3D location of the test point cloud 116 is determined by the relationship between the test point cloud 116 and R. ref and the coordinates of the 3D position of the test point cloud 116. The coordinates may be the same as the coordinates of the 3D position of the 3D point of the reference point cloud 112 that corresponds to the 3D position of the test point cloud 116.

[0030]

[0040] The electronic device 102 determines the number and V of 3D points within a neighborhood of the corresponding 3D position of the test point cloud 116.ref The electronic device 102 may further estimate a local density at each 3D position of the determined 3D positions in the test point cloud 116 based on the estimated local density values ​​at each 3D position of the test point cloud 116 that correspond to the 3D positions of the 3D points of the reference point cloud 112.

[0031]

[0041] The electronic device 102 may be further configured to calculate a value of a density distortion metric for the test point cloud 116 based on the first and second local density maps. The electronic device 102 may generate a difference density map based on a comparison of the first and second local density maps. The difference density map may be generated based on differences between local density values ​​at 3D points in the reference point cloud 112 and 3D locations in the test point cloud 116 (i.e., locations corresponding to the locations of the 3D points in the reference point cloud 112). According to an embodiment, the electronic device 102 may calculate a mean square error based on the local density values ​​represented by the first and second local density maps. For a 3D point in the reference point cloud 112, a difference may be determined between the local density value at the 3D point and the local density value at a 3D location in the test point cloud 116 (corresponding to the location of the 3D point). Similarly, such differences may be determined for other 3D points in the reference point cloud 112. The mean squared error may be calculated based on the sum of the squared differences and the number of 3D points in the reference point cloud 112. A value of the density distortion metric may be calculated further based on the calculated mean squared error.

[0032]

[0042] According to an embodiment, the electronic device 102 can determine the presence of geometry reconstruction artifacts (such as holes (missing 3D points of the reference point cloud 112 in the test point cloud 116)) or surface irregularities of the test point cloud 116 (such as deformation, erosion, or dilation of the surface of the test point cloud 116 compared to the surface of the reference point cloud 112) based on the value of the density distortion metric. The value of the density distortion metric can be directly proportional to the presence of artifacts or surface irregularities in the test point cloud 116.

[0033]

[0043] Based on the calculated value of the density distortion metric, the electronic device 102 may be further configured to control the display device 104 to render information associated with the reconstruction quality of the test point cloud 116. For example, the electronic device 102 may control a user interface on the display device 104 to render the information. According to an embodiment, the user interface may render the calculated value of the density distortion metric as information associated with the reconstruction quality of the test point cloud 116. This information may also include data indicative of changes in the value of the density distortion metric for different test point clouds (which may have been reconstructed from the encoded point cloud data of the reference point cloud 112) or different rate-distortion points. According to an embodiment, the data may be rendered in tabular or graphical format.

[0034]

[0044] In some embodiments, the information can be rendered using indicators that indicate regions of the test point cloud 116 that contain geometry reconstruction artifacts and surface irregularities. The electronic device 102 can control a user interface to indicate one or more regions of the test point cloud 116 that may contain geometry reconstruction artifacts (i.e., holes) or surface irregularities (i.e., deformations, erosions, or dilations).

[0035]

[0045] In some embodiments, the presence of holes may be indicated in certain regions of the second density map or the difference density map. The local density in those regions (i.e., the regions representing holes) may be lower than the density in other regions of the second density map. Some of the determined 3D locations in the test point cloud 116 may fall within such regions of the second density map. The local density of some of the 3D locations in the test point cloud 116 may be lower than a density threshold (as shown in FIG. 5 using dark spots on the soldier's body). Meanwhile, the local density difference value in the certain region (i.e., the region representing holes) may be higher than the values ​​in other regions of the difference density map.

[0036]

[0046] FIG. 2 is a block diagram illustrating an exemplary electronic device for estimating a density distortion metric for processing point cloud geometry, according to an embodiment of the present disclosure. The description of FIG. 2 is provided with reference to the elements of FIG. 1. Referring to FIG. 2, a block diagram 200 of an electronic device 102 is shown. The electronic device 102 may include a circuit 202, a memory 204, an input / output (I / O) device 206, and a network interface 208. In at least one embodiment, the memory 204 may store a point cloud codec 110. The point cloud codec 110 may include a machine learning-based encoder 210A and a machine learning-based decoder 210B. In at least one embodiment, the I / O device 206 may also include a display device 212. The circuit 202 may be communicatively coupled to the memory 204, the I / O device 206, and the network interface 208 via wired or wireless communication of the electronic device 102.

[0037]

[0047] The circuitry 202 may include suitable logic, circuits, and interfaces that may be configured to execute program instructions associated with a set of operations to be performed by the electronic device 102 . The set of operations may include, but is not limited to, obtaining a reference point cloud 112, encoding the reference point cloud 112 to generate encoded point cloud data 114, decoding the encoded point cloud data 114 to generate a test point cloud 116, generating a first local density map (of the reference point cloud 112) representing local density values ​​at each 3D point of the reference point cloud 112, determining 3D positions in the test point cloud 116 corresponding to the positions of the 3D points of the reference point cloud 112, generating a second local density map of the test point cloud 116 representing local density values ​​at each 3D position of the determined 3D positions in the test point cloud 116, calculating a value of a density distortion metric based on a difference between the first local density map and the second local density map, and controlling the display device 104 to render information associated with the reconstruction quality of the test point cloud 116 based on the calculated value. Circuit 202 may include one or more special-purpose processing units, which may be implemented as an integrated processor or as a cluster of processors that collectively perform the functions of the one or more special-purpose processing units. Circuit 202 may be implemented based on several processor technologies known in the art. Example implementations of circuit 202 may include an x86-based processor, a graphics processing unit (GPU), a reduced instruction set computing (RISC) processor, an application-specific integrated circuit (ASIC) processor, a complex instruction set computing (CISC) processor, a microcontroller, a central processing unit (CPU), and / or other computing circuitry.

[0038]

[0048] The memory 204 may include suitable logic, circuitry, and / or interfaces that may be configured to store instructions executable by the circuitry 202. The memory 204 may be configured to store an operating system and associated applications. The memory 204 may be further configured to store the acquired reference point cloud 112 of the object, the encoded point cloud data 114, and the test point cloud 116. The memory 204 may be further configured to store the first local density map, the second local density map, the difference density map, values ​​of the density distortion metric, and information associated with the reconstruction quality of the test point cloud 116. The point cloud codec 110 included in the memory 204 may be implemented as a combination of programmable instructions stored in the memory 204 and logic units (or programmable logic units) on the hardware circuitry of the electronic device 102. Examples of implementations of memory 204 include, but are not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), hard disk drive (HDD), solid state drive (SSD), CPU cache, and / or secure digital (SD) card.

[0039]

[0049] The machine learning-based encoder 210A and the machine learning-based decoder 210B (included in the point cloud codec 110) can each be a system of computational networks or artificial neurons arranged as nodes in multiple layers. The multiple layers can include an input layer, one or more hidden layers, and an output layer. Each of the multiple layers can include one or more nodes (or artificial neurons, represented, for example, by circles). The output of every node in the input layer can be connected to at least one node in the hidden layer(s). Similarly, the input of each hidden layer can be connected to the output of at least one node in the other layers. The output of each hidden layer can be connected to the input of at least one node in the other layers. The node(s) in the final layer can receive inputs from at least one hidden layer and output a result. The number of layers and the number of nodes in each layer can be determined from hyperparameters of the machine learning-based encoder 210A and the machine learning-based decoder 210B, respectively. Such hyperparameters can be set before, during, or after training the machine learning-based encoder 210A and the machine learning-based decoder 210B, respectively, with a training dataset.

[0040]

[0050] Each node may correspond to a mathematical function (e.g., a sigmoid function or a rectified linear unit) including a set of parameters that can be adjusted during training. The set of parameters may include, for example, weight parameters, regularization parameters, etc. Each node may use the mathematical function to calculate an output based on one or more inputs from nodes in other layer(s) (e.g., previous layer(s)) of the corresponding machine learning model. All or some of the nodes may correspond to the same or different mathematical functions. During each training of the machine learning-based encoder 210A or the machine learning-based decoder 210B, one or more parameters of each node may be updated based on whether the output of the final layer for a given input (from the training dataset) matches the correct result based on a loss function of the machine learning-based encoder 210A or the machine learning-based decoder 210B. The above process may be repeated for the same or different inputs until a minimum value of the loss function is achieved and the training error is minimized. Several training methods, such as gradient descent, stochastic gradient descent, batch gradient descent, gradient boosting, metaheuristics, etc., are known in the art.

[0041]

[0051] Each of the machine learning-based encoder 210A and the machine learning-based decoder 210B may include electronic data, which may be implemented, for example, as a software component of an application executable on an electronic device (e.g., electronic device 102). Each of the machine learning-based encoder 210A and the machine learning-based decoder 210B may rely on libraries, external scripts, or other logic / instructions executed by a processing device, such as circuit 202. Each of the machine learning-based encoder 210A and the machine learning-based decoder 210B may include code and routines configured to enable a computing device, such as circuit 202, to perform one or more operations to encode or decode 3D blocks associated with the reference point cloud 112. Additionally or alternatively, each of the machine learning-based encoder 210A and the machine learning-based decoder 210B may be implemented using hardware, including a processor, a microprocessor (e.g., performing or controlling the execution of one or more operations), an FPGA, or an ASIC. Alternatively, in some embodiments, the neural network model may be implemented using a combination of hardware and software.

[0042]

[0052] The machine learning-based encoder 210A may receive the reference point cloud 112 for compression. According to an embodiment, the received reference point cloud 112 may be divided into a set of 3D blocks. The machine learning-based encoder 210A may include encoder circuitry and / or software for encoding each 3D block of the set of 3D blocks based on applying the machine learning-based encoder 210A to each 3D block to determine a set of encoded blocks. The set of encoded blocks may correspond to encoded point cloud data 114. Encoding the set of 3D blocks may convert the reference point cloud 112 into a bitstream (i.e., an encoded bitstream) of compressed point cloud data (i.e., encoded point cloud data 114).

[0043]

[0053] The machine learning-based decoder 210B may receive the encoded bitstream (i.e., the encoded point cloud data 114). Similarly, the machine learning-based decoder 210B may include decoder circuitry and / or software for decoding each encoded block of the set of encoded blocks to determine a set of decoded blocks based on applying the machine learning-based decoder 210B to each encoded block. The machine learning-based decoder 210B may binarize and merge the set of decoded blocks to obtain a reconstructed point cloud (i.e., the test point cloud 116). The machine learning-based decoder 210B may include information associated with the division of the reference point cloud 112 in a signaling bitstream (received with the encoded point cloud data 114 and for binarizing and merging the set of decoded blocks).

[0044]

[0054] The I / O device 206 may include suitable logic, circuitry, interfaces, and / or code that may be configured to receive user input. For example, the user input may include the number of bits to be used to encode each 3D point of the reference point cloud 112. The I / O device 206 may be further configured to provide output based on the user input. For example, the output may include a calculated value of a density distortion metric. The I / O device 206 may include various input and output devices that may be configured to communicate with the circuit 202. Examples of input devices may include, but are not limited to, a touchscreen, a keyboard, a mouse, a joystick, and / or a microphone. Examples of output devices may include, but are not limited to, the display device 212. In some embodiments, the display device 212 may correspond to the display device 104. The functionality of the display device 212 (i.e., the display device 104) may be incorporated, in whole or at least in part, into the electronic device 102.

[0045]

[0055] The display device 212 may include suitable logic, circuitry, interfaces, and / or code that may be configured to render the reference point cloud 112 and the test point cloud 116 on a display screen of the display device 212. The display device 212 may be further configured to render information associated with the reconstruction quality of the test point cloud 116 based on the calculated value of the density distortion metric upon receiving a control signal from the circuit 202. The display device 212 may further render a first local density map, a second local density map, and a difference density map. According to an embodiment, the display device 212 may include a touch screen for receiving user input. The display device 212 may be implemented by several known technologies, such as, but not limited to, a liquid crystal display (LCD) display, a light emitting diode (LED) display, a plasma display, and / or organic LED (OLED) display technology, and / or other display technologies. According to an embodiment, the display device 212 may refer to a display screen of a smart glasses device, a 3D display, a see-through display, a projection-based display, an electrochromic display, and / or a transparent display.

[0046]

[0056] The network interface 208 may include suitable logic, circuitry, interfaces, and / or code that may be configured to establish communications between the electronic device 102, the display device 104, and the server 106 over the communications network 108. The network interface 208 may be implemented using various known technologies to support wired or wireless communications between the electronic device 102 and the communications network 108. The network interface 208 may include, but is not limited to, an antenna, a radio frequency (RF) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a coder-decoder (CODEC) chipset, a subscriber identity module (SIM) card, and / or a local buffer.

[0047]

[0057] The network interface 208 can communicate via wireless communication with networks such as the Internet, an intranet, and / or a wireless network such as a cellular telephone network, a wireless local area network (LAN), and / or a metropolitan area network (MAN). The wireless communications may use any of a number of communications standards, protocols, and technologies, such as Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Long Term Evolution (LTE), Fifth Generation (5G) New Radio (NR), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Light Fidelity (Li-Fi), Wi-MAX, protocols for email, instant messaging, and / or short message service (SMS), etc.

[0048]

[0058] The functions or operations performed by electronic device 102, such as those described in Figure 1, may be performed by circuitry 202. The operations performed by circuitry 202 are described in detail in, for example, Figures 3, 4, 5, 6, 7A, and 7B.

[0049]

[0059] FIG. 3 illustrates an exemplary processing pipeline for estimating local density at 3D points of a reference point cloud and associated 3D locations in a test point cloud, according to an embodiment of the present disclosure. The description of FIG. 3 is provided with reference to elements of FIGS. 1 and 2. Referring to FIG. 3, an exemplary processing pipeline 300 for estimating local density at each 3D point of a reference point cloud 302 and a 3D location in a test point cloud 304 is shown. The processing pipeline 300 illustrates a sequence of operations that may begin at 306 and end at 320. The sequence of operations may be performed by the circuitry 202 of the electronic device 102. The reference point cloud 302 may be an exemplary point cloud that may be the same as or similar to the reference point cloud 112 of FIG. 1. Similarly, the test point cloud 304 may be an exemplary point cloud that may be the same as or similar to the test point cloud 116 of FIG. 1. The reference point cloud 302 may be an uncompressed point cloud of an object. The circuit 202 can generate the test point cloud 304 based on the encoded point cloud data (which can be obtained after encoding the reference point cloud 302).

[0050]

[0060] In 306, the reference point cloud 302 (PC ref) may be determined. In at least one embodiment, the circuit 202 may be configured to determine a bounding box for the reference point cloud 302. The bounding box may include all 3D points of the reference point cloud 302. By way of example and not limitation, the first bounding box may correspond to a cubic region. The circuit 202 may be configured to determine dimensions (i.e., length, width, and height) of the cubic region. For example, length "a", width "b", and height "c" may be the dimensions of the bounding box. The dimensions of the cubic region may be such that the cubic region includes all 3D points of the reference point cloud 302.

[0051]

[0061] In 308, the reference point cloud 302 (PC ref In at least one embodiment, the circuit 202 can be configured to determine the number of 3D points in the reference point cloud 302.

[0052]

[0062] At 310, a radius may be calculated. In at least one embodiment, the circuit 202 may be configured to calculate a radius to be used to sample the 3D points of the reference point cloud 302. The radius may be calculated based on a bounding box and the number of 3D points in the reference point cloud 302. The bounding box may include multiple surfaces of equal dimensions, which may be determined based on the dimensions of the bounding box. Each surface "S" may have a radius of 0. R " can be determined as the square of the cube root of the volume of the bounding box. For example, the volume of the bounding box can be determined as "a x b x c" (where the length, width, and height (i.e., dimensions) of the bounding box are "a", "b", and "c", respectively). Therefore, "S R " can be determined using equation (1) as follows: TIFF2025528682000002.tif18157

[0053]

[0063] Once multiple surfaces are determined, the circuit 202 can calculate a radius based on the number of 3D points in the reference point cloud 302 and the number of 3D points in each surface of the reference point cloud 302. The radius is calculated based on the number of 3D points on each surface of the reference point cloud 302 ("N R ") and surface fraction ("S R ") and the number of 3D points in the reference point cloud 302 ("N points-reference The radius can be formulated using equation (2) as follows: TIFF2025528682000003.tif23157

[0054]

[0064] The sphere volume can be calculated at 312. In at least one embodiment, the circuit 202 can be configured to calculate the sphere volume based on the radius.

[0055]

[0065] At 314, a first number of 3D points in a neighborhood of each 3D point of the reference point cloud 302 can be determined. In at least one embodiment, the circuit 202 can be configured to determine, from the 3D points of the reference point cloud 302, a first number of 3D points in a neighborhood of each 3D point of the reference point cloud 302. The neighborhood of each 3D point of the reference point cloud 302 can include a number of points that can be included in a sphere of the calculated radius and the calculated volume. The first number of 3D points in the neighborhood of each 3D point of the reference point cloud 302 can be determined by a function of the number of 3D points in the neighborhood of the corresponding 3D point (p ref ) (i.e., coordinates x, y, z), the reference point cloud 302, and the calculated radius (calculated in 310). The center (or origin) of the sphere can be determined based on the position of the corresponding 3D point in the reference point cloud 302 (i.e., p ref (x,y,z)).

[0056]

[0066] At 316, a first local density value can be estimated at each 3D point of the reference point cloud 302. In at least one embodiment, the circuit 202 calculates a sphere volume and a corresponding 3D point (e.g., p ref ) and the number of first 3D points within a neighborhood of the 3D point (p ref ) is the first local density value at the 3D point (p ref ) to the sphere volume. The circuit 202 may be further configured to generate a first local density map of the reference point cloud 302 based on the first local density value at each 3D point of the reference point cloud 302.

[0057]

[0067] At 318, a second number of 3D points within a neighborhood of each 3D location of the set of 3D locations within the test point cloud 304 can be determined. In at least one embodiment, the circuit 202 can be configured to determine, from the 3D points of the test point cloud 304, a second number of 3D points within a neighborhood of each 3D location of the set of 3D locations within the test point cloud 304. The circuit 202 can determine each 3D location of the set of 3D locations that can correspond to a 3D location of a 3D point of the reference point cloud 302. The neighborhood of each 3D location within the test point cloud 304 can include multiple 3D points of the test point cloud 304. The multiple 3D points of the test point cloud 304 can be arranged within a sphere of the calculated radius (calculated at 310) and the calculated sphere volume (calculated at 312). The second number of 3D points within a neighborhood of a 3D location (of the set of 3D locations) within the test point cloud 304 can be calculated based on the number of 3D points (p ref ) and the coordinates of the corresponding 3D position (i.e., p refThe center (or origin) of the sphere can be determined based on the coordinates of the 3D point (p (x, y, z)), the test point cloud 304, and the calculated radius. The center (or origin) of the sphere can be the coordinates of a 3D position within the test point cloud 304. The coordinates of the 3D position within the test point cloud 304 can be determined based on the coordinates of the 3D point (p ref ) and the coordinates of the corresponding 3D position (i.e., p ref (x,y,z)).

[0058]

[0068] At 320, a second local density value at each 3D location within the test point cloud 304 may be estimated. In at least one embodiment, the circuit 202 may be configured to estimate the second local density value at each 3D location of the set of 3D locations within the test point cloud 304. The estimation may be based on a sphere volume and the number of second 3D points within a neighborhood of a corresponding 3D location of the set of 3D locations within the test point cloud 304. The second local density value at a 3D location within the test point cloud 304 may be estimated as a ratio of the number of second 3D points within a neighborhood of the 3D location to the sphere volume. The circuit 202 may be further configured to generate a second local density map of the test point cloud 304 based on the second local density value at each 3D location of the set of 3D locations within the test point cloud 304.

[0059]

[0069] FIG. 4 is a diagram illustrating a comparison of a region of a reference point cloud with a corresponding region of a test point cloud, according to an embodiment of the present disclosure. The description of FIG. 4 is provided in conjunction with elements of FIGS. 1, 2, and 3. Referring to FIG. 4, an exemplary scenario 400 is shown. The exemplary scenario 400 illustrates a reference point cloud 402 and a test point cloud 404. Additionally, 3D points within a region 406 of the reference point cloud 402 and 3D points within a region 408 of the test point cloud 404 are illustrated. The reference point cloud 402 can be an exemplary point cloud that can be the same as or similar to the reference point cloud 112 of FIG. 1. Similarly, the test point cloud 404 can be an exemplary point cloud that can be the same as or similar to the test point cloud 116 of FIG. 1. The reference point cloud 402 can be an uncompressed point cloud of an object, such as a human head. The circuit 202 can reconstruct the test point cloud 404 based on encoded point cloud data, which can be obtained after encoding the reference point cloud 402.

[0060]

[0070] According to an embodiment, the circuit 202 can be configured to calculate a radius of the sphere 410 based on the number of the first 3D point in the reference point cloud 302 and a bounding box that encloses all of the 3D points of the reference point cloud 402. The circuit 202 can also be configured to calculate a volume of the sphere 410 based on the radius of the sphere 410. For example, the center of the sphere 410 can correspond to the 3D position of the 3D point 412. The neighborhood of the 3D point 412 can correspond to the area enclosed by the sphere 410. The circuit 202 can be configured to determine the number of the first 3D points in the neighborhood of the 3D point based on the position (i.e., coordinates) of the 3D point 412, the reference point cloud 402, and the radius of the sphere 410. For example, the number of 3D points in the neighborhood of the 3D point 412 can be determined to be five.

[0061]

[0071] The circuitry 202 may be further configured to estimate a local density value at the 3D point 412 based on a volume of the sphere 410 and a number of the first 3D points in a neighborhood of the 3D point 412. Similarly, local density values ​​at other 3D points in the region 406 and 3D points in other regions of the reference point cloud 402 may be estimated based on the volume of the sphere 410 and a number of the first 3D points in a neighborhood of the other 3D points. The circuitry 202 may be further configured to generate a first local density map of the reference point cloud 402. Each location on the first local density map may represent an estimated local density value at a 3D point of the reference point cloud 402.

[0062]

[0072] According to one embodiment, the circuit 202 can be configured to determine a 3D position within the region 408 that corresponds to the 3D position of a 3D point within the region 406. For example, the circuit 202 can determine a 3D position 414 within the region 408. Similarly, the circuit 202 can determine other 3D positions within the region 408 that correspond to the 3D positions of the 3D points within the region 406. The determined 3D position 414 can correspond to the position of the 3D point 412. The coordinates of the 3D position 414 can be identical to the coordinates of the 3D point 412 (if the origins of the reference point cloud 402 and the test point cloud 404 are the same). The coordinates of the 3D position 414 can correspond to the center of a sphere 416 having a volume that can be identical to the volume of the sphere 410. The circuit 202 can determine the number of second 3D points within a neighborhood of the 3D position 414 within the test point cloud 404. The neighborhood of the 3D position 414 can correspond to the area enclosed by the sphere 416. The number of second 3D points within the neighborhood of the 3D location 414 may be determined based on the coordinates of the 3D location 414 (i.e., the coordinates of the 3D point 412), the test point cloud 404, and the radius of the sphere 416 (i.e., the radius of the sphere 410). For example, the number of second 3D points within the neighborhood of the 3D location 414 may be determined to be 1.

[0063]

[0073] At a 3D location 414 within the test point cloud 404, the circuit 202 can be configured to estimate a local density value. The circuit 202 can estimate the local density value at the 3D location 414 based on the volume of the sphere 416 and the number of second 3D points within a neighborhood of the 3D location 414 within the test point cloud 404. Similarly, the circuit 202 can estimate local density values ​​at each determined 3D location within the region 408 and other determined 3D locations within the test point cloud 404. The estimation can be based on the volume of the sphere 416 and the number of second 3D points within a neighborhood of the other determined 3D locations within the test point cloud 404. The circuit 202 can further be configured to generate a second local density map of the test point cloud 404. Each location on the second local density map can represent an estimated local density value at the determined 3D location within the test point cloud 404.

[0064]

[0074] The local density values ​​at the 3D points in region 406 may be higher than the local density values ​​at the determined 3D locations in region 408. This may be because the number of 3D points in the neighborhood of each 3D point in region 406 is reduced compared to the number of 3D points in the neighborhood of each determined 3D location in region 408. The circuit 202 may compare the local density values ​​at the 3D points in region 406 with the local density values ​​at the determined 3D locations in region 408. Based on the comparison, the circuit 202 may determine the occurrence of geometry reconstruction artifacts in the test point cloud 404 during the generation of the test point cloud 404 (or the reconstruction of the reference point cloud 402).

[0065]

[0075] FIG. 5 illustrates an exemplary scenario for generating a difference density map based on local density maps associated with a reference point cloud and a test point cloud, according to an embodiment of the present disclosure. FIG. 5 is described with reference to elements of FIGS. 1, 2, 3, and 4. Referring to FIG. 5, a first local density map 502, a second local density map 504, and a difference density map 506 that can be generated based on the first local density map 502 and the second local density map 504 are illustrated. The first local density map 502 can represent local density values ​​at 3D points in the reference point cloud 302 of FIG. 3. The second local density map 504 can represent local density values ​​at 3D positions in the test point cloud 304 that correspond to the 3D positions of the 3D points in the reference point cloud 302 of FIG. 3.

[0066]

[0076] According to an embodiment, the circuit 202 can be configured to quantize each of the first local density map 502 and the second local density map 504. Each of the local density values ​​(at the 3D points of the reference point cloud 302) represented by the first local density map 502 can be quantized to a defined number of quantization levels. Similarly, each of the local density values ​​(at the determined 3D positions corresponding to the 3D positions of the 3D points) represented by the second local density map 504 can be quantized to a defined number of quantization levels. For example, the defined number of quantization levels can be "2 16 -1, and the quantization step size Δ x " is "max x -min x Therefore, if the local density value at a 3D point in the reference point cloud 302 or at a 3D position in the test point cloud 304 (corresponding to the 3D position of the 3D point) is "x", then the quantized value "x q " can be expressed using equation (3) as follows: TIFF2025528682000004.tif21157

[0067]

[0077] The circuit 202 can be configured to generate a difference density map 506 based on the difference (obtained through use of an accumulator 508) between the quantized local density values ​​represented by the first local density map 502 and the quantized local density values ​​that can be represented by the second local density map 504. Each location of the difference density map 506 can represent a local density difference value associated with a 3D point of the reference point cloud 302. For a 3D point, the associated local difference density value can be determined based on the difference between the quantized local density value at the 3D point and the quantized local density value at a determined 3D location of the test point cloud 304. The quantized local density value at the 3D point can be obtained from the first local density map 502, and the quantized local density value at the determined 3D location can be obtained from the second local density map 504.

[0068]

[0078] FIG. 6 illustrates an exemplary scenario for calculating a density distortion metric based on local density maps of a reference point cloud and a test point cloud, according to an embodiment of the present disclosure. The description of FIG. 6 is provided in conjunction with elements of FIGS. 1, 2, 3, 4, and 5. Referring to FIG. 6, an exemplary scenario 600 is illustrated. The exemplary scenario 600 illustrates 3D points of a reference point cloud 602 and a test point cloud 604. The circuit 202 can select the reference point cloud 602 as reference data and the test point cloud 604 as test data. Additionally, 3D positions of the 3D points of the reference point cloud 602 and 3D positions in the test point cloud 604 are illustrated, which may correspond to the 3D positions of the 3D points of the reference point cloud 602. The reference point cloud 602 can be an uncompressed point cloud. The test point cloud 604 can be reconstructed based on encoded point cloud data, which can be obtained after encoding the reference point cloud 602.

[0069]

[0079] The reference point cloud 602 can include 3D points 602A-602G, and the test point cloud 604 can include 3D points 604A-604D. During generation of the test point cloud 604, one or more of the 3D points 602A-602G may be lost or moved. Once the reference point cloud 602 and the test point cloud 604 are generated, the circuit 202 can be configured to generate a first local density map of the reference point cloud 602 and a second local density map of the test point cloud 604.

[0070]

[0080] The first local density map of the reference point cloud 602 may represent a local density value at each of the 3D points 602A-602G. To determine the local density values, the circuit 202 may further determine coordinates (e.g., "x," "y," and "z" coordinate values) of 3D locations 606A-606G at which the 3D points 602A-602G of the reference point cloud 602 may reside. For example, the 3D point 602A may reside at location 606A. Similarly, the 3D point 602G may reside at location 606G of the reference point cloud 602. Each 3D location of the 3D locations 606A-606G may be represented by a "ref(x i ,y i ,z i )" where: The file is TIFF2025528682000005.tif7150. Once the coordinates of the 3D locations 606A-606G are determined, local density values ​​at the 3D locations 606A-606G can be determined. A first local density map of the reference point cloud 602 can represent the local density values ​​determined at each 3D location 606A-606G (where a 3D point of the 3D points 602A-602G resides). Thus, the first local density map of the reference point cloud 602 can be TIFF2025528682000006.tif11157. In the above formula, "LD_ref(x i ,y i ,z i ) may represent the local density value at 3D position “i” of the reference point cloud 602.

[0071]

[0081] The circuit 202 may be further configured to determine 3D positions within the test point cloud 604 that may correspond to the 3D positions 606A-606G of the 3D points 602A-602G. The determined 3D positions within the test point cloud 604 may be 3D positions 608A-608G. Each of the determined 3D positions 608A-608G within the test point cloud 604 may correspond to one of the 3D positions 606A-606G of the 3D points 602A-602G (when the origins of the reference point cloud 602 and the test point cloud 604 are the same). Thus, the coordinates of the 3D positions 608A-608G may be the same as the determined coordinates of the 3D positions 606A-606G. For example, the coordinates of the 3D position 608A may be the same as the coordinates of the position 606A. Similarly, the coordinates of 3D position 608G can be the same as the coordinates of position 606G. Each of the 3D positions 608A-608G of the test point cloud 604 can be expressed as "tes(x i ,y i ,z i )" where: The file is TIFF2025528682000007.tif11157.

[0072]

[0082] Once the coordinates of the locations 608A-608G are determined, local density values ​​at the 3D locations 608A-608G can be determined. A second local density map of the test point cloud 604 can represent the local density values ​​determined at each 3D location 608A-608G (corresponding to the 3D locations 606A-606G at which the 3D points 602A-602G reside). Thus, the second local density map of the test point cloud 604 can represent: It can be represented as TIFF2025528682000008.tif11157. i ,y i ,z i ) may represent the local density value at 3D position “i” within the test point cloud 604.

[0073]

[0083] The circuit 202 may be further configured to calculate a local density difference value associated with each of the 3D points 602A-602G of the reference point cloud 602. For example, the local density difference value associated with the 3D point 602A may be calculated. The calculation may be performed using the local density value at the 3D location 606A (i.e., “LD_ref(x 606A ,y 606A ,z 606A )") and the local density value at 3D location 608A (i.e., "LD_tes(x 608A ,y 608A ,z 608A )"). Similarly, local density difference values ​​associated with other 3D points 602B-602G can be calculated. Thus, the circuit 202 can generate a first difference density map that can represent the local density difference values ​​associated with the 3D points 602A-602G.

[0074]

[0084] The circuit 202 can obtain a local density difference value associated with each of the 3D points 602A-602G from the difference density map. For example, the local density difference value (i.e., LD_ref(x 606A ,y 606A ,z 606A )-LD_tes(x 608A ,y 608A ,z 608A ) can be obtained. Similarly, local density difference values ​​for the other 3D points 602B-602G can be obtained. The circuit 202 can then calculate the sum of the squares of the local density difference values ​​associated with the 3D points 602A-602G.

[0075]

[0085] According to an embodiment, the circuit 202 may be further configured to calculate a first mean square error (MSE) based on the first local density map and the second local density map. The first MSE may be determined based on a sum of squares of the local density difference values. The first MSE may be expressed using equation (4) as follows: TIFF2025528682000009.tif23157Equation (4) is "N A It can be seen that this can be generalized to a reference point cloud that can contain " 3D points. The generalized first MSE can be expressed using equation (5) as follows: TIFF2025528682000010.tif23157

[0076]

[0086] According to one embodiment, the density distortion metric (“D3”) is a function of the mean squared error (“MSE”) and a set number of quantization levels (i.e., “2 16 The density distortion metric can be calculated based on the density distortion metric (D(k)) and the density distortion metric (D(k)) using equation (6) as follows: TIFF2025528682000011.tif26157

[0077]

[0087] A calculated value of the density distortion metric can be obtained for a selected rate-distortion point or the number of bits used to encode each 3D point of the reference point cloud 602. Based on changes in the rate-distortion point or the number of bits, the value of the density distortion metric can change.

[0078]

[0088] According to an embodiment, the density distortion metric may indicate a peak signal-to-noise ratio (PSNR) associated with a test point cloud (e.g., test point cloud 604). Equation (6) may be modified to equation (7) as follows: TIFF2025528682000012.tif26157

[0079]

[0089] 7A and 7B are exemplary graphs illustrating the variation of density distortion metric values ​​with respect to the number of bits used to encode 3D points of a reference point cloud, according to an embodiment of the present disclosure. The description of Figures 7A and 7B is provided with reference to elements of Figures 1, 2, 3, 4, 5, and 6. Referring to Figures 7A and 7B, exemplary graphs 700A and 700B are shown. The exemplary graphs 700A and 700B illustrate the variation of calculated values ​​of the density distortion metric (D3-PSNR) with respect to the number of bits per point (BPP) used to encode 3D points of a reference point cloud.

[0080]

[0090] The circuit 202 can calculate a value of the density distortion metric for each test point cloud, which can be generated by the first machine learning-based encoder ( FIG. 7A ) or the second machine learning-based encoder ( FIG. 7B ), based on encoded point cloud data obtained by encoding each 3D point of the reference point cloud within the BPP range. The first machine learning-based encoder or the second machine learning-based encoder can generate the test point cloud based on determining the spatial occupancy of voxels from the encoded point cloud data of the reference point cloud.

[0081]

[0091] The density distortion metric value may indicate the reconstruction quality of the test point cloud. The reconstruction quality of the test point cloud may be a calculated value of the density distortion metric value, which may be determined based on the local density maps of the reference point cloud and the test point cloud. The reconstruction quality may increase monotonically with increasing BPP used to encode each 3D point of the reference point cloud. However, in some scenarios, the reconstruction quality of a test point cloud generated based on point cloud data encoded with a particular BPP may be lower than the reconstruction quality of another test point cloud generated based on point cloud data encoded with a lower BPP.

[0082]

[0092] For example, as shown in region 702 of exemplary graph 700A, the reconstruction quality of a first test point cloud may be lower than the reconstruction quality of a second test point cloud. The first test point cloud may be generated based on point cloud data encoded at 0.3 BPP, while the second test point cloud may be generated based on point cloud data encoded at 0.25 BPP. Similarly, as shown in region 704 of exemplary graph 700B, the reconstruction quality of a third test point cloud may be lower than the reconstruction quality of a fourth test point cloud. The third test point cloud may be generated based on point cloud data encoded at 0.061 BPP, while the fourth test point cloud may be generated based on point cloud data encoded at 0.026 BPP. The reduction in the reconstruction quality of the test point clouds may occur due to the presence of density distortions, such as geometry reconstruction artifacts (e.g., holes) or surface irregularities (e.g., deformations, erosions, or dilations), in the test point clouds. The presence of density distortions may result from errors in the estimation of local occupancies (i.e., occupancies of 3D points within a voxel).

[0083]

[0093] The degradation of reconstruction quality due to such errors can be observed using the density distortion metric values ​​(without using point-to-point or point-to-plane metrics). This is because the appearance of geometric reconstruction artifacts or surface irregularities can lead to larger changes in the density map of the test point cloud compared to the changes in the density map of the reference point cloud. Such changes can lead to a degradation of the density distortion metric values ​​(observed in regions 702 and 704) and the reconstruction quality of the test point cloud.

[0084]

[0094] According to an embodiment, the circuit 202 can be configured to control the display device 104 (or the display device 212) to render information associated with the reconstruction quality of the test point cloud based on the calculated value of the density distortion metric. This information can correspond to the exemplary graphs 700A and 700B. Based on the rendered information, the circuit 202 can be configured to receive an input that enables selection of a rate-distortion (RD) point from a plurality of RD points as the optimal rate that can be used to encode the reference point cloud. The RD point can correspond to a BPP (e.g., 0.25 BPP (as shown in FIG. 7A) or 0.026 BPP (as shown in FIG. 7B)) to be used to encode each 3D point of the reference point cloud. The selection can be performed based on a determination that the calculated value of the density distortion metric exceeds a threshold.

[0085]

[0095] According to an embodiment, the circuit 202 can be configured to train the machine learning-based encoder 210A on a point cloud encoding task based on a first auxiliary loss using the calculated value of the density distortion metric. For example, as shown in FIG. 7A, the value of the density distortion metric can be used to train the machine learning-based encoder 210A on a point cloud encoding task based on the first auxiliary loss. The value can be calculated for a test point cloud, which can be generated based on point cloud data encoded at 0.25 BPP.

[0086]

[0096] According to an embodiment, the circuit 202 can be configured to train the machine learning-based decoder 210B on a point cloud reconstruction task based on a second auxiliary loss using the calculated value of the density distortion metric. For example, as shown in FIG. 7B, the value of the density distortion metric can be used to train the machine learning-based decoder 210B on a point cloud reconstruction task based on the second auxiliary loss. The value can be calculated for a test point cloud that can be generated based on point cloud data encoded at 0.061 BPP.

[0087]

[0097] FIG. 8 illustrates an exemplary processing pipeline for estimating local density at 3D points of reference data and associated 3D locations in test data, according to an embodiment of the present disclosure. FIG. 8 is described with reference to elements of FIGS. 1 , 2 , 3 , 4 , 5 , 6 , and 7 . Referring to FIG. 8 , an exemplary processing pipeline 800 for estimating local density at each 3D point of reference data 802 and test data 804 is shown. The processing pipeline 800 illustrates a sequence of operations that may begin at 806 and end at 820. The sequence of operations may be performed by the circuitry 202 of the electronic device 102. During operation, the circuitry 202 may select the test point cloud 304 as the reference data 802 and the reference point cloud 302 as the test data 804.

[0088]

[0098] At 806, a bounding box of the reference data 802 can be determined. In at least one embodiment, the circuit 202 can be configured to determine a bounding box of the reference data 802 (i.e., the test point cloud 304). The bounding box can include all 3D points of the test point cloud 304. The circuit 202 can be configured to determine the dimensions (i.e., length, width, and height) of the bounding box. For example, the length "p", width "q", and height "r" can be the dimensions of the bounding box. The dimensions of the cubic region can be such that the cubic region includes all 3D points of the test point cloud 304.

[0089]

[0099] At 808, the number of 3D points in the reference data can be determined. In at least one embodiment, the circuit 202 can be configured to determine the number of 3D points in the reference data 802 (i.e., the test point cloud 304).

[0090]

[0100] At 810, a radius may be calculated. In at least one embodiment, the circuit 202 may be configured to calculate a radius to be used to sample the 3D points of the reference data 802 (i.e., the test point cloud 304). The radius may be calculated based on a bounding box and the number of 3D points in the test point cloud 304. The bounding box (of the reference data 802) may include multiple surfaces of equal dimensions, which may be determined based on the dimensions of the bounding box. Each surface "S" may have a radius of 0. T " can be determined as the square of the cube root of the volume of the bounding box. For example, the volume of the bounding box can be determined as "p x q x r" (where the length, width, and height (i.e., dimensions) of the bounding box are "p", "q", and "r", respectively). Therefore, "S T " can be determined using equation (8) as follows: TIFF2025528682000013.tif21157

[0091]

[0101] Once the multiple surfaces are determined, the circuit 202 can calculate a radius based on the number of 3D points in the reference data 802 (i.e., the test point cloud 304) and the number of 3D points in each surface of the test point cloud 304. The radius is calculated based on the number of 3D points on each surface of the test point cloud 304 ("N T ") and surface fraction ("S T ”) and the number of 3D points in the test point cloud 304 (“N points-test The radius can be formulated using equation (9) as follows: TIFF2025528682000014.tif23157

[0092]

[0102] The sphere volume can be calculated at 812. In at least one embodiment, the circuit 202 can be configured to calculate the sphere volume based on the radius.

[0093]

[0103] At 814, a third number of 3D points in a neighborhood of each 3D point of the reference data 802 can be determined. In at least one embodiment, the circuit 202 can be configured to determine a third number of 3D points in a neighborhood of each 3D point of the reference data 802 (i.e., the test point cloud 304) from the 3D points of the reference data 802 (i.e., the test point cloud 304). The neighborhood of each 3D point of the reference data 802 (i.e., the test point cloud 304) can include a number of points that can be included in a sphere of the calculated radius and the calculated volume. The third number of 3D points in the neighborhood of each 3D point of the reference data 802 (i.e., the test point cloud 304) can be determined based on the number of 3D points in the neighborhood of the corresponding 3D point (p tes) (i.e., coordinates x, y, z), the reference data 802 (i.e., the test point cloud 304), and the calculated radius (calculated in 810). The center (or origin) of the sphere can be determined based on the position (i.e., coordinates x, y, z) of the corresponding 3D point in the reference data 802 (i.e., the test point cloud 304) and the calculated radius (calculated in 810). tes (x,y,z)).

[0094]

[0104] At 816, a third local density value can be estimated at each 3D point of the reference data 802. In at least one embodiment, the circuit 202 calculates the sphere volume and the corresponding 3D point (e.g., p tes ) and the number of third 3D points within a neighborhood of the 3D point (p tes ) is the third local density value at the 3D point (p tes ) to the sphere volume. The circuit 202 may be further configured to generate a third local density map of the reference data 802 (i.e., the test point cloud 304) based on the third local density value at each 3D point of the reference data 802 (i.e., the test point cloud 304).

[0095]

[0105] At 818, a fourth number of 3D points within a neighborhood of each 3D location of the set of 3D locations in the test data 804 can be determined. In at least one embodiment, the circuit 202 can be configured to determine a fourth number of 3D points within a neighborhood of each 3D location of the set of 3D locations in the test data 804 (i.e., the reference point cloud 302) from the 3D points of the test data 804 (i.e., the reference point cloud 302). The circuit 202 can determine each 3D location of the set of 3D locations that can correspond to the 3D location of a 3D point of the reference data (i.e., the test point cloud 304). The neighborhood of each 3D location in the test data 804 (i.e., the reference point cloud 302) can include multiple 3D points of the test data 804 (i.e., the reference point cloud 302). The multiple 3D points of the test data 804 can be placed within a sphere of the calculated radius (calculated at 810) and the calculated sphere volume (calculated at 812). The number of fourth 3D points in the neighborhood of a 3D location (of the set of 3D locations) in the test data 804 (i.e., the reference point cloud 302) is calculated by multiplying the number of fourth 3D points (p tes ) and the coordinates of the corresponding 3D position (i.e., p tes The center (or origin) of the sphere can be determined based on the coordinates of the 3D point (p (x, y, z)), the test data 804 (i.e., the reference point cloud 302), and the calculated radius. The center (or origin) of the sphere can be the coordinates of a 3D position in the test data 804 (i.e., the reference point cloud 302). The coordinates of the 3D position in the test data 804 (i.e., the reference point cloud 302) can be determined based on the 3D point (p ref ) and the coordinates of the corresponding 3D position (i.e., p tes (x,y,z)).

[0096]

[0106] At 820, a fourth local density value at each 3D location in the test data 804 may be estimated. In at least one embodiment, the circuit 202 may be configured to estimate the fourth local density value at each 3D location of the set of 3D locations in the test data 804 (i.e., the reference point cloud 302). The estimation may be based on a sphere volume and the number of fourth 3D points in a neighborhood of a corresponding 3D location of the set of 3D locations in the test data 804 (i.e., the reference point cloud 302). The fourth local density value at a 3D location in the test data 804 (i.e., the reference point cloud 302) may be estimated as a ratio of the number of fourth 3D points in a neighborhood of the 3D location to the calculated sphere volume. The circuit 202 may be further configured to generate a fourth local density map of the test data 804 (i.e., the reference point cloud 302) based on the fourth local density value at each 3D location of the set of 3D locations in the test data 804 (i.e., the reference point cloud 302).

[0097]

[0107] 9 is a diagram illustrating the selection of regions of a reference point cloud and regions of a test point cloud for generating a local density map, according to an embodiment of the present disclosure. Referring to FIG. 9, an exemplary scenario 900 is shown. The exemplary scenario 900 shows a reference point cloud 902 and a test point cloud 904. The reference point cloud 902 can be an uncompressed point cloud of an object, such as a human head. The circuit 202 can reconstruct the test point cloud 904 based on encoded point cloud data, which can be obtained after encoding the reference point cloud 902.

[0098]

[0108] At T−1, the circuit 202 may select the reference point cloud 902 as the reference data and the test point cloud 904 as the test data. According to an embodiment, the circuit 202 may be configured to calculate a radius of a sphere 906 based on the number of 3D points in the reference point cloud 902 and a bounding box that encloses all of the 3D points in the reference point cloud 902. The circuit 202 may further be configured to calculate a volume of the sphere 906 based on the radius of the sphere 906. For example, the center of the sphere 906 may correspond to the 3D position of the 3D point 908. The circuit 202 may determine a first number of 3D points in a neighborhood of the 3D point 908 based on the position of the 3D point 908, the reference data (i.e., the reference point cloud 902), and the radius of the sphere 906. Then, a first local density value at the 3D point 908 may be determined based on the volume of the sphere 906 and the first number of 3D points in the neighborhood of the 3D point 908. Therefore, a first local density value at each 3D point of the reference data (i.e., the reference point cloud 902) can be determined based on shifting the center of the sphere 906 to the 3D position of the corresponding 3D point and determining the number of first 3D points within a neighborhood of the corresponding 3D point. The circuit 202 can be further configured to generate a first local density map of the reference data (i.e., the reference point cloud 902), which can represent the local density value at each 3D point of the reference data.

[0099]

[0109] The circuit 202 may be further configured to determine a 3D position in the test data (i.e., the test point cloud 904) that corresponds to the 3D position of the 3D point in the reference data (i.e., the reference point cloud 402). For example, a 3D position 910 may be determined that corresponds to the 3D position of the 3D point 908. Similarly, other 3D positions in the test data (i.e., the test point cloud 904) that correspond to the 3D positions of other 3D points in the reference data (i.e., the reference point cloud 902) may be determined. The circuit 202 may determine a number of second 3D points within a neighborhood of the 3D position 910 in the test data (i.e., the test point cloud 904). The neighborhood may correspond to an area enclosed by a sphere 912. The number of second 3D points within the neighborhood may be determined based on the 3D position 910, the test data (i.e., the test point cloud 904), and the radius of the sphere 912 (which may be the same as the radius of the sphere 906). A second local density value at the 3D location 910 can then be determined based on the volume of the sphere 912 and the number of second 3D points within a neighborhood of the 3D location 910. Accordingly, second local density values ​​at other determined 3D locations within the test data (i.e., the test point cloud 904) can be determined. The circuit 202 can be configured to generate a second local density map of the test data (i.e., the test point cloud 904) that can represent the local density values ​​at each 3D location of the test data.

[0100]

[0110] At T-2, the circuit 202 may select the test point cloud 904 as the reference data and the reference point cloud 902 as the test data. According to an embodiment, the circuit 202 may be configured to calculate a radius of a sphere 914 based on the number of 3D points in the test point cloud 904 and a bounding box that encloses all of the 3D points in the test point cloud 904. The circuit 202 may further be configured to calculate a volume of the sphere 914 based on the radius of the sphere 914. For example, the center of the sphere 914 may correspond to a 3D position of a 3D point 916 in the test point cloud 904. The circuit 202 may determine a number of third 3D points in a neighborhood of the 3D point 916 based on the position of the 3D point 916, the reference data (i.e., the test point cloud 904), and the radius of the sphere 914. Then, a third local density value at the 3D point 916 may be determined based on the volume of the sphere 914 and the number of third 3D points in a neighborhood of the 3D point 916. Thus, a third local density value at each 3D point of the reference data (i.e., test point cloud 904) can be determined based on shifting the center of the sphere 914 to the 3D position of the corresponding 3D point and determining the number of third 3D points within a neighborhood of the corresponding 3D point. The circuit 202 can be further configured to generate a third local density map of the reference data (i.e., test point cloud 904), which can represent the local density value at each 3D point of the reference data.

[0101]

[0111] The circuit 202 may be further configured to determine a 3D position in the test data (i.e., the reference point cloud 902) that corresponds to the 3D position of the 3D point in the reference data (i.e., the test point cloud 902). For example, a 3D position 918 may be determined that corresponds to the 3D position of the 3D point 916. Similarly, other 3D positions in the test data (i.e., the reference point cloud 902) that correspond to the 3D positions of other 3D points in the reference data (i.e., the test point cloud 904) may be determined. The circuit 202 may determine a number of fourth 3D points in a neighborhood of the 3D position 918 in the test data (i.e., the reference point cloud 902). The neighborhood may correspond to an area enclosed by a sphere 920. The number of fourth 3D points in the neighborhood may be determined based on the 3D position 918, the test data (i.e., the reference point cloud 902), and the radius of the sphere 920 (which may be the same as the radius of the sphere 914). A fourth local density value at the 3D location 918 can then be determined based on the volume of the sphere 920 and the number of fourth 3D points within the vicinity of the 3D location 918. Accordingly, fourth local density values ​​at other determined 3D locations within the test data (i.e., the reference point cloud 902) can be determined. The circuit 202 can be configured to generate a fourth local density map of the test data (i.e., the reference point cloud 902) that can represent the local density values ​​at each 3D location of the test data.

[0102]

[0112] FIG. 10 illustrates an exemplary scenario for calculating a density distortion metric based on local density maps of reference data and test data, according to an embodiment of the present disclosure. The description of FIG. 10 is provided in conjunction with elements of FIGS. 1, 2, 3, 4, 5, 6, 7, 8, and 9. Referring to FIG. 10, an exemplary scenario 1000 is illustrated. The exemplary scenario 1000 illustrates 3D points of the reference point cloud 602 and the test point cloud 604 of FIG. 6. During operation, the circuit 202 can select the test point cloud 604 as the reference data and the reference point cloud 602 as the test data. Additionally, 3D positions of the 3D points of the test point cloud 604 and 3D positions within the reference point cloud 602 that can correspond to the 3D positions of the 3D points of the test point cloud 604 are illustrated. The circuit 202 can be configured to generate a third local density map of the test point cloud 604 and a fourth local density map of the reference point cloud 602.

[0103]

[0113] The third local density map of the test point cloud 604 may represent a local density value at each of the 3D points 604A-604D. To determine the local density values, the circuit 202 may further determine coordinates (e.g., "x," "y," and "z" coordinate values) of 3D locations 610A-610D at which the 3D points 604A-604D of the test point cloud 604 may reside. Each 3D location 610A-610D may be represented by a "tes(x i ,y i ,z i )" where: TIFF2025528682000015.tif11157. The third local density map of the test point cloud 604 is TIFF2025528682000016.tif11157. In the above formula, "LD_tes(x i ,y i ,z i) may represent the local density value at 3D position “i” in the test point cloud 604.

[0104]

[0114] The circuit 202 may further determine 3D positions within the reference point cloud 602 that may correspond to the 3D positions 610A-610D of the 3D points 604A-604D. The determined 3D positions within the reference point cloud 602 may be 3D positions 612A-612D. The coordinates of the 3D positions 612A-612D may be the same as the determined coordinates of the 3D positions 610A-610D. Each of the determined 3D positions 612A-612D of the reference point cloud 602 may be represented by a 3D coordinate system, such as "ref(x i ,y i ,z i )" where: The file is TIFF2025528682000017.tif11157.

[0105]

[0115] Once the coordinates of the 3D locations 612A-612D are determined, local density values ​​at the 3D locations 612A-612D can be determined. A fourth local density map of the reference point cloud 602 can represent the local density values ​​determined at each 3D location of the 3D locations 612A-612D, It can be represented as TIFF2025528682000018.tif11157. "LD_ref(x i ,y i ,z i ) may represent the local density value at 3D position “i” within the reference point cloud 602.

[0106]

[0116] The circuit 202 can be further configured to calculate a local density difference value associated with each of the 3D points 604A-604D of the test point cloud 604. For example, the local density difference value associated with the 3D point 604A is the local density value at the 3D location 610A (i.e., “LD_tes(x 610A ,y 610A ,z 610A)") and the local density value at 3D position 612A (i.e., "LD_ref(x 612A ,y 612A ,z 612A )”). Similarly, local density difference values ​​associated with 3D points 604B, 604C, and 604D can be calculated. Thus, circuit 202 can generate a second difference density map that can represent the local density difference values ​​associated with 3D points 604A-604D, and can obtain the local density difference value associated with each of 3D points 604A-604D from the difference density map. For example, the local density difference value of 3D point 604A (i.e., LD_tes(x 610A ,y 610A ,z 610A )-LD_ref(x 612A ,y 612A ,z 612A ) can be obtained. Similarly, local density difference values ​​for the other 3D points 604B, 604C, and 604D can be obtained. The circuit 202 can then calculate the sum of the squares of the local density difference values ​​associated with the 3D points 604A-604D.

[0107]

[0117] According to an embodiment, the circuit 202 may be further configured to calculate a second MSE based on the third local density map and the fourth local density map. The second MSE may be determined based on a sum of squares of the local density difference values. By way of example and not limitation, the second MSE may be expressed using equation (10) given as follows: TIFF2025528682000019.tif23157Equation (10) is "N B The generalized second MSE can be expressed using equation (11), given as follows: TIFF2025528682000020.tif23157

[0108]

[0118] According to one embodiment, the circuit 202 calculates the first MSE, the second MSE, and the number of 3D points in the reference point cloud 602 ("N A ”) and the number of 3D points in the test point cloud 604 (“N B ") and calculates a value for the density distortion metric ("D3"). The calculation can include two steps. In the first step, the circuit 202 can determine the total MSE. For example, the total MSE can be determined using one of the following equations (12), (13), (14), and (15). These equations are given as follows: TIFF2025528682000021.tif13150 TIFF2025528682000022.tif15157 TIFF2025528682000023.tif15157 TIFF2025528682000024.tif15157

[0109]

[0119] In a first step, the circuit 202 can determine the density distortion metric (“D3”) using equation (16) as follows: TIFF2025528682000025.tif15157

[0110]

[0120] Figure 11 is a flowchart illustrating operations of an exemplary method for estimating a density distortion metric for processing point cloud geometry, according to an embodiment of the present disclosure. Figure 11 is described with reference to elements of Figures 1, 2, 3, 4, 5, 6, 7A, 7B, 8, 9, and 10. Referring to Figure 11, a flowchart 1100 is shown. Operations 1102 through 1118 can be implemented by any computing system, such as the electronic device 102 of Figure 1, or the circuitry 202 of the electronic device 102. Operations can start at 1102 and proceed to 1104.

[0111]

[0121] At 1104, a reference point cloud 112 for the object can be obtained. In at least one embodiment, the circuit 202 can be configured to obtain the reference point cloud 112 for the object. Details of obtaining the reference point cloud 112 for the object are described, for example, in FIG. 1.

[0112]

[0122] At 1106, the reference point cloud 112 may be encoded to generate encoded point cloud data 114. In at least one embodiment, the circuit 202 may be configured to encode the reference point cloud 112 to generate encoded point cloud data 114. Details of generating the encoded point cloud data 114 are described, for example, in FIG.

[0113]

[0123] At 1108, the encoded point cloud data 114 may be decoded to generate the test point cloud 116. In at least one embodiment, the circuit 202 may be configured to decode the encoded point cloud data 114 to generate the test point cloud 116. Details of generating the test point cloud 116 based on decoding the encoded point cloud data 114 are described, for example, in FIG.

[0114]

[0124] At 1110, a first local density map of the reference point cloud 112 may be generated. In at least one embodiment, the circuit 202 may be configured to generate the first local density map of the reference point cloud 112. The first local density map may represent local density values ​​at each 3D point of the reference point cloud 112. Details of generating the first local density map of the reference point cloud 112 are described, for example, in FIGS. 1, 3, 5, and 6.

[0115]

[0125] At 1112, 3D positions within the test point cloud 116 can be determined that correspond to the positions of the 3D points of the reference point cloud 112. In at least one embodiment, the circuit 202 can be configured to determine 3D positions within the test point cloud 116 that correspond to the positions of the 3D points of the reference point cloud 112. Details of determining 3D positions within the test point cloud 116 are described, for example, in Figures 1, 3, 4, 5, and 6.

[0116]

[0126] At 1114, a second local density map of the test point cloud 116 may be generated. In at least one embodiment, the circuit 202 may be configured to generate the second local density map of the test point cloud 116. The second local density map may represent local density values ​​at each of the determined 3D locations within the test point cloud 116. Details of generating the second local density map of the test point cloud 116 are described, for example, in FIGS. 1, 3, 4, 5, and 6.

[0117]

[0127] At 1116, a value of a density distortion metric can be calculated for the test point cloud 116 based on the first local density map and the second local density map. In at least one embodiment, the circuit 202 can be configured to calculate a value of the density distortion metric for the test point cloud 116 based on the first local density map and the second local density map. Details of calculating the density distortion metric are described, for example, in FIGS. 1, 6, and 7.

[0118]

[0128] At 1118, a display device can be controlled to render information associated with the reconstruction quality of the test point cloud 116 based on the calculated value of the density distortion metric. In at least one embodiment, the circuit 202 can be configured to control the display device to render information associated with the reconstruction quality of the test point cloud 116 based on the calculated value of the density distortion metric. Details of controlling the display device to render the information are described, for example, in Figures 1 and 7. Control can proceed to an end.

[0119]

[0129] Although flowchart 1100 is shown as separate operations such as 1104, 1106, 1108, 1110, 1112, 1114, 1116, and 1118, the disclosure is not limited in this respect. Thus, in particular embodiments, such separate operations may be further divided into additional operations, combined into fewer operations, or eliminated, depending on the implementation, without detracting from the essence of the disclosed embodiments.

[0120]

[0130] Various embodiments of the present disclosure may provide a non-transitory computer-readable medium and / or storage medium having stored thereon computer-executable instructions executable by a machine and / or computer to operate an electronic device (e.g., electronic device 102). The computer-executable instructions may cause the machine and / or computer to perform operations including obtaining a reference point cloud 112 for an object. The operations may further include encoding the reference point cloud 112 to generate encoded point cloud data 114. The operations may further include decoding the encoded point cloud data 114 to generate a test point cloud 116. The operations may further include generating a first local density map of the reference point cloud 112. The first local density map may represent local density values ​​at each 3D point of the reference point cloud 112. The operations may further include determining 3D positions within the test point cloud 116 corresponding to positions of the 3D points of the reference point cloud 112. The operations may further include generating a second local density map of the test point cloud 116. The second local density map may represent a local density value at each of the determined 3D locations within the test point cloud 116. The operations may further include calculating a value of a density distortion metric for the test point cloud 116 based on the first local density map and the second local density map. The operations may further include controlling the display device 104 to render information associated with a reconstruction quality of the test point cloud 116 based on the calculated value of the density distortion metric.

[0121]

[0131] An example embodiment of the present disclosure may include an electronic device (such as the electronic device 102 of FIG. 1 ) that may include a circuit (such as the circuit 202) that may be communicatively coupled to a display device (such as the display device 104 of FIG. 1 ). The electronic device 102 may further include a memory (such as the memory 204 of FIG. 2 ). The memory 204 may be configured to store a point cloud codec (such as the point cloud codec 110) that includes a machine learning-based encoder 210A and a machine learning-based decoder 210B. The circuit 202 may be configured to obtain a reference point cloud 112 for an object. The circuit 202 may be further configured to encode the reference point cloud 112 to generate encoded point cloud data 114. The circuit 202 may be further configured to decode the encoded point cloud data 114 to generate a test point cloud 116. The circuit 202 may be further configured to generate a first local density map representing local density values ​​at each 3D point of the reference point cloud 112. The circuit 202 may be further configured to determine 3D positions within the test point cloud 116 that correspond to the positions of the 3D points of the reference point cloud 112. The circuit 202 may be further configured to generate a second local density map of the test point cloud 116 representing a local density value at each of the determined 3D positions within the test point cloud 116. The circuit 202 may be further configured to calculate a value of a density distortion metric for the test point cloud 116 based on the first local density map and the second local density map. The circuit 202 may be further configured to control the display device 114 to render information associated with the reconstruction quality of the test point cloud 116 based on the calculated value of the density distortion metric.

[0122]

[0132] According to an embodiment, the circuit 202 may be further configured to determine a bounding box of the reference point cloud 112. The circuit 202 may be further configured to determine a number of 3D points of the reference point cloud. The circuit 202 may be further configured to calculate a radius to be used for sampling the 3D points of the reference point cloud 112. A first radius may be calculated based on the bounding box and the number of 3D points of the reference point cloud 112. The circuit 202 may be further configured to calculate a sphere volume based on the radius.

[0123]

[0133] According to an embodiment, the circuitry 202 may be further configured to determine, from the 3D points of the reference point cloud 112, a number of first 3D points in a neighborhood of each 3D point of the reference point cloud 112. The number of first 3D points in a neighborhood of each 3D point of the reference point cloud 112 may be determined based on the radius, the coordinates of the corresponding 3D point of the reference point cloud 112, and the reference point cloud 112. The circuitry 202 may be further configured to determine a local density value at each 3D point of the reference point cloud 112 based on the sphere volume and the number of first 3D points in a neighborhood of the corresponding 3D point.

[0124]

[0134] According to an embodiment, the circuit 202 may be further configured to determine a number of second 3D points within a neighborhood of each 3D location of the determined 3D location from the 3D points of the test point cloud 116. The number of second 3D points within the neighborhood may be determined based on a radius. The circuit 202 may be further configured to determine local density values ​​of the second local density map based on the sphere volume and the number of second 3D points within a neighborhood of a corresponding 3D location of the determined 3D location.

[0125]

[0135] According to an embodiment, the circuit 202 may be further configured to quantize each of the first and second local density maps based on a determined number of quantization levels, and the value of the density distortion metric may be calculated further based on the quantization.

[0126]

[0136] According to an embodiment, the circuit 202 may be further configured to calculate a mean squared error based on the first local density map and the second local density map. A density distortion metric value may be calculated further based on the calculated mean squared error. The density distortion metric may indicate a PSNR associated with the test point cloud 116.

[0127]

[0137] According to an embodiment, the circuit 202 may be further configured to select an RD point from the plurality of RD points as the optimal rate to be used to encode the reference point cloud 112. The selection may be made based on a determination that the calculated value of the density distortion metric exceeds a threshold value.

[0128]

[0138] According to an embodiment, the circuit 202 may be further configured to train the machine learning-based encoder 210A on the point cloud encoding task based on the first auxiliary loss using the calculated value of the density distortion metric.

[0129]

[0139] According to an embodiment, the circuit 202 may be further configured to train the machine learning-based decoder 210B on a point cloud reconstruction task based on a second auxiliary loss using the calculated value of the density distortion metric.

[0130]

[0140] The present disclosure can be implemented in the form of hardware or in the form of a combination of hardware and software. The present disclosure can be implemented in a centralized manner in at least one computer system, or in a distributed manner where different elements can be distributed across several interconnected computer systems. Any computer system or other device adapted to perform the methods described herein can be suitable. The combination of hardware and software can be a general-purpose computer system including a computer program that, when loaded and executed, can control the computer system to perform the methods described herein. The present disclosure can be implemented in the form of hardware including portions of integrated circuits that also perform other functions.

[0131]

[0141] The present disclosure may also be embodied in a computer program product, which includes all features that enable the implementation of the methods described herein and which is capable of executing these methods when loaded into a computer system. A computer program in this context means any expression, in any language, code or notation, of a set of instructions intended to cause a system having information processing capabilities to perform a particular function, either directly, or after a) conversion into another language, code or notation, or b) reproduction in a different content form, or both.

[0132]

[0142] While the present disclosure has been described with reference to particular embodiments, those skilled in the art will recognize that various modifications can be made and equivalents substituted without departing from the scope of the disclosure. Additionally, many modifications can be made to adapt a particular situation or content to the teachings of the disclosure without departing from the scope of the disclosure. Therefore, it is intended that the present disclosure not be limited to the disclosed embodiments, but rather to include all embodiments falling within the scope of the appended claims. [Explanation of symbols]

[0133] 100 Network Environment 102 Electronic Devices 104 Display Devices 106 Server 108 Communication Network 110 Point Cloud Codec 112 Reference Point Cloud 114 Encoded Point Cloud Data 116 test point clouds 200 Block Diagram 202 circuits 204 memory 206 Input / Output (I / O) Devices 208 Network Interface 210A Machine Learning Based Encoder 210B Machine Learning Based Decoder 212 Display Devices 300 Processing Pipeline 302 Reference Point Cloud 304 test point cloud 306 Bounding Box Determination 308 Determining the Number of 3D Points 310 Radius Calculation 312 Calculating the volume of a sphere 314 Counting Nearby 3D Points 316 First Local Density Estimation Counting 318 Nearby 3D Points 320 Second Local Density Estimation 400 Example Scenarios 402 Reference Point Cloud 404 Test Point Cloud 406 area 408 areas 410 balls 412 3D points 414 3D position 416 sphere 502 First local density map 504 Second local density map 506 Difference Density Map 508 Accumulator 600 Example Scenarios 602 Reference Point Cloud 602A~602G 3D points 604 test point cloud 604A~604D 3D points 606A~606G 3D position 608A~608G 3D position 610A~610D 3D position 612A~612D 3D position 700A, 700B Example graphs 702 area 704 area 800 Exemplary Processing Pipeline 802 Reference Data 804 Test Data 806 Bounding Box Determination 808 Determining the number of 3D points 810 Radius Calculation 812 Calculating the volume of a sphere Counting 814 Nearby 3D Points 816 Third Local Density Estimation Counting 818 Nearby 3D Points 820 Fourth Local Density Estimation 900 Example Scenarios 902 Reference Point Cloud 904 test point cloud 906 sphere 908 3D points 910 3D position 912 sphere 914 ball 916 3D points 918 3D position 920 balls 1000 Example Scenarios 1100 Flowchart 1102 start 1104 Get the reference point cloud of an object 1106 Encode the reference point cloud to generate encoded point cloud data 1108 Decode the encoded point cloud data to generate a test point cloud 1110. Generate a first local density map of the reference point cloud, the first local density map representing local density values ​​at each three-dimensional (3D) point of the reference point cloud. 1112 Determine the 3D positions on the test point cloud that correspond to the 3D point positions on the reference point cloud 1114. Generate a second local density map of the test point cloud, the second local density map representing local density values ​​corresponding to the determined 3D positions. 1116. Calculate a density distortion metric value for the test point cloud based on the difference between the first local density map and the second local density map. 1118 Controlling a display device to render information associated with the reconstruction quality of the test point cloud based on the calculated value.

Claims

1. 1. An electronic device comprising: Obtaining a reference point cloud for the object; encoding the reference point cloud to generate encoded point cloud data; decoding the encoded point cloud data to generate a test point cloud; generating a first local density map of the reference point cloud, the first local density map representing local density values ​​at each three-dimensional (3D) point of the reference point cloud; determining 3D positions in the test point cloud that correspond to positions of 3D points in the reference point cloud; generating a second local density map of the test point cloud, the second local density map representing a local density value at each 3D location of the determined 3D locations within the test point cloud; calculating a value of a density distortion metric for the test point cloud based on the first local density map and the second local density map; controlling a display device to render information associated with a reconstruction quality of the test point cloud based on the calculated value; and a circuit configured to perform An electronic device comprising:

2. The circuit comprises: determining a bounding box of the reference point cloud; determining a number of 3D points in the reference point cloud; calculating a radius to be used to sample the 3D points of the reference point cloud; the radius is calculated based on the bounding box and the number of the 3D points in the reference point cloud; calculating a sphere volume based on the radius; further configured to perform 2. The electronic device according to claim 1 .

3. The circuit comprises: determining, from the 3D points of the reference point cloud, a first number of 3D points within a neighborhood of each 3D point of the reference point cloud; the number of first 3D points within the neighborhood of each 3D point of the reference point cloud is determined based on coordinates of a corresponding 3D point of the reference point cloud, the reference point cloud, and the radius; determining the local density value at each 3D point of the reference point cloud based on the sphere volume and the number of the first 3D points within the neighborhood of the corresponding 3D point; further configured to perform 3. The electronic device according to claim 2 .

4. The circuit comprises: determining a second number of 3D points within a neighborhood of each 3D position of the determined 3D position from the 3D points of the test point cloud; the number of the second 3D points in the neighborhood is determined based on the radius; and determining the local density value of the second local density map based on the sphere volume and the number of the second 3D points within the neighborhood of a corresponding 3D location of the determined 3D location; further configured to perform 3. The electronic device according to claim 2 .

5. the circuitry is further configured to quantize each of the first local density map and the second local density map based on a determined number of quantization levels; the value of the density-distortion metric is calculated further based on the quantization.

2. The electronic device according to claim 1 .

6. the circuitry is further configured to calculate a mean square error based on the first local density map and the second local density map; the value of the density distortion metric is calculated further based on the calculated mean squared error.

2. The electronic device according to claim 1 .

7. The electronic device of claim 1 , wherein the density distortion metric indicates a peak signal-to-noise ratio (PSNR) associated with the test point cloud.

8. the circuitry is further configured to select, from a plurality of rate-distortion (RD) points, an RD point as an optimal rate to be used to encode the reference point cloud; the selecting is performed based on a determination that the calculated value of the density distortion metric exceeds a threshold.

2. The electronic device according to claim 1 .

9. 10. The electronic device of claim 1, further comprising: a memory configured to store a point cloud codec including a machine learning based encoder and a machine learning based decoder.

10. 10. The electronic device of claim 9, wherein the circuitry is further configured to train the machine learning based encoder on a point cloud encoding task based on a first auxiliary loss using the calculated value of the density-distortion metric.

11. 10. The electronic device of claim 9, wherein the circuitry is further configured to train the machine learning based decoder on a point cloud reconstruction task based on a second auxiliary loss using the calculated value of the density distortion metric.

12. The circuit comprises: selecting the reference point cloud as reference data; selecting the test point cloud as test data; calculating a first mean square error based on the first local density map and the second local density map; further configured as follows:

2. The electronic device according to claim 1 .

13. The circuit comprises: selecting the test point cloud as reference data; selecting the reference point cloud as test data; generating a third local density map of the reference data, the third local density map representing a local density value at each 3D point of the reference data; determining 3D positions in the test data that correspond to positions of 3D points in the reference data; generating a fourth local density map of the test data, the fourth local density map representing a local density value at each of the determined 3D locations within the test data; calculating a second mean square error based on the third local density map and the fourth local density map, the value of the density distortion metric is calculated further based on the calculated first mean squared error, the calculated second mean squared error, a number of 3D points in the reference point cloud, and a number of 3D points in the test point cloud; further configured to perform 13. The electronic device according to claim 12.

14. The electronic device of claim 13 , wherein the density-distortion metric indicates a PSNR associated with the test data.

15. The circuit comprises: determining a bounding box for the reference data; determining a number of 3D points in the reference data; calculating a radius to be used to sample the 3D points of the reference data; the radius is calculated based on the bounding box and the number of the 3D points in the reference data; calculating a sphere volume based on the radius; further configured to perform 14. The electronic device according to claim 13,

16. The circuit comprises: determining, from the 3D points of the reference data, a third number of 3D points within a neighborhood of each 3D point of the reference data; the number of third 3D points within the neighborhood of each 3D point of the reference data is determined based on coordinates of corresponding 3D points of the reference data, the reference data, and the radius; determining a third local density value at each 3D point of the reference data based on the sphere volume and the number of third 3D points within the neighborhood of the corresponding 3D point; further configured to perform 16. The electronic device according to claim 15,

17. The circuit comprises: determining a fourth number of 3D points from the test data 3D points within a neighborhood of each 3D position of the determined 3D position; the number of the fourth 3D points in the neighborhood is determined based on the radius; and determining the local density value of the fourth local density map based on the sphere volume and the number of the fourth 3D points within the neighborhood of a corresponding 3D location of the determined 3D location; further configured to perform 16. The electronic device according to claim 15,

18. 1. A method comprising: In electronic devices, obtaining a reference point cloud of the object; encoding the reference point cloud to generate encoded point cloud data; decoding the encoded point cloud data to generate a test point cloud; generating a first local density map of the reference point cloud, the first local density map representing local density values ​​at each three-dimensional (3D) point of the reference point cloud; determining 3D positions in the test point cloud that correspond to positions of 3D points in the reference point cloud; generating a second local density map of the test point cloud, the second local density map representing a local density value at each of the determined 3D locations within the test point cloud; calculating a value of a density distortion metric for the test point cloud based on the first local density map and the second local density map; controlling a display device to render information associated with a reconstruction quality of the test point cloud based on the calculated value; A method comprising:

19. calculating a mean square error based on the first local density map and the second local density map; the value of the density distortion metric is calculated further based on the calculated mean squared error.

20. The method of claim 18.

20. A non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed by an electronic device, cause the electronic device to perform operations, the operations including: Obtaining a reference point cloud for the object; encoding the reference point cloud to generate encoded point cloud data; decoding the encoded point cloud data to generate a test point cloud; generating a first local density map of the reference point cloud, the first local density map representing local density values ​​at each three-dimensional (3D) point of the reference point cloud; determining 3D positions in the test point cloud that correspond to positions of 3D points in the reference point cloud; generating a second local density map of the test point cloud, the second local density map representing a local density value at each 3D location of the determined 3D locations within the test point cloud; calculating a value of a density distortion metric for the test point cloud based on the first local density map and the second local density map; controlling a display device to render information associated with a reconstruction quality of the test point cloud based on the calculated value; and Including, 1. A non-transitory computer-readable medium comprising:

Images