Quantization step parameters for point cloud compression

Abstract

To provide a technique to parameterize the quantization scheme of the attribute coding of point cloud compression algorithms.SOLUTION: On the basis of fixed-point arithmetic, an algorithm calculates a quantization step size (QS) in fixed-point notation, given a user-input quantization parameter (QP).SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] [Cross - Reference to Related Applications] This application claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application No. 62 / 819,830, filed on Mar. 18, 2019, entitled "QUANTIZATION STEP PARAMETER FOR POINT CLOUD COMPRESSION", the entire disclosure of which is hereby incorporated by reference for all purposes.

[0002] The present invention relates to 3D graphics. More specifically, the present invention relates to the coding of 3D graphics.

Background Art

[0003] Point clouds have been considered as a candidate format for the transmission of 3D data captured by 3D scanners, LIDAR sensors or used in common applications such as virtual reality / augmented reality (VR / AR). A point cloud is a set of points in 3D space. In addition to the spatial position (X, Y, Z), each point usually has associated attributes such as color (R, G, B) or even reflectance and a time - stamp over time (e.g., within a LIDAR image). To obtain a high - fidelity representation of the target 3D object, a device captures a point cloud of thousands or even millions of points. Further, for dynamic 3D scenes used in VR / AR applications, each single frame often has its own high - density point cloud, and as a result, millions of point clouds are to be transmitted per second. For the feasible transmission of such large amounts of data, compression is often applied.

[0004] In 2017, MPEG issued a Call for Proposals (CfP) for point cloud compression. After evaluating several proposals, MPEG is considering two different techniques for point cloud compression: a 3D native coding technique (based on octree and similar coding methods), or conventional video coding after projection from 3D to 2D. For dynamic 3D scenes, MPEG uses Test Model Software (TMC2), which is based on patch surface modeling, projection of patches from 3D images to 2D images, and encoding the 2D images with a video encoder such as HEVC. This method has been found to be more efficient than native 3D coding and can achieve competitive bitrates at acceptable quality.

[0005] When encoding a point cloud, TMC2 encodes auxiliary information related to patch projection, such as patch locations and bounding box sizes within a 2D canvas image. For temporal coding of this auxiliary information, patch matching is used between patches from the current point cloud and patches from immediately decoded point clouds to make predictions. This procedure is limited to neighbors and involves performing delta coding for all frames in the sequence. [Overview of the project] [Problems that the invention aims to solve]

[0006] This specification describes techniques for parameterizing the quantization scheme of attribute coding in a point cloud compression algorithm. Based on fixed-point operations, the algorithm calculates the quantization step size (QS) in fixed-point notation, given a user input quantization parameter (QP). [Means for solving the problem]

[0007] In one embodiment, a method programmed into the non-temporary memory of the device includes the steps of: acquiring 3D content; performing geometry coding of the 3D content; and performing fixed-point attribute coding using quantization parameters. The quantization parameters are the same for chromaticity and luminance. The quantization parameters are different for chromaticity and luminance. The chromaticity quantization parameter is proportional to the luminance quantization parameter. The chromaticity quantization parameter is a function of the luminance quantization parameter. The quantization parameters correspond to the quantization step size. A table is used for the correspondence between the quantization parameters and the quantization step size.

[0008] In another embodiment, the device includes non-temporary memory for storing an application, the application being for performing the steps of: acquiring 3D content; performing geometry coding of the 3D content; and performing fixed-point attribute coding using quantization parameters; and a processor coupled to the memory and configured to process the application. The quantization parameters are the same for chromaticity and luminance. The quantization parameters are different for chromaticity and luminance. The chromaticity quantization parameter is proportional to the luminance quantization parameter. The chromaticity quantization parameter is a function of the luminance quantization parameter. The quantization parameters correspond to the quantization step size. A table is used for the correspondence between the quantization parameters and the quantization step size.

[0009] In another embodiment, the system includes one or more devices for acquiring three-dimensional content, and an encoder for encoding the three-dimensional content, wherein the encoding includes performing geometry coding of the three-dimensional content and performing fixed-point attribute coding using quantization parameters. The quantization parameters are the same for chromaticity and luminance. The quantization parameters are different for chromaticity and luminance. The chromaticity quantization parameter is proportional to the luminance quantization parameter. The chromaticity quantization parameter is a function of the luminance quantization parameter. The quantization parameters correspond to the quantization step size. A table is used for the correspondence between the quantization parameters and the quantization step size. [Brief explanation of the drawing]

[0010] [Figure 1] This is a flowchart of attribute coding methods according to several embodiments. [Figure 2] This figure shows the results of region-adaptive hierarchical transform (RAHT) using QP according to several embodiments. [Figure 3] This figure shows the results of region-adaptive hierarchical transform (RAHT) using QP according to several embodiments. [Figure 4] This is a block diagram of an exemplary computer device configured to implement a QP point cloud compression method according to several embodiments. [Modes for carrying out the invention]

[0011] This specification describes techniques for parameterizing the quantization scheme of attribute coding in a point cloud compression algorithm. Based on fixed-point operations, the algorithm calculates the quantization step size (QS) in fixed-point notation, given a user input quantization parameter (QP).

[0012] MPEG (Moving Picture Experts Group) currently specifies a standard for point cloud compression (PCC). Point clouds are used to represent three-dimensional scenes and objects and consist of volume elements (voxels) described by their geometric or visual properties. TMC13 is a test model software maintained and distributed by MPEG that constantly incorporates new proposals approved by contributors. The standard's geometric property-based compression scheme (called G-PCC) can perform attribute coding using a quantization framework and is also implemented in the TMC13 software. Quantization is the process of creating an irreversible compression scheme by mapping a range of values ​​to a single value. In the context of G-PCC, quantization reduces the dynamic range of the transformed attribute coefficients.

[0013] Compression of 3D data using 3D native technologies is typically performed using the spatial coordinates of points that form a mesh or point cloud. While either method can represent 3D content, point clouds are preferred in some applications. In the latter case, octree can effectively perform geometry compression. In addition to geometry, attributes such as color and reflectivity can also be compressed. In this case, transformations such as region-adaptive hierarchical transform (RAHT) and lifting transform have proven to be the most successful methods. The most frequent applications of this compression architecture, as envisioned in the MPEG test model software TMC13, include static and dynamically acquired point clouds.

[0014] Traditionally, only QS (Quadratic Score) was used for quantization purposes. This specification describes mapping from QP (Quadratic Particle) to QS using fixed-point operations. Furthermore, mapping QP to QS using fixed-point operations enables a Rate Distortion Optimization (RDO) framework based on QP. In addition, by selecting different QPs for each channel, improved bit allocation can be performed between attribute channels, for example, the chromaticity QP can be different from the luminance QP.

[0015] QP enables the use of finer rate distortion assignment and fixed-point calculations. Different quantization parameters can be used for luminance and chromaticity, or they can generate relationships between QPs of different attribute channels.

[0016] Figure 1 shows flowcharts of attribute coding methods according to several embodiments. Point cloud data typically includes geometry and color attribute information. In step 100, geometry coding of the 3D content (e.g., point cloud) is implemented. Geometry coding can be implemented in any way, such as 3D native coding techniques (based on octree and similar coding methods) or conventional video coding after projection from 3D to 2D. In step 102, attribute transfer is performed. In step 104, fixed-point attribute coding is performed. Graph transformations can be used for color and attribute coding. Fixed-point attribute coding transmits QP instead of QS in the bitstream. In some embodiments, the same QP is used for luminance and chromaticity, and in some embodiments, different QPs are used. Using different QPs for chromaticity and luminance allows for improved bit allocation. QP is transmitted instead of QS in the TMC13 bitstream. The updated configuration file uses QP instead of QS luminance (QSLuma) or QS chromaticity (QSChroma). In step 106, the bitstream is coded. After geometry coding and fixed-point attribute coding, a bitstream is generated and sent to the decoder. In step 108, geometry decoding is implemented. Geometry decoding is based on geometry coding to ensure proper decoding. In step 110, fixed-point attribute decoding is performed. Fixed-point attribute decoding utilizes QP transmitted in the bitstream. In some embodiments, fewer or additional steps are implemented. In some embodiments, the order of the steps is changed.

[0017] The table below shows the correspondence between QS and QP under CTC conditions. TIFF0007861359000001.tif65156

[0018] Using equation (1), the quantization step Δ is obtained from the quantization parameter QP. step We seek. Using the fixed-point notation (scale k) of TIFF0007861359000002, Equation (1) becomes as follows. In TIFF0007861359000003, here In TIFF0007861359000004, it is as follows. Assume α = 4 and k = 8, and find the quantization. TIFF0007861359000005.tif5157

[0019] When converting the quantization of the transformed attribute to QP and executing, the Lagrangian rate distortion function J can be defined as follows. J(QP) = R(QP) + λ(QP) D(QP)

[0020] The bitrate allocation between luminance and chrominance is controlled by QP.

[0021] Considering that luminance and chrominance have specific characteristics, different QPs can be used for each channel to achieve better bit allocation. QPL (QP for luminance) and QPC (QP for chrominance) are quantization parameters for luminance and chrominance, respectively. Therefore, QPC as a function of QPL is as follows. QPC = φ(QPL).

[0022] Considering that different attributes have specific characteristics, different QPs can be used for each attribute to achieve better bit allocation. QP can be a base quantization parameter, and QPA 0..n-1 can be the quantization parameter for attribute A0 and A n-1 . QPA i is described as a function of QP as follows. TIFF0007861359000006.tif949

[0023] Figures 2 and Figure 3 show the results of RAHT using QP according to several embodiments.

[0024] Figure 4 shows a block diagram of an exemplary computer device configured to implement a QP point cloud compression method according to several embodiments. The computer device 400 can be used to acquire, store, calculate, process, communicate, and / or display information such as images and videos, including 3D content. The computer device 400 can implement any form of point cloud compression. Generally, a suitable hardware structure for implementing the computer device 400 includes a network interface 402, memory 404, a processor 406, an I / O device 408, a bus 410, and a storage device 412. The choice of processor is not critical as long as a suitable processor with sufficient speed is selected. The memory 404 can be any conventional computer memory known in the art. The storage device 412 can include a hard drive, CD-ROM, CDRW, DVD, DVDRW, high-definition disk / drive, ultra-high-definition drive, flash memory card, or any other storage device. The computer device 400 can include one or more network interfaces 402. An example of a network interface includes a network card connected to Ethernet or another type of LAN. The I / O device 408 may include one or more of the following: a keyboard, mouse, monitor, screen, printer, modem, touchscreen, button interface, and other devices. The QP point cloud compression application 430 used to implement the QP point cloud compression method is likely to be stored in the storage device 412 and memory 404 and processed as the application normally would. The computer device 400 may include more or fewer components than those shown in Figure 4. In some embodiments, QP point cloud compression hardware 420 is included. The computer device 400 in Figure 4 includes the application 430 and hardware 420 for the QP point cloud compression method, but the QP point cloud compression method may be implemented in the computer device as hardware, firmware, software, or any combination thereof.For example, in some embodiments, the QP point cloud compression application 430 is programmed in memory and executed using a processor. In another example, in some embodiments, the QP point cloud compression hardware 420 is programmed hardware logic including gates specifically designed to implement the QP point cloud compression method.

[0025] In some embodiments, the QP point cloud compression application 430 includes several applications and / or modules. In some embodiments, a module also includes one or more submodules. In some embodiments, fewer or additional modules may be included.

[0026] In some embodiments, the QP point cloud compression hardware 420 includes camera components such as a lens, an image sensor, and / or other optional camera components.

[0027] Examples of suitable computer devices include personal computers, laptop computers, computer workstations, servers, mainframe computers, handheld computers, personal digital assistants, cellular phones / mobile phones, smart home appliances, game consoles, digital cameras, digital camcorders, camera phones, smartphones, portable music players, tablet computers, mobile devices, video players, video disc writers / players (e.g., DVD writers / players, high-definition disc writers / players, ultra-high-definition disc writers / players), televisions, home entertainment systems, augmented reality devices, virtual reality devices, smart jewelry (e.g., smartwatches), vehicles (e.g., autonomous vehicles), or any other suitable computer devices.

[0028] To utilize the QP point cloud compression method, the device acquires or receives 3D content and processes and / or transmits the content in an optimized manner to enable proper and efficient display of the 3D content. The QP point cloud compression method can be implemented with or without user assistance.

[0029] During operation, the QP point cloud compression method, in addition to using different QPs for luminance and chromaticity or any given set of attributes, allows for finer rate distortion allocation by using QP instead of QS, thereby enabling better bitrate allocation.

[0030] Several embodiments of quantization step parameters for point cloud compression 1. A method programmed into the device's non-temporary memory, Steps to acquire 3D content, The steps include performing geometry coding of the aforementioned 3D content, The steps include: performing fixed-point attribute coding using quantization parameters, A method that includes this.

[0031] 2. The method according to paragraph 1, wherein the quantization parameters are the same for chromaticity and luminance.

[0032] 3. The quantization parameters are different for chromaticity and luminance, as described in paragraph 1.

[0033] 4. The method described in item 3, wherein the chromaticity quantization parameter is proportional to the luminance quantization parameter.

[0034] 5. The method described in item 3, wherein the chromaticity quantization parameter is a function of the luminance quantization parameter.

[0035] 6. The quantization parameter is the method described in paragraph 1, corresponding to the quantization step size.

[0036] 7. The method of paragraph 6, wherein a table is used for the correspondence between the quantization parameter and the quantization step size.

[0037] 8. A device, Non-temporary memory for storing applications, wherein the applications are Steps to acquire 3D content, The steps include performing geometry coding of the aforementioned 3D content, The steps include: performing fixed-point attribute coding using quantization parameters, Non-temporary memory, which is for executing this, A processor coupled to the memory and configured to process the application, A device that includes this.

[0038] 9. The apparatus described in paragraph 8, wherein the quantization parameters are the same for chromaticity and luminance.

[0039] 10. The apparatus described in paragraph 8, wherein the quantization parameters differ with respect to chromaticity and luminance.

[0040] 11. The apparatus described in item 10, wherein the chromaticity quantization parameter is proportional to the luminance quantization parameter.

[0041] 12. The apparatus described in Section 10, wherein the chromaticity quantization parameter is a function of the luminance quantization parameter.

[0042] 13. The apparatus described in Section 8, wherein the quantization parameter corresponds to the quantization step size.

[0043] 14. The apparatus described in paragraph 13, which utilizes a table for the correspondence between the quantization parameters and the quantization step size.

[0044] 15. A system, One or more devices for acquiring 3D content, An encoder for encoding the three-dimensional content, wherein the encoding is Performing geometry coding of the aforementioned 3D content, Using quantization parameters, perform fixed-point attribute coding, Encoders, including, A system that includes this.

[0045] 16. The system described in paragraph 15, wherein the quantization parameters are the same for chromaticity and luminance.

[0046] 17. The system described in paragraph 15, wherein the quantization parameters differ with respect to chromaticity and luminance.

[0047] 18. The system described in Section 17, wherein the chromaticity quantization parameter is proportional to the luminance quantization parameter.

[0048] 19. The system described in Section 17, wherein the chromaticity quantization parameter is a function of the luminance quantization parameter.

[0049] 20. The quantization parameters correspond to the quantization step size in the system described in Section 15.

[0050] 21. The system described in paragraph 20, which uses a table for the correspondence between the quantization parameters and the quantization step size.

[0051] To facilitate understanding of the structure and operating principles of the present invention, the invention has been described in relation to specific embodiments incorporating detailed features. Such references to specific embodiments and their details in this specification are not intended to limit the claims appended herein. It will be readily apparent to those skilled in the art that various other modifications can be made to the embodiments selected for illustrative purposes without departing from the spirit and scope of the invention as defined by the claims. [Explanation of symbols]

[0052] 100 Geometry Coding 102 Attribute Transfer 104 Fixed point attribute encoding 106-bit stream coding 108 Geometry Decoding 110 Fixed-point attribute decoding 400 Computer devices 402 Network Interface 404 memory 406 Processors 408 I / O devices 410 Bus 412 Storage device 420 QP Point Cloud Compression Hardware 430 QP Point Cloud Compression Application

Claims

1. A method programmed into the device's non-temporary memory, A step of acquiring 3D content, wherein the 3D content is a point cloud, A step of performing geometry coding of the three-dimensional content, wherein the geometry coding is geometry-based point cloud compression (G-PCC) or MPEG-type video coding after projection from 3D to 2D, A step of performing fixed-point attribute coding, wherein the method involves referring to a table that associates quantization parameters (QP) represented by integers with quantization step sizes (QS) represented by integers, and determining the corresponding quantization size step based on a pre-specified quantization parameter using fixed-point calculations, wherein the fixed-point attribute coding is a domain adaptive hierarchy transformation (RAHT) or lifting transformation using fixed-point calculations, A step of sending QP in a bitstream instead of quantization step size (QS), A method characterized by including the following.

2. The method according to claim 1, characterized in that the attribute to be encoded is chromaticity and luminance, and the quantization parameter is the same for chromaticity and luminance.

3. The method according to claim 1, characterized in that the attribute to be encoded is chromaticity and luminance, and the quantization parameters differ with respect to chromaticity and luminance.

4. The method according to claim 3, characterized in that the chromaticity quantization parameter is proportional to the luminance quantization parameter.

5. The method according to claim 3, characterized in that the chromaticity quantization parameter is a function of the luminance quantization parameter.

6. It is a device, Non-temporary memory for storing applications, wherein the applications are A step of acquiring 3D content, wherein the 3D content is a point cloud, A step of performing geometry coding of the three-dimensional content, wherein the geometry coding is geometry-based point cloud compression (G-PCC) or MPEG-type video coding after projection from 3D to 2D, A step of performing fixed-point attribute coding, wherein the method involves referring to a table that associates quantization parameters (QP) represented by integers with quantization step sizes (QS) represented by integers, and determining a corresponding quantization size step based on a pre-specified quantization parameter using fixed-point calculations, wherein the fixed-point attribute coding is a domain adaptive hierarchy transformation (RAHT) or lifting transformation using fixed-point calculations. A step of sending QP in a bitstream instead of the quantization step size, Non-temporary memory, which is for executing this, A processor coupled to the memory and configured to process the application, An apparatus characterized by including

7. The apparatus according to claim 6, characterized in that the attribute to be encoded is chromaticity and luminance, and the quantization parameter is the same for chromaticity and luminance.

8. The apparatus according to claim 6, characterized in that the encoded attributes are chromaticity and luminance, and the quantization parameters differ with respect to chromaticity and luminance.

9. The apparatus according to claim 8, characterized in that the chromaticity quantization parameter is proportional to the luminance quantization parameter.

10. The apparatus according to claim 8, characterized in that the chromaticity quantization parameter is a function of the luminance quantization parameter.

11. It is a system, One or more devices for acquiring 3D content, wherein the 3D content is a point cloud, and An encoder for encoding the three-dimensional content, wherein the encoding is The process involves performing geometry coding of the aforementioned three-dimensional content, wherein the geometry coding is geometry-based point cloud compression (G-PCC) or MPEG-type video coding after projection from 3D to 2D. The process involves performing fixed-point attribute coding, which involves referring to a table that associates quantization parameters (QP) represented by integers with quantization step sizes (QS) represented by integers, and determining the corresponding quantization size step based on the pre-specified quantization parameters using fixed-point calculations, wherein the fixed-point attribute coding is a domain adaptive hierarchical transformation (RAHT) or lifting transformation using fixed-point calculations. Instead of the quantization step size, the QP is transmitted as a bitstream, Encoders, including, A system characterized by including

Images