Point cloud data processing method, device and medium

Abstract

The present disclosure provides a point cloud data processing method, device and medium. In a method for performing point cloud attribute decoding in a point cloud decoder, a sorting method for a point set of a point cloud is determined. Based on the first signaling information in the encoded bit stream of the point cloud, a quantization parameter for quantizing the transform coefficients of the Haar-based attribute encoding of the point cloud is determined. The Haar-based attribute encoding is configured to perform point cloud compression on the point set of the point cloud based on the Haar transform. According to the quantization parameter determined based on the first signaling information, an entropy decoding method for decoding the quantized transform coefficients is determined. Based on the entropy decoding method, the quantized transform coefficients are further determined. Based on the determined sorting method and the determined quantized transform coefficients, the attributes of the point cloud are reconstructed.

Description

[0001] Incorporation by reference

[0002] This application claims priority to U.S. Patent Application No. 17 / 446,263, filed on August 27, 2021, entitled “METHOD AND APPARATUS FOR HAAR-BASED POINT CLOUD CODING,” which claims priority to U.S. Provisional Application No. 63 / 159,409, filed on March 10, 2021, entitled “ON HAAR-BASED ATTRIBUTE CODING FOR POINT CLOUDS.” The entire disclosure of the prior application is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure describes embodiments generally related to point cloud coding, including improvements to Haar-based attribute coding of point clouds. Background Art

[0004] The background description provided herein is intended to present the background of the present application as a whole. To the extent that the work of the presently named inventors is described in the background section and in various aspects of this specification, it is not intended that it be prior art at the time of filing this application, and it is neither expressly nor impliedly admitted that it is prior art to the present application.

[0005] Various technologies have been developed to capture and represent the world in three-dimensional (3D) space, such as objects in the world, the world's environment, and so on. A 3D representation of the world enables more immersive interactions and communications. Point clouds can be used as 3D representations of the world. A point cloud is a collection of points in 3D space, each point having associated attributes such as color, material properties, texture information, intensity attributes, reflectivity attributes, motion-related attributes, modality attributes, and / or various other attributes. Such point clouds can include large amounts of data, and storage and transmission can be expensive and time-consuming. Summary of the Invention

[0006] Aspects of the present disclosure provide methods and apparatus for point cloud compression and decompression. According to one aspect of the present disclosure, a method for performing point cloud attribute decoding in a point cloud decoder is provided. In the method, a sorting method for a point set of a point cloud can be determined. Based on first signaling information in an encoded bit stream of the point cloud, a quantization parameter for quantizing the transform coefficients of the Haar-based attribute encoding of the point cloud can be determined. The Haar-based attribute encoding can be configured to perform point cloud compression on the point set of the point cloud based on a Haar transform. Based on the quantization parameter determined based on the first signaling information, an entropy decoding method for decoding the quantized transform coefficients can be determined. Based on the entropy decoding method, the quantized transform coefficients can be further determined. Based on the determined sorting method and the determined quantized transform coefficients, the attributes of the point cloud can be reconstructed.

[0007] In one example, the sorting method may be determined based on second signaling information in the encoded bitstream of the point cloud. When the second signaling information is equal to a first value, the sorting method is Morton sorting. In another example, when the second signaling information is equal to a second value, the sorting method is Hilbert sorting.

[0008] In some embodiments, the first signaling information may be set based on one of the quantization parameter, an application of the point cloud, and a characteristic of the point cloud.

[0009] In the method, when the entropy decoding method is exponential Golomb coding, the order k of the exponential Golomb coding can be determined based on third signaling information in the encoded bit stream, wherein the third signaling information indicates the order k of the exponential Golomb coding.

[0010] In this method, when the entropy decoding method is exponential Golomb coding, the order k of the exponential Golomb coding can be determined based on the quantization parameter. The max_k is a maximum order of the Exponential Golomb coding used to decode the quantized transform coefficients, the trans_coeff_qp is a value of the quantization parameter, and the s is a positive integer equal to six or eight.

[0011] In some embodiments, to determine the quantized transform coefficient based on an entropy decoding method, the quantized transform coefficient is decoded based on the exponential Golomb coding and the determined order k of the exponential Golomb coding, where the absolute value of a threshold is subtracted from the quantized transform coefficient, and the threshold is equal to one of 1, 2, and 3.

[0012] In some embodiments, in order to determine the quantized transform coefficient based on the entropy decoding method, a first quantized transform coefficient x among the quantized transform coefficients may be decoded. The first quantized transform coefficient x may be encoded by the exponential Golomb coding, so that the first part is encoded by the exponential Golomb coding with the order k equal to zero. Encode and encode the second part (x and 2 k The modular operation of ) is encoded.

[0013] In the method, the set of points in the point cloud may be sorted based on the determined sorting method. The sorted set of points in the point cloud may be partitioned into a plurality of segments based on a partition size indicated by partition signaling information in the encoded bitstream. Each of the plurality of segments in the point cloud may be reconstructed separately based on the determined quantized transform coefficients.

[0014] In some embodiments, the number of the plurality of segments may be equal to The N may be the number of point sets in the point cloud, and the M may be a positive integer. It can be an integer greater than or equal to N / M.

[0015] In this method, to reconstruct a point cloud, a scaling factor of 1 / 2 of a Haar transform matrix associated with the Haar-based attribute encoding may be determined. Based on the Haar-based attribute encoding, an inverse quantization operation may be performed on the quantized transform coefficients to obtain transform coefficients. Based on the Haar transform matrix and the transform coefficients, an inverse transform operation may be performed on the point cloud to obtain attribute residuals. Attributes of the point cloud may be further reconstructed based on the attribute residuals and predicted attributes of the point cloud obtained through the Haar-based attribute encoding.

[0016] The device for processing point cloud data is configured to perform the above-mentioned method. For example, the device for processing point cloud data includes a processing circuit, and the processing circuit is configured to perform one or more methods described above. For example, the device may include a processing circuit for determining a sorting method for a set of points in the point cloud. The processing circuit may also be used to determine, based on first signaling information in an encoded bit stream of the point cloud, a quantization parameter for quantizing the transform coefficients of the Haar-based attribute encoding of the point cloud. The processing circuit is also used to determine, based on the quantization parameter determined based on the first signaling information, an entropy decoding method for decoding the quantized transform coefficients. The processing circuit is further used to determine the quantized transform coefficients based on the entropy decoding method. The processing circuit is also used to reconstruct the attributes of the point cloud based on the determined sorting method and the determined quantized transform coefficients.

[0017] In some embodiments, the processing circuit is further configured to sort the set of points of the point cloud based on the determined sorting method. The processing circuit is then configured to partition the sorted set of points of the point cloud into a plurality of segments based on a partition size indicated by partition signaling information in the encoded bitstream. Subsequently, the processing circuit may be configured to reconstruct each of the plurality of segments in the point cloud based on the determined quantized transform coefficients. In some embodiments, the number of the plurality of segments is equal to The N may be the number of point sets in the point cloud, and the M may be a positive integer. It can be an integer greater than or equal to N / M.

[0018] According to one aspect of the present disclosure, a non-volatile computer-readable storage medium is provided. The non-volatile computer-readable storage medium stores instructions, and when the instructions are executed by at least one processor, the at least one processor performs one or more of the methods described above. For example, in this method, a sorting method for a set of points in the point cloud can be determined. Based on the first signaling information in the encoded bit stream of the point cloud, a quantization parameter for quantizing the transform coefficients of the Haar-based attribute encoding in the point cloud can be determined. Based on the quantization parameter determined based on the first signaling information, an entropy decoding method for decoding the quantized transform coefficients can be determined. Based on the entropy decoding method, the quantized transform coefficients can be further determined. Based on the determined sorting method and the determined quantized transform coefficients, the attributes of the point cloud can be reconstructed. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Other features, properties, and various advantages of the disclosed subject matter will become further apparent from the following detailed description and accompanying drawings, in which:

[0020] Figure 1 is a schematic diagram of a simplified block diagram of a communication system according to an embodiment;

[0021] Figure 2 is a schematic diagram of a simplified block diagram of a streaming system according to another embodiment;

[0022] Figure 3 shows a block diagram of an encoder for encoding a point cloud frame according to some embodiments;

[0023] Figure 4 shows a block diagram of a decoder for decoding a compressed bitstream corresponding to a point cloud frame according to some embodiments;

[0024] Figure 5 shows a block diagram of an encoder for encoding a point cloud frame according to some embodiments;

[0025] Figure 6 shows a block diagram of a decoder for decoding a compressed bitstream corresponding to a point cloud frame according to some embodiments;

[0026] Figure 7 A block diagram illustrating a forward transform in lifting-based attribute coding according to some embodiments;

[0027] Figure 8 A block diagram illustrating an inverse transform in lifting-based attribute coding according to some embodiments;

[0028] Figure 9A A diagram illustrating forward transform in attribute coding based on Region Adaptive Hierarchical Transform (RAHT) according to some embodiments of the present disclosure is shown;

[0029] Figure 9B A diagram illustrating inverse transform in attribute coding based on Region Adaptive Hierarchical Transform (RAHT) according to some embodiments of the present disclosure;

[0030] Figure 10 shows a block diagram of partitioning a point cloud into segments according to some embodiments of the present disclosure;

[0031] Figure 11 shows a flow chart outlining an exemplary decoding process according to some embodiments;

[0032] Figure 12 is a schematic diagram of a computer system according to an embodiment. DETAILED DESCRIPTION

[0033] In recent years, point clouds have become increasingly popular. For example, they can be used for object detection and localization in autonomous vehicles. They can also be used in Geographic Information Systems (GIS) for mapping and in cultural heritage to visualize and archive cultural heritage objects and collections.

[0034] A point cloud can contain a collection of high-dimensional points, typically three-dimensional (3D). Each of these high-dimensional points can include 3D position information as well as other properties, such as color and reflectivity. High-dimensional points can be captured in a variety of settings using multiple cameras and depth sensors, or lidar, and can consist of thousands or billions of points to realistically represent the original scene.

[0035] Therefore, compression techniques are needed to reduce the amount of data required to represent point clouds, allowing for faster transmission or reduced storage. ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) has created an ad hoc group (MPEG-PCC) to standardize compression techniques for static or dynamic point clouds. Separately, China's Audio and Video Coding Standards Working Group has also created an ad hoc group (AVS-PCC) to standardize point cloud compression.

[0036] Figure 1 A simplified block diagram of a communication system (100) according to an embodiment of the present disclosure is shown. The communication system (100) includes a plurality of terminal devices capable of communicating with each other via, for example, a network (150). For example, the communication system (100) includes a pair of terminal devices (110) and (120) interconnected via the network (150). Figure 1 In the example of , a first pair of terminal devices (110) and (120) can perform a one-way transmission of point cloud data. For example, the terminal device (110) can compress a point cloud (e.g., a plurality of points representing a structure) captured by a sensor (105) connected to the terminal device (110). The compressed point cloud (e.g., in the form of a bit stream) can be transmitted to another terminal device (120) via a network (150). The terminal device (120) can receive the compressed point cloud from the network (150), decompress the bit stream to reconstruct the point cloud, and appropriately display the reconstructed point cloud. One-way data transmission may be common in media service applications, etc.

[0037] exist Figure 1 In the example of , the terminal devices (110) and (120) may be illustrated as a server and a personal computer, but the principles of the present disclosure may not be limited thereto. Embodiments of the present disclosure may be used in laptop computers, tablet computers, smart phones, game consoles, media players and / or dedicated three-dimensional (3D) devices. Network (150) represents any number of networks that transmit compressed point clouds between terminal devices (110) and (120). Network (150) may include, for example, cabled (wired) and / or wireless communication networks. Network (150) may exchange data in circuit-switched channels and / or packet-switched channels. Representative networks include telecommunications networks, local area networks, wide area networks and / or the Internet. For the purposes of this discussion, unless explained below, the architecture and topology of network (150) may be unimportant to the operation of the present disclosure.

[0038] Figure 2 Shown is a simplified block diagram of a streaming system (200) according to an embodiment. Figure 2An example of an application of the disclosed subject matter is point cloud. The disclosed subject matter can also be applied to other point cloud-enabled applications, such as 3D telepresence applications, virtual reality applications, etc.

[0039] The streaming system (200) may include a capture subsystem (213). The capture subsystem (213) may include a point cloud source (201), such as a light detection and ranging (lidar) system, a 3D camera, a 3D scanner, a graphics generation component that generates an uncompressed point cloud in software, and a component that generates, for example, an uncompressed point cloud (202). In an example, the point cloud (202) includes points captured by a 3D camera. The point cloud (202) is depicted as a thick line to emphasize the high amount of data compared to a compressed point cloud (204) (a bitstream of the compressed point cloud). The compressed point cloud (204) may be generated by an electronic device (220) that includes an encoder (203) coupled to the point cloud source (201). The encoder (203) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter as described in more detail below. The compressed point cloud (204) (or a bitstream of the compressed point cloud (204)) (depicted as thin lines to emphasize the lower data volume when compared to the point cloud stream (202)) can be stored on the streaming server (205) for future use. One or more streaming client subsystems (e.g., Figure 2 The client subsystems (206) and (208) in the embodiment of the present invention can access the streaming server (205) to retrieve the copies (207) and (209) of the compressed point cloud (204). The client subsystem (206) can include a decoder (210) in, for example, an electronic device (230). The decoder (210) decodes the input copy (207) of the compressed point cloud and creates an output stream of a reconstructed point cloud (211) that can be rendered on a rendering device (212).

[0040] It should be noted that the electronic devices (220) and (230) may include other components (not shown). For example, the electronic device (220) may include a decoder (not shown), and the electronic device (230) may also include an encoder (not shown).

[0041] In some streaming systems, the compressed point clouds (204), (207), and (209) (e.g., a bitstream of the compressed point clouds) may be compressed according to some standard. In some examples, a video coding standard is used in the compression of the point clouds. Examples of such standards include High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), and the like.

[0042] Figure 3 A block diagram of a V-PCC encoder (300) for encoding a point cloud frame according to some embodiments is shown. In some embodiments, the V-PCC encoder (300) can be used in the communication system (100) and the streaming system (200). For example, the encoder (203) can be configured and operated in a manner similar to the V-PCC encoder (300).

[0043] The V-PCC encoder (300) receives a point cloud frame as an uncompressed input and generates a bitstream corresponding to the compressed point cloud frame. In some embodiments, the V-PCC encoder (300) may receive the point cloud frame from a point cloud source such as the point cloud source (201).

[0044] exist Figure 3 In the example of , the V-PCC encoder (300) includes a patch generation module (306), a patch packing module (308), a geometric image generation module (310), a texture image generation module (312), a patch information module (304), an occupancy map module (314), a smoothing module (336), image filling modules (316) and (318), a group expansion module (320), video compression modules (322), (323) and (332), an auxiliary patch information compression module (338), an entropy compression module (334) and a multiplexer (324).

[0045] According to one aspect of the present disclosure, a V-PCC encoder (300) converts a 3D point cloud frame into an image-based representation, as well as some metadata (e.g., occupancy map and patch information) for converting the compressed point cloud back into a decompressed point cloud. In some examples, the V-PCC encoder (300) can convert the 3D point cloud frame into a geometry image, a texture image, and an occupancy map, and then encode the geometry image, texture image, and occupancy map into a bitstream using video coding techniques. Generally speaking, a geometry image is a 2D image whose pixels are filled with geometry values ​​associated with the points projected onto the pixels, and the pixels filled with geometry values ​​can be referred to as geometry samples. A texture image is a 2D image whose pixels are filled with texture values ​​associated with the points projected onto the pixels, and the pixels filled with texture values ​​can be referred to as texture samples. An occupancy map is a 2D image whose pixels are filled with values ​​indicating whether a patch is occupied or unoccupied.

[0046] A patch may generally refer to a contiguous subset of a surface described by a point cloud. In one example, a patch comprises points whose surface normal vectors deviate from each other by less than a threshold amount. The patch generation module (306) segments the point cloud into a set of patches. The set of patches may or may not overlap, such that each patch can be described by a depth field relative to a plane in 2D space. In some embodiments, the patch generation module (306) aims to decompose the point cloud into a minimum number of patches with smooth boundaries while also minimizing reconstruction error.

[0047] The patch information module (304) can collect patch information indicating the size and shape of the patch. In some examples, the patch information can be packed into an image frame and then encoded by the auxiliary patch information compression module (338) to generate compressed auxiliary patch information.

[0048] The patch packing module (308) is used to map the extracted patches onto a two-dimensional (2D) grid while minimizing unused space and ensuring that each M×M (e.g., 16×16) block in the grid is associated with a unique patch. Effective patch packing can directly affect compression efficiency by minimizing unused space or ensuring temporal consistency.

[0049] The geometry image generation module (310) can generate a 2D geometry image associated with the geometry of the point cloud at a given patch location. The texture image generation module (312) can generate a 2D texture image associated with the texture of the point cloud at a given patch location. The geometry image generation module (310) and the texture image generation module (312) use the 3D to 2D mapping calculated in the packing process to store the geometry and texture of the point cloud as images. To better handle the situation where multiple points are projected to the same sample, each patch is projected onto two images called layers. In one example, the geometry image is represented by a WxH monochrome frame in YUV420-8bit format. To generate the texture image, the texture generation process uses the reconstructed / smoothed geometry to calculate the color associated with the resampled points.

[0050] The occupancy map module (314) can generate an occupancy map that describes fill information for each cell. For example, the occupancy map includes a binary map that indicates whether each cell in the grid belongs to empty space or to a point cloud. In one example, the occupancy map uses binary information to describe whether each pixel is filled. In another example, the occupancy map uses binary information to describe whether each pixel block is filled.

[0051] The occupancy map generated by the occupancy map module (314) can be compressed using lossless encoding or lossy encoding. When lossless encoding is used, the entropy compression module (334) is used to compress the occupancy map; when lossy encoding is used, the video compression module (332) is used to compress the occupancy map.

[0052] Note that the patch packing module (308) may leave some empty space between the packed 2D patches in the image frame. The image padding modules (316) and (318) may fill in the empty space (referred to as padding) to generate an image frame suitable for 2D video and image codecs. Image padding, also known as background padding, may fill in unused space with redundant information. In some examples, good background padding minimally increases the bitrate and does not introduce significant coding artifacts around patch boundaries.

[0053] The video compression modules (322), (323), and (332) can encode 2D images such as filled geometry images, filled texture images, and occupancy maps based on a suitable video coding standard such as HEVC, VVC, etc. In one example, the video compression modules (322), (323), and (332) are independent components that operate independently. It should be noted that in another example, the video compression modules (322), (323), and (332) can be implemented as a single component.

[0054] In some examples, a smoothing module (336) is configured to generate a smoothed image of the reconstructed geometric image. The smoothed image can be provided to a texture image generation module (312). The texture image generation module (312) can then adjust the generation of the texture image based on the reconstructed geometric image. For example, when a patch shape (e.g., a geometric shape) is slightly distorted during encoding and decoding, the distortion can be taken into account when generating the texture image to correct for the distortion in the patch shape.

[0055] In some embodiments, the group dilation module (320) is used to add pixels to the boundaries of objects to reduce compression artifacts and increase coding gain.

[0056] The multiplexer (324) may multiplex the compressed geometry image, the compressed texture image, the compressed occupancy map, and / or the compressed auxiliary patch information into the compressed bitstream.

[0057] Figure 4A block diagram of a V-PCC decoder (400) according to some embodiments is shown, wherein the V-PCC decoder (400) is used to decode a compressed bitstream corresponding to a point cloud frame. In some embodiments, the V-PCC decoder (400) can be used in a communication system (100) and a streaming system (200). For example, the decoder (210) can be configured and operated in a manner similar to the V-PCC decoder (400). The V-PCC decoder (400) receives a compressed bitstream and generates a reconstructed point cloud based on the compressed bitstream.

[0058] exist Figure 4 In the example of , the V-PCC decoder (400) includes a demultiplexer (432), video decompression modules (434) and (436), an occupancy map decompression module (438), an auxiliary patch information decompression module (442), a geometry reconstruction module (444), a smoothing module (446), a texture reconstruction module (448), and a color smoothing module (452).

[0059] The demultiplexer (432) can receive the compressed bit stream and separate it into a compressed texture image, a compressed geometry image, a compressed occupancy map and compressed auxiliary patch information.

[0060] The video decompression modules (434) and (436) can decode compressed images according to appropriate standards (e.g., HEVC, VVC, etc.) and output decompressed images. For example, the video decompression module (434) decodes compressed texture images and outputs decompressed texture images; the video decompression module (436) decodes compressed geometric images and outputs decompressed geometric images.

[0061] The occupancy map decompression module (438) may decode the compressed occupancy map according to an appropriate standard (eg, HEVC, VVC, etc.) and output a decompressed occupancy map.

[0062] The auxiliary patch information decompression module (442) can decode the compressed auxiliary patch information according to an appropriate standard (eg, HEVC, VVC, etc.) and output decompressed auxiliary patch information.

[0063] The geometry reconstruction module (444) can receive the decompressed geometry image and generate a reconstructed point cloud geometry based on the decompressed occupancy map and the decompressed auxiliary patch information.

[0064] The smoothing module (446) can smooth inconsistencies at patch edges. The smoothing process is intended to mitigate potential discontinuities that may appear at patch boundaries due to compression artifacts. In some embodiments, a smoothing filter can be applied to pixels at patch boundaries to mitigate distortion that may be caused by compression / decompression.

[0065] The texture reconstruction module (448) can determine texture information of points in the point cloud based on the decompressed texture image and the smoothed geometry.

[0066] The color smoothing module (452) can smooth out inconsistencies in shading. Non-adjacent patches in 3D space are often packed next to each other in 2D video. In some examples, block-based video codecs may blend pixel values ​​from non-adjacent patches. The goal of color smoothing is to reduce visible artifacts that appear at patch boundaries.

[0067] Figure 5 A block diagram of a G-PPC encoder (500) according to some embodiments is shown. The encoder (500) can be configured to receive point cloud data and compress the point cloud data to generate a bitstream carrying the compressed point cloud data. In an embodiment, the encoder (500) can include a position quantization module (510), a duplicate point removal module (512), an octree encoding module (530), an attribute transfer module (520), a level of detail (LOD) generation module (540), an attribute prediction module (550), a residual quantization module (560), an arithmetic coding module (570), an inverse residual quantization module (580), an addition module (581), and a memory (590) for storing reconstructed attribute values.

[0068] As shown, an input point cloud (501) may be received at an encoder (500). Positions (e.g., 3D coordinates) of the point cloud (501) are provided to a quantization module (510). The quantization module (510) is configured to quantize the coordinates to generate quantized positions. A duplicate point removal module (512) is configured to receive the quantized positions and perform a filtering process to identify and remove duplicate points. An octree encoding module (530) is configured to receive the filtered positions from the duplicate point removal module (512) and perform an octree-based encoding process to generate a sequence of occupancy codes that describe a 3D grid of voxels. The occupancy codes are provided to an arithmetic coding module (570).

[0069] The attribute transfer module (520) is configured to receive attributes of the input point cloud and, when multiple attribute values ​​are associated with each voxel, perform an attribute transfer process to determine the attribute value of each voxel. The attribute transfer process can be performed on the reordered points output from the octree encoding module (530). The attributes after the transfer operation are provided to the attribute prediction module (550). The LOD generation module (540) is configured to operate on the reordered points output from the octree encoding module (530) and reorganize the points into different LODs. The LOD information is provided to the attribute prediction module (550).

[0070] The attribute prediction module (550) processes the points according to the LOD-based order indicated by the LOD information from the LOD generation module (540). The attribute prediction module (550) generates an attribute prediction for the current point based on the reconstructed attributes of the set of neighboring points of the current point stored in the memory (590). A prediction residual may then be obtained based on the original attribute value received from the attribute transfer module (520) and the locally generated attribute prediction. When candidate indices are used in various attribute prediction processes, the index corresponding to the selected prediction candidate may be provided to the arithmetic coding module (570).

[0071] The residual quantization module (560) is configured to receive the prediction residual from the attribute prediction module (550) and perform quantization to generate a quantized residual. The quantized residual is provided to the arithmetic coding module (570).

[0072] The inverse residual quantization module (580) is configured to receive the quantized residual from the residual quantization module (560) and generate a reconstructed prediction residual by performing an inverse operation of the quantization operation performed at the residual quantization module (560). The addition module (581) is configured to receive the reconstructed prediction residual from the inverse residual quantization module (580) and receive the corresponding attribute prediction from the attribute prediction module (550). By combining the reconstructed prediction residual and the attribute prediction, a reconstructed attribute value is generated and stored in the memory (590).

[0073] The arithmetic coding module (570) is configured to receive the occupancy code, the candidate index (if used), the quantized residual (if generated), and other information, and perform entropy coding to further compress the received values ​​or information. Thus, a compressed bitstream (502) carrying the compressed information can be generated. The bitstream (502) can be transmitted or otherwise provided to a decoder that decodes the compressed bitstream, or can be stored in a storage device.

[0074] Figure 6 A block diagram of a G-PCC decoder (600) according to an embodiment is shown. The decoder (600) can be configured to receive a compressed bitstream and perform point cloud data decompression to decompress the bitstream to generate decoded point cloud data. In an embodiment, the decoder (600) can include an arithmetic decoding module (610), an inverse residual quantization module (620), an octree decoding module (630), a level of detail (LOD) generation module (640), an attribute prediction module (650), and a memory (660) for storing reconstructed attribute values.

[0075] As shown, a compressed bitstream (601) can be received at an arithmetic decoding module (610). The arithmetic decoding module (610) is configured to decode the compressed bitstream (601) to obtain a quantized residual (if generated) and an occupancy code of the point cloud. The octree decoding module (630) is configured to determine the reconstructed positions of a plurality of points in the point cloud based on the occupancy code. The LOD generation module (640) is configured to reorganize the plurality of points into different LODs based on the reconstructed positions and determine an LOD-based order. The inverse residual quantization module (620) is configured to generate a reconstructed residual based on the quantized residual received from the arithmetic decoding module (610).

[0076] The attribute prediction module (650) is configured to perform an attribute prediction process to determine attribute predictions for a plurality of points based on an LOD-based order. For example, the attribute prediction for a current point may be determined based on the reconstructed attribute values ​​of a plurality of neighboring points of the current point stored in the memory (660). In some examples, the attribute prediction may be combined with the respective reconstruction residuals to generate a reconstructed attribute for the current point.

[0077] In one example, the sequence of reconstructed attributes generated from the attribute prediction module (650) along with the reconstructed positions generated from the octree decoding module (630) corresponds to the decoded point cloud (602) output from the decoder (600). Additionally, the reconstructed attributes are also stored in the memory (660) and can subsequently be used to derive attribute predictions for subsequent points.

[0078] In various embodiments, the encoder (300), decoder (400), encoder (500) and / or decoder (600) can be implemented using hardware, software or a combination thereof. For example, the encoder (300), decoder (400), encoder (500) and / or decoder (600) can be implemented using processing circuitry, such as one or more integrated circuits (ICs) operating with or without software, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc. In another example, the encoder (300), decoder (400), encoder (500) and / or decoder (600) can be implemented as software or firmware, wherein the software or firmware includes instructions stored in a non-volatile (or non-volatile) computer-readable storage medium. These instructions, when executed by a processing circuit, such as one or more processors, cause the processing circuitry to perform the functions of the encoder (300), decoder (400), encoder (500) and / or decoder (600).

[0079] It should be noted that the attribute prediction module (550) and the attribute prediction module (650) configured to implement the attribute prediction techniques disclosed herein may be included in other decoders or encoders, wherein the other decoders or encoders may have the same Figure 5 and Figure 6 The illustrated structures may be similar or different structures. Additionally, in various examples, the encoder (500) and decoder (600) may be included in the same device or in separate devices.

[0080] In the Test Model (TMC) 13, geometric information and associated attributes of the point cloud (such as color or reflectance) can be compressed separately. The geometric information (3D coordinates of the point cloud) can be encoded by octree partitioning with the occupancy information of the point cloud. The attributes can then be compressed based on the reconstructed geometry using prediction, lifting, and Region Adaptive Hierarchical Transform (RAHT) techniques.

[0081] The prediction-based attribute encoding of point clouds can be described as follows: For simplicity, a level of detail (LoD) can be assumed in the prediction-based attribute encoding.

[0082] Let (P i ) i＝1...N Be the set of locations associated with multiple points in the point cloud, and let (M i ) i＝1...N For (P i ) i＝1...N First, the points can be sorted in ascending order according to their associated Morton codes. Let I be the array of point indices sorted in ascending order. The encoder / decoder can compress / decompress the points according to the order defined by I. At each iteration i, point P can be selected. i . We can analyze P i The distance to s (eg, s=64) previous points, and P can be selected i More precisely, the attribute value (a) can be predicted by using a linear interpolation process based on the distances of the nearest neighbors of point i. i ) i∈1...N .make is the set of k nearest neighbors of the current point i, let Be the decoded / reconstructed attribute value of the set of k nearest neighbors, and let The predicted attribute value can be calculated based on the attribute value of the neighbors and the distance from the neighbor set to the current point according to the following equation (1):

[0083]

[0084] Prediction-based attribute coding in AVS-PCC may be different from attribute coding in MPEG. One difference is the sorting order. The Hilbert sorting order may be used for prediction-based attribute coding in AVS-PCC. The encoding process for prediction-based attribute coding in AVS-PCC may be as follows: First, a lookup table may be used to determine the Hilbert code of a point in a point cloud. The Hilbert code values ​​of the points may then be used to sort all the points in the point cloud. Points with smaller Hilbert codes may appear before points with larger Hilbert codes.

[0085] Lifting-based attribute coding can be built on top of prediction-based attribute coding. Compared with prediction-based attribute coding, lifting-based attribute coding introduces two additional steps: (a) introducing an update operator; and (b) using an adaptive quantization strategy.

[0086] To illustrate, you can Figure 7 and Figure 8 The operation of the promoted attribute encoding is shown in FIG. Figure 7 A block diagram of a forward transform 700 in lifting-based attribute coding is shown, and Figure 8 A block diagram of an inverse transform 800 in lifting-based attribute coding is shown.

[0087] like Figure 7 As shown in , the attribute signal at level N can be divided into a high-pass signal H(N) and a low-pass signal L(N). L(N) can generate a prediction signal P(N) based on the prediction process 702. A difference signal D(N) can be generated based on the difference between H(N) and L(N). The difference signal D(N) can be further updated to generate an updated signal U(N). The sum of U(N) and L(N) can generate an updated low-pass signal L'(N). At the subsequent level (N-1), L'(N) can be further divided into a high-pass signal H(N-1) and a low-pass signal L(N-1). L(N-1) can generate a prediction signal P(N-1) at level N-1. The difference signal D(N-1) at level N-1 can be generated based on the difference between H(N-1) and L(N-1). The difference signal D(N-1) can be further updated to generate an updated signal U(N-1) at level N-1. The sum of U(N-1) and L(N-1) may generate an updated low-pass signal L'(N-1) at level N-1.

[0088] The difference signal D(N-1) and the updated low-pass signal L'(N-1) can be further decomposed into D(N-2) and L'(N-2). The segmentation step can be applied repeatedly until the updated low-pass signal L'(0) of the base layer is obtained.

[0089] exist Figure 8 In the embodiment, an inverse transform 800 of the attribute coding based on lifting is provided. Figure 8 As shown in , a low-pass signal L(0) can be generated at level zero based on the difference between the updated low-pass signal L'(0) and the updated signal U(0). By updating the difference signal D(0), an updated signal U(0) is obtained. L(0) can be further used to generate a prediction signal P(0) based on the prediction process 802. P(0) is further added to D(0) to generate a high-pass signal H(0). H(0) and L(0) are merged to generate an updated low-pass signal L'(1) at level one. The merging step can be repeatedly applied until a high-pass signal H(N) and a low-pass signal L(N) are generated at level N. H(N) and L(N) can be further merged to form a reconstructed attribute signal.

[0090] Figure 9A The forward transform 900A of the RAHT-based attribute coding is shown, and Figure 9B The inverse transform 900B of the RAHT-based attribute coding is shown. Figure 9A and Figure 9B middle, and And w0 is the input coefficient F l+1,2n The sign of the weight, and w1 is the input coefficient F l+1,2n+1 The sign of the weight.

[0091] A Haar-based attribute coding may also be introduced into AVS-PCC. The Haar-based attribute coding technique may be described as follows: The Haar-based attribute coding may be configured to perform point cloud compression on multiple points of a point cloud based on a Haar transform.

[0092] In the first step of Haar-based attribute encoding, multiple points in the point cloud can be sorted using Morton order before attribute encoding, where points with smaller Morton codes can appear before points with larger Morton codes. The coordinates of the kth point in the point cloud can be expressed as (X k , Y k , Z k ), k = 0, ..., N-1, where X k 、Y K and Z k It can be expressed as three L-bit numbers in equations (2)-(4):

[0093]

[0094]

[0095]

[0096] The Morton code of the k-th point can be expressed as 3L bits, which can be shown in the following equation (5):

[0097]

[0098] The corresponding properties of the points in the Morton order can be expressed as A k , k=0,1,…,N-1.

[0099] In the second step of Haar-based attribute encoding, the Haar transform can be applied to the attributes of a pair of consecutive points of the Morton order in equation (6) as follows:

[0100]

[0101] in, represents the largest integer less than or equal to x (x>0), and the Haar transform matrix T can be expressed in the following equation (7):

[0102]

[0103] in, are the Haar transform coefficients. Is the scaling factor. If N is an odd number, then the last point A N1 There is no point to pair with it. The last point is A N1 The corresponding Haar transform coefficient can be expressed in the following equation (8):

[0104]

[0105] High-frequency components Can be grouped to form part of the final Haar transform coefficients. For the low frequency component H 2n (in ) and H N1 , if N is an odd number, another round of Haar transform can be applied to consecutive component pairs until only one component remains. Further, all high frequency components and the last component can be grouped to form the final Haar transform coefficients, which can be expressed as Where n=0, 1,…, N-1.

[0106] In a third step of the Haar-based property encoding, the transform coefficients may be quantized and the quantized transform coefficients may be further encoded.

[0107] In a fourth step of the Haar-based attribute encoding, an inverse Haar transform may be applied to the quantized transform coefficients to obtain a reconstructed attribute. A residual may be calculated by subtracting the reconstructed attribute from the original attribute.

[0108] In the final step of the Haar-based attribute encoding, the attribute residual may be quantized and the quantized attribute residual may be further encoded.

[0109] It should be noted that the quantization step size of the residual given the attribute is Q 集合 , the quantization step size Q of the transform coefficients 变换 It can be given in the following equation (9):

[0110]

[0111] In addition, the encoding method for the attribute residual and the quantized transform coefficients can be the same. For example, given a value x, the encoding method can be described in steps (a)-(e) as follows:

[0112] (a) Encode a flag indicating whether x is equal to 0 (when x is equal to 0, flag = 1;

[0113] When x is not equal to 0, flag = 0);

[0114] (b) If x is not equal to 0, encode the sign of x (when x is less than or equal to 0, sign = 0;

[0115] When x is greater than 0, sign = 1);

[0116] (c) Encode a flag indicating whether the absolute value of x is equal to 1 (when |x| is equal to 1, flag = 1;

[0117] When |x|>1, flag=0, where |x| represents the absolute value of x);

[0118] (d) if |x|>1, encode a flag indicating whether the absolute value of x is equal to 2 (flag=1 when |x| is equal to 2; flag=0 when |x|>2); and,

[0119] (e) If |x|>2, encode |x|-3 using the 0th order exponential Golomb code.

[0120] There may be some problems with the Haar-based attribute coding used in AVS-PCC. For example, in Haar-based attribute coding, the attributes of the entire point cloud are used to construct multiple layers of Haar transform. For a point cloud with N points, the Haar transform must be applied to layer to obtain all final transform coefficients, where is the smallest integer not less than m=n2(N). Accordingly, if N is large, it will cause a large delay. In addition, the order of points, quantization scheme, encoding of quantized transform coefficients, and quantized attribute residuals can also be improved.

[0121] The present disclosure includes methods for improving Haar-based attribute coding.

[0122] To improve point sorting, in an embodiment, instead of using Morton order sorting (also referred to as Morton sorting or Morton order), a different sorting order, such as Hilbert sorting (or Hilbert order), may be used to sort multiple points of a point cloud before Haar-based attribute encoding. Since points sorted by Hilbert order can generally have a higher correlation with neighboring points, attribute encoding performance gains can be achieved.

[0123] In an embodiment, a sorting method flag such as sort_method may be used to indicate which sorting method to use in Haar-based attribute encoding. The sorting method flag may be carried in a high-level syntax, such as a sequence parameter set (SPS), an attribute parameter set (APS), a slice header, etc. As an example, the sorting method flag may be carried in an APS, as shown in Table 1.

[0124] Table 1: Sorting method flags carried in the Attribute Parameter Set (APS)

[0125]

[0126] As shown in Table 1, a sorting method flag (e.g., aps_sort_method) can specify a sorting method before Haar-based attribute encoding. When the sorting method flag (e.g., aps_sort_method) is equal to a first value (e.g., 0), it indicates that Morton sorting is used. When the sorting method flag (e.g., aps_sort_method) is equal to a second value (e.g., 1), it indicates that another sorting method such as Hilbert sorting is used. Note that an increasing sorting order is typically used. However, a decreasing sorting order may also be used. In addition, the sorting method is not limited to Morton order and Hilbert order, and in other embodiments, one or more different sorting methods may be used. In another embodiment, the Hilbert sorting order is always used, and there is no need to add the aps_sort_method flag in the attribute parameter set.

[0127] In an embodiment, instead of determining the quantization parameters of the transform coefficients based on the quantization parameters of the attribute residuals, the quantization parameters may be specified in a high-level syntax (e.g., SPS, APS, slice header, etc.). Accordingly, different quantization parameters of the transform coefficients may be set based on different applications of the point cloud (e.g., object detection and localization, mapping, and visualization and archiving of cultural heritage objects) or different characteristics (e.g., geometric shape information, color, and reflectivity). As an example, the quantization parameters may be carried in the APS, which may be shown in Table 2.

[0128] Table 2: Syntax showing the quantization parameters carried in APS

[0129]

[0130] As shown in Table 2, the quantization parameter flag (e.g., aps_tran_coeff_qp) can specify the quantization parameter used to quantize the transform coefficients. At the encoder side, the quantization parameter can be set based on the quantization parameter of the attribute residual or based on other criteria, such as the application of the point cloud (e.g., object detection and localization, mapping and / or visualization and archiving of cultural heritage objects) or characteristics (e.g., geometric shape information, color and / or reflectivity), etc.). Further, the determined quantization parameter can be signaled to the decoder side through the quantization parameter flag (e.g., aps_tran_coeff_qp). In another embodiment, instead of sending aps_trans_coeff_qp directly in the bitstream, the difference between aps_trans_coeff_qp and the qp used to encode the attribute residual (expressed as aps_trans_qp_delta) can be carried in the bitstream.

[0131] It can be observed that the characteristics of the quantized transform coefficients depend not only on the characteristics of the original (unquantized) transform coefficients, but also on the quantization parameter. When the quantization parameter is large, the quantized transform coefficients can be more concentrated around the zero value, and the range of the quantized transform coefficients can be smaller. However, with a smaller quantization parameter, the quantized transform coefficients tend to spread more over a larger range. Therefore, when the quantization parameter is different, different encoding methods can be used.

[0132] In applications, an exponential Golomb code may be applied to entropy encode the quantized transform coefficients. In one embodiment, the order of the exponential Golomb code may be specified in a high-level syntax such as an SPS, APS, or slice header, along with the quantization parameters of the transform coefficients. As an example, the order of the exponential Golomb code may be specified in an APS, as shown in Table 3.

[0133] Table 3: Syntax showing the order of the Exponential Golomb code in APS

[0134]

[0135] In Table 3, as described above, the quantization parameter flag aps_tran_coeff_qp can specify the quantization parameter used to quantize the transform coefficients. When encoding the absolute value of the quantized transform coefficient minus the threshold, the encoding flag (e.g., aps_trans_coeff_eg_order) can specify the order of the exponential Golomb code. The threshold can be 2 or other values ​​(e.g., 1 or 3), depending on the overall quantization transform coding scheme.

[0136] Once the order k of the EG code is determined by the coding flag (e.g., aps_trans_coeff_eg_order), the entropy coding process of encoding a non-negative integer x using the k-order EG code can be performed as follows: (a) First, use the 0-order EG code to encode encode; and then (b) encode in binary form (x and 2 k ), where x can be a quantized transform coefficient.

[0137] In another embodiment, the order k of the exponential Golomb code can be implicitly determined based on the quantization parameter of the transform coefficients by the principle that a smaller quantization parameter requires a relatively larger order. In one embodiment, the order k can be specified in the following equation (10):

[0138] Where max_k is the maximum order of the exponential Golomb code used to encode the quantized transform coefficients, trans_coeff_qp is the value of the quantization parameter used in the quantization of the transform coefficients, and s is a positive scalar and is typically set to 6 or 8, etc. In some embodiments, when the value of the quantization parameter is equal to zero, lossless coding may be indicated. As the value of the quantization parameter increases, the quantization step size increases accordingly. In some embodiments, the value of max_k may be set to 2 or 3.

[0139] In AVS-PCC, as mentioned above, for a point cloud with N points, it is necessary to apply the Haar transform (or Haar-based transform) layers to obtain all final transform coefficients, which may cause large delay if N is large.

[0140] In one embodiment, a point cloud having N points may be selectively sorted first. For example, the point cloud may be selectively sorted using the Morton order or the Hilbert order as described above. The sorting order is not limited to the Morton order and the Hilbert order, and one or more different sorting orders may be used in other embodiments. The sorted N points may be further partitioned into a plurality of segments. Each segment in the plurality of segments may contain M points, except that the last segment may contain less than M points. The number of segments may be expressed as This partition can be, for example, Figure 10 Shown in.

[0141] For each segment, multiple layers of Haar transforms can be applied. Because the segments are uncorrelated, they can be encoded / decoded in parallel. Parallel encoding / decoding can help reduce encoding / decoding latency. Additionally, if each segment is encoded / decoded serially, memory requirements can be significantly reduced.

[0142] In AVS-PCC, due to the scaling factor or In the transformation matrix T shown in equation (7), fixed point operations are used in both the forward Haar-based transformation and the inverse Haar-based transformation.

[0143] In an embodiment, the forward Haar-based transform and the inverse Haar-based transform may be the same, as shown in equation (1). In another embodiment, the scaling factor in the transform matrix of the forward Haar-based transform may be one, as shown in equation (11). The scaling factor in the transform matrix of the inverse Haar-based transform may be 1 / 2, as shown in equation (12).

[0144]

[0145]

[0146] If N is an odd number, then the last point A N1 There is no point to pair with it. The last point is A N1 The corresponding transformation coefficient of can be expressed in the following equation (13):

[0147] H N1 =2×A N1 Equation (13)

[0148] Therefore, during the inverse Haar-based transform, if N is an odd number, the transform coefficient of the last point can be divided by 2. Therefore, only integer operations are required, which can reduce the complexity of the transform and inverse transform.

[0149] It should be noted that the present disclosure is not limited to TMC13 software, MPEG-PCC, or AVS-PCC standards. The present disclosure provides a general solution for most PCC systems. In addition, the methods and embodiments of the present disclosure can be used alone or in combination in any order. Furthermore, each of the methods (or embodiments) can be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors can execute a program stored in a non-volatile computer-readable medium.

[0150] Figure 11 A flowchart outlining a process (1100) according to an embodiment of the present disclosure is shown. The process (1100) can be used in a decoding process of a point cloud. In various embodiments, the process (1100) can be performed by a processing circuit, for example, a processing circuit in a terminal device (110), a processing circuit that performs the functions of an encoder (203) and / or a decoder (210), a processing circuit that performs the functions of an encoder (300), a decoder (400), an encoder (500), and / or a decoder (600), etc. In some embodiments, the process (1100) can be implemented using software instructions, so that when the processing circuit executes the software instructions, the processing circuit performs the process (1100).

[0151] like Figure 11 As shown in , the process (1100) starts at (S1101) and proceeds to (S1110).

[0152] At (S1110), a sorting method of a point set in a point cloud may be determined based on first signaling information in an encoded bitstream of the point cloud.

[0153] At (S1120), a quantization parameter for quantizing transform coefficients for Haar-based attribute encoding of the point cloud may be determined based on second signaling information in an encoded bitstream of the point cloud.

[0154] At (S1130), an entropy decoding method for decoding the quantized transform coefficient may be determined according to the quantization parameter determined based on the second signaling information.

[0155] At (S1140), quantized transform coefficients may be further determined based on an entropy decoding method.

[0156] At (S1150), attributes of the point cloud may be reconstructed based on the determined sorting method and the determined quantized transform coefficients.

[0157] In an example, the sorting method may be determined to be Morton sorting based on the first signaling information being equal to a first value. In another example, the sorting method may be determined to be Hilbert sorting based on the first signaling information being equal to a second value.

[0158] In some embodiments, the second signaling information may be set based on one of a quantization parameter, an application of the point cloud, and a characteristic of the point cloud.

[0159] In the process (1100), when the entropy decoding method is exponential Golomb coding, an order k of the exponential Golomb coding may be determined based on third signaling information in the encoded bitstream, wherein the third signaling information indicates the order k of the exponential Golomb coding.

[0160] In the process (1100), when the entropy decoding method is exponential Golomb coding, the order k of the exponential Golomb coding can be determined based on the quantization parameter. max_k is the maximum order of Exponential Golomb coding used to decode the quantized transform coefficients, trans_coeff_qp is the value of the quantization parameter, and s is a positive integer equal to six or eight.

[0161] In some embodiments, to determine the quantized transform coefficients based on an entropy decoding method, the quantized transform coefficients minus the absolute value of a threshold value may be decoded based on an exponential Golomb coding and a determined exponential Golomb coding order k. The threshold value may be equal to one of 1, 2, and 3.

[0162] In some embodiments, in order to determine the quantized transform coefficients based on the entropy decoding method, the first quantized transform coefficient x among the quantized transform coefficients may be decoded. The first quantized transform coefficient x may be encoded by exponential Golomb coding, such that the first part is encoded by exponential Golomb coding with order k equal to zero. Encode and encode the second part (x and 2 k The modular operation of ) is encoded.

[0163] In process (1100), a set of points in a point cloud may be sorted based on a determined sorting method. The sorted set of points in the point cloud may be further partitioned into a plurality of segments based on a partition size indicated by partition signaling information in the encoded bitstream. Each of the plurality of segments of the point cloud may be reconstructed based on the determined quantized transform coefficients.

[0164] In some embodiments, the number of multiple segments may be equal to Where N can be the number of point sets in the point cloud, and M can be a positive integer.

[0165] In process 1100, to reconstruct a point cloud, a scaling factor of 1 / 2 of a Haar transform matrix associated with Haar-based attribute coding may be determined. An inverse quantization operation may be performed on the quantized transform coefficients to obtain transform coefficients according to the Haar-based attribute coding. An inverse transform operation may be performed on the point cloud based on the Haar transform matrix and the transform coefficients to obtain attribute residuals. Attributes of the point cloud may be further reconstructed using the attribute residuals and predicted attributes of the point cloud obtained by the Haar-based attribute coding.

[0166] As mentioned above, the above techniques can be implemented as computer software via computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 12 A computer system (1800) is shown that is suitable for implementing certain embodiments of the disclosed subject matter.

[0167] The computer software may be encoded in any suitable machine code or computer language, and may be assembled, compiled, linked, or other mechanisms to create code comprising instructions, which may be directly executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), or the like, or executed through decoding, microcode, or the like.

[0168] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablets, servers, smartphones, gaming devices, IoT devices, and the like.

[0169] Figure 12 The components shown for the computer system (1800) are exemplary in nature and are not intended to limit the scope of use or functionality of computer software implementing embodiments of the present application. Nor should the configuration of the components be interpreted as having any dependency or requirement on any one or combination of components shown in the exemplary embodiment of the computer system (1800).

[0170] The computer system (1800) may include certain human-computer interface input devices. Such human-computer interface input devices may respond to input from one or more human users through tactile input (e.g., keyboard input, swiping, data glove movement), audio input (e.g., sound, applause), visual input (e.g., gestures), and olfactory input (not shown). The human-computer interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sound), images (e.g., scanned images, photographic images obtained from a still camera), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0171] The human-computer interface input device may include one or more of the following (only one of which is shown): keyboard (1801), mouse (1802), touchpad (1803), touch screen (1810), data gloves (not shown), joystick (1805), microphone (1806), scanner (1807), camera (1808).

[0172] The computer system (1800) may also include certain human-computer interface output devices. Such human-computer interface output devices may stimulate one or more senses of a human user through, for example, tactile output, sound, light, and smell / taste. Such human-computer interface output devices may include tactile output devices (e.g., tactile feedback through a touch screen (1810), a data glove (not shown), or a joystick (1805), but there may also be tactile feedback devices that are not used as input devices), audio output devices (e.g., speakers (1809), headphones (not shown)), visual output devices (e.g., screens (1810) including cathode ray tube screens, liquid crystal screens, plasma screens, organic light emitting diode screens, each with or without touch screen input capabilities, each with or without tactile feedback capabilities - some of which may output two-dimensional visual output or output in more than three dimensions through means such as stereoscopic image output; virtual reality glasses (not shown), holographic displays, and cigarette boxes (not shown)), and printers (not shown).

[0173] The computer system (1800) may also include human-accessible storage devices and their associated media, such as optical media including high-density read-only / rewritable optical discs (CD / DVD ROM / RW) (1820) with CD / DVD or similar media (1821), thumb drives (1822), removable hard drives or solid-state drives (1823), traditional magnetic media such as tapes and floppy disks (not shown), special-purpose ROM / ASIC / PLD-based devices such as security software dongles (not shown), and the like.

[0174] Those skilled in the art will also understand that the term "computer-readable media" used in connection with the disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0175] The computer system (1800) may also include an interface to one or more communication networks. For example, the network may be wireless, wired, or optical. The network may also be a local area network, a wide area network, a metropolitan area network, an in-vehicle network, an industrial network, a real-time network, a delay-tolerant network, and the like. Networks also include local area networks such as Ethernet, wireless local area networks, cellular networks (GSM, 3G, 4G, 5G, LTE, etc.), television wired or wireless wide area digital networks (including cable television, satellite television, and terrestrial broadcast television), in-vehicle and industrial networks (including CANBus), and the like. Some networks typically require an external network interface adapter for connecting to some universal data port or peripheral bus (1849) (for example, a USB port of the computer system (1800)); other systems are typically integrated into the core of the computer system (1800) by connecting to a system bus as described below (for example, an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smartphone computer system). By using any of these networks, the computer system (1800) can communicate with other entities. The communication can be one-way, for receiving only (e.g., wireless television), one-way, for sending only (e.g., a CAN bus to certain CAN bus devices), or two-way, such as to other computer systems via a local or wide area digital network. Each of the above networks and network interfaces can use certain protocols and protocol stacks.

[0176] The aforementioned human-machine interface devices, human-accessible storage devices, and network interfaces may be connected to the core (1840) of the computer system (1800).

[0177] The core (1840) may include one or more central processing units (CPUs) (1841), graphics processing units (GPUs) (1842), specialized programmable processing units in the form of field programmable gate arrays (FPGAs) (1843), hardware accelerators (1844) for specific tasks, and the like. These devices, as well as read-only memory (ROM) (1845), random access memory (1846), internal mass storage (e.g., internal non-user accessible hard disk drives, solid-state drives, etc.) (1847), and the like, may be connected via a system bus (1848). In some computer systems, the system bus (1848) may be accessed in the form of one or more physical plugs to allow expansion with additional central processing units, graphics processing units, and the like. Peripheral devices may be attached directly to the core's system bus (1848) or connected via a peripheral bus (1849). Peripheral bus architectures include PCI (Peripheral Controller Interface), USB (Universal Serial Bus), and the like.

[0178] The CPU (1841), GPU (1842), FPGA (1843), and accelerator (1244) can execute certain instructions, which, when combined, can constitute the aforementioned computer code. The computer code can be stored in ROM (1845) or RAM (1846). Transient data can also be stored in RAM (1846), while permanent data can be stored, for example, in internal mass storage (1847). Fast storage and retrieval of any memory device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (1841), GPUs (1842), mass storage (1847), ROM (1845), RAM (1846), etc.

[0179] The computer readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be specially designed and constructed for the purposes of this application, or may be medium and code well known and available to those skilled in the art of computer software.

[0180] As an example and not a limitation, a computer system having architecture (1800), in particular a core (1840), can provide functionality as a processor (including a CPU, GPU, FPGA, accelerator, etc.) to execute software contained in one or more tangible computer-readable media. Such computer-readable media can be media associated with the aforementioned user-accessible mass storage, as well as specific memory of the core (1840) having non-volatile properties, such as a core internal mass storage (1847) or ROM (1845). Software for implementing various embodiments of the present application can be stored in such a device and executed by the core (1840). Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software can enable the core (1840), in particular the processor therein (including a CPU, GPU, FPGA, etc.), to perform a specific process or a specific portion of a specific process described herein, including defining a data structure stored in RAM (1846) and modifying such a data structure according to a software-defined process. Additionally or alternatively, the computer system may provide functionality hardwired in logic or otherwise contained in circuitry (e.g., accelerator (1844)) that may operate in place of or in conjunction with software to perform a particular process or a particular portion of a particular process described herein. Where appropriate, references to software may include logic and vice versa. Where appropriate, references to computer-readable media may include circuitry (e.g., an integrated circuit (IC)) storing the executing software, circuitry containing the executing logic, or both. The present application includes any suitable combination of hardware and software.

[0181] Although this application has described a number of exemplary embodiments, various modifications, permutations, and equivalent substitutions of the embodiments are within the scope of this application. Therefore, it should be understood that those skilled in the art will be able to design a variety of systems and methods that, although not explicitly shown or described herein, embody the principles of this application and are therefore within the spirit and scope of this application.

Claims

1. A point cloud data processing method, characterized in that: include: Determine the sorting method of the point set of the point cloud; determining, based on first signaling information in an encoded bitstream of the point cloud, a quantization parameter for quantizing transform coefficients of a Haar-based attribute encoding of the point cloud, the Haar-based attribute encoding being configured to perform point cloud compression on the set of points of the point cloud based on a forward Haar transform matrix associated with the encoded point cloud; determining, according to the quantization parameter determined based on the first signaling information, an entropy decoding method for decoding the quantized transform coefficients; Determining the quantized transform coefficients based on the entropy decoding method, and performing an inverse quantization operation on the quantized transform coefficients according to the Haar-based attribute coding to obtain transform coefficients; performing an inverse transform operation on the point cloud based on an inverse Haar transform matrix associated with the decoded point cloud and the transform coefficients to generate an attribute residual, wherein the inverse Haar transform matrix is ​​equal to the product of a factor ½ and the forward Haar transform matrix; as well as Based on the determined sorting method and the attribute residuals, attributes of the point cloud are reconstructed.

2. The method according to claim 1, characterized in that The method for determining the ranking further comprises: Determining a sorting method for the set of points of the point cloud based on second signaling information in the encoded bitstream of the point cloud, wherein: When the second signaling information is equal to the first value, the sorting method is Morton sorting; and When the second signaling information is equal to a second value, the sorting method is Hilbert sorting.

3. The method according to claim 1, characterized in that The first signaling information is set based on one of the quantization parameter, an application of the point cloud, and a characteristic of the point cloud.

4. The method according to claim 1, wherein The determining the entropy decoding method further comprises: When the entropy decoding method is exponential Golomb coding, an order k of the exponential Golomb coding is determined based on third signaling information in the encoded bit stream, and the third signaling information indicates the order k of the exponential Golomb coding.

5. The method according to claim 1, characterized in that The determining the entropy decoding method further comprises: When the entropy decoding method is exponential Golomb coding, the order k of the exponential Golomb coding is determined based on the quantization parameter. , wherein the max_k is the maximum order of the exponential Golomb coding used to decode the quantized transform coefficients, the trans_coeff_qp is the value of the quantization parameter, and the s is a positive integer equal to six or eight.

6. The method according to claim 4, characterized in that The determining the quantized transform coefficients further comprises: Based on the Exponential Golomb coding and the determined order k of the Exponential Golomb coding, the quantized transform coefficient minus an absolute value of a threshold is decoded, the threshold being equal to one of 1, 2, and 3.

7. The method according to claim 4, characterized in that The determining the quantized transform coefficients further comprises: Decoding a first quantized transform coefficient x among the quantized transform coefficients, wherein the first quantized transform coefficient x is encoded by the exponential Golomb coding, so that the first part is encoded by the exponential Golomb coding with the order k equal to zero. and encode the second part in binary form, wherein the second part is x and Modulo operation.

8. The method according to claim 2, characterized in that Further including: sorting the set of points in the point cloud based on the determined sorting method; partitioning the ordered set of points of the point cloud into a plurality of segments based on a partition size indicated by partition signaling information in the coded bitstream; and Each of a plurality of segments in the point cloud is reconstructed based on the determined quantized transform coefficients.

9. The method according to claim 8, characterized in that The number of the plurality of segments is equal to , N is the number of point sets in the point cloud, and M is a positive integer.

10. The method according to any one of claims 1 to 9, characterized in that The reconstruction further comprises: Attributes of the point cloud are reconstructed according to the attribute residuals and predicted attributes of the point cloud obtained by the Haar-based attribute encoding.

11. A device for processing point cloud data, characterized in that: Includes processing circuitry for: Determine the sorting method of the point set of the point cloud; determining, based on first signaling information in an encoded bitstream of the point cloud, a quantization parameter for quantizing transform coefficients of a Haar-based attribute encoding of the point cloud, the Haar-based attribute encoding being configured to perform point cloud compression on the set of points of the point cloud based on a forward Haar transform matrix associated with the encoded point cloud; determining, according to the quantization parameter determined based on the first signaling information, an entropy decoding method for decoding the quantized transform coefficients; Determining the quantized transform coefficients based on the entropy decoding method, and performing an inverse quantization operation on the quantized transform coefficients according to the Haar-based attribute coding to obtain transform coefficients; performing an inverse transform operation on the point cloud based on an inverse Haar transform matrix associated with the decoded point cloud and the transform coefficients to generate an attribute residual, wherein the inverse Haar transform matrix is ​​equal to the product of a factor ½ and the forward Haar transform matrix; as well as Based on the determined sorting method and the attribute residuals, attributes of the point cloud are reconstructed.

12. The device according to claim 11, characterized in that The processing circuit is further configured to: Determining a sorting method for the set of points of the point cloud based on second signaling information in the encoded bitstream of the point cloud, wherein: When the second signaling information is equal to the first value, the sorting method is Morton sorting; and When the second signaling information is equal to a second value, the sorting method is Hilbert sorting.

13. The device according to claim 11, characterized in that The first signaling information is determined based on one of the quantization parameter, an application of the point cloud, and a characteristic of the point cloud.

14. The device according to claim 11, characterized in that The processing circuit is further configured to: When the entropy decoding method is exponential Golomb coding, an order k of the exponential Golomb coding is determined based on third signaling information in the encoded bit stream, and the third signaling information indicates the order k of the exponential Golomb coding.

15. The device according to claim 11, characterized in that The processing circuit is further configured to: When the entropy decoding method is exponential Golomb coding, the order k of the exponential Golomb coding is determined based on the quantization parameter. , wherein the max_k is the maximum order of the exponential Golomb coding used to decode the quantized transform coefficients, the trans_coeff_qp is the value of the quantization parameter, and the s is a positive integer equal to six or eight.

16. The device according to claim 14, characterized in that The processing circuit is further configured to: Based on the Exponential Golomb coding and the determined order k of the Exponential Golomb coding, the quantized transform coefficient minus an absolute value of a threshold is decoded, the threshold being equal to one of 1, 2, and 3.

17. The device according to claim 14, characterized in that The processing circuit is further configured to: Decoding a first quantized transform coefficient x among the quantized transform coefficients, wherein the first quantized transform coefficient x is encoded by the exponential Golomb coding, so that the first part is encoded by the exponential Golomb coding with the order k equal to zero. and encode the second part in binary form, wherein the second part is x and Modulo operation.

18. The device according to any one of claims 12 to 17, characterized in that The processing circuit is further configured to: sorting the set of points in the point cloud based on the determined sorting method; partitioning the ordered set of points of the point cloud into a plurality of segments based on a partition size indicated by partition signaling information in the coded bitstream; and Each of a plurality of segments in the point cloud is reconstructed based on the determined quantized transform coefficients.

19. A device for processing point cloud data, characterized in that: include: one or more computer-readable non-volatile storage media for storing computer program code; as well as, One or more computer processors, configured to access the computer program code and operate according to instructions of the computer program code to execute the method according to any one of claims 1 to 10.

20. A non-volatile computer-readable medium storing instructions, characterized in that: When at least one processor executes the instructions to process point cloud data, the at least one processor executes the method according to any one of claims 1 to 10.

Images