Method and apparatus for point cloud coefficient coding

Abstract

Methods, apparatus, and computer readable media for point cloud coefficient encoding are provided. Transform coefficients associated with the point cloud data are divided into one or more sets based on the set-index value and the symbol-index value. Entropy encoding is performed on the set-index values of the divided transform coefficients, the entropy encoding comprises aggregating the set-index values to form a super symbol, and bypass encoding is performed on the symbol-index values of the divided transform coefficients. The point cloud data is compressed based on the entropy encoded symbol-index values and the bypass encoded set-index values.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application No. 62 / 958,839 filed in the U.S. Patent and Trademark Office on January 9, 2020, and U.S. Patent Application No. 17 / 110,691 filed in the U.S. Patent and Trademark Office on December 3, 2020, both of which are incorporated herein by reference in their entirety. Technical Field

[0002] The present application relates to graph-based point cloud compression (G-PCC), and more particularly, to methods and apparatus for point cloud coefficient encoding. Background Art

[0003] Advanced three-dimensional (3D) representations of the world enable more immersive interactions and communications, and also enable machines to understand, interpret, and navigate our world. 3D point clouds emerge as an enabling representation of such information. Many use cases associated with point cloud data have been identified, and corresponding requirements for point cloud representation and compression have been developed. For example, point clouds can be used for object detection and localization in autonomous driving. Point clouds can also be used for mapping in geographic information systems (GIS), and in cultural heritage for visualizing and archiving cultural heritage objects and collections.

[0004] A point cloud is a collection of points in 3D space, each with associated attributes such as color and material properties. Point clouds can be used to reconstruct an object or scene as a collection of these points. These points can be captured using multiple cameras, depth sensors, or lidar sensors in various setups, and can consist of thousands to billions of points to realistically represent the reconstructed scene.

[0005] Compression techniques are needed to reduce the amount of data used to represent point clouds. Therefore, techniques for lossy compression of point clouds are needed for use in real-time communication and six degrees of freedom (6DoF) virtual reality. In addition, techniques for lossless point cloud compression are sought in the context of dynamic mapping for autonomous driving and cultural heritage applications. The Moving Picture Experts Group (MPEG) has begun work on standards for compression that addresses geometric structure and attributes such as color and reflectivity, scalable / progressive coding, coding of sequences of point clouds captured over time, and random access to subsets of point clouds. However, current point cloud data encoding methods are still unsatisfactory in terms of complexity / memory and compression efficiency. Summary of the Invention

[0006] The present application provides a method and apparatus for encoding point cloud coefficients, which can improve the encoding of point cloud coefficients.

[0007] According to an embodiment, a method for encoding point cloud coefficients is performed by at least one processor and includes partitioning transform coefficients associated with point cloud data into one or more sets based on set-index values ​​and symbol-index values. The set-index values ​​of the partitioned transform coefficients may be entropy encoded, the entropy encoding including aggregating the set-index values ​​to form super-symbols, and the symbol-index values ​​of the partitioned transform coefficients may be bypass encoded. The point cloud data may be compressed based on the entropy-encoded set-index values ​​and the bypass-encoded symbol-index values.

[0008] According to an embodiment, an apparatus for point cloud coefficient encoding includes: at least one memory configured to store computer program code, and a processor configured to access the at least one memory and perform the above-mentioned point cloud coefficient encoding method according to the computer program code.

[0009] According to an embodiment, a non-transitory computer-readable storage medium stores instructions that cause at least one processor to perform the above-mentioned point cloud coefficient encoding method.

[0010] In embodiments of the present application, transform coefficients associated with point cloud data are decomposed into set-index values ​​and sign-index values, with the sign-index value specifying the position of the transform coefficient within a set. The decomposed transform coefficients can be partitioned into one or more sets based on the set-index values ​​and the sign-index values. The set-index values ​​of the partitioned transform coefficients can be entropy encoded, and the sign-index values ​​of the partitioned transform coefficients can be bypass encoded. Point cloud data can be compressed based on the entropy-encoded set-index values ​​and the bypass-encoded sign-index values. In this manner, encoding of transform coefficients from lifting, predictive transform, and RAHT can be performed via frequency-sorted lookup table index encoding, cache index encoding, and direct encoding of the sign values. In practice, this may require multiple lookup tables and caches with many (typically 32 to potentially as many as 256) entries to cover a single-byte codeword. These lookup tables and caches may also need to be updated regularly, the frequency of which may imply different trade-offs in terms of computational requirements and coding efficiency. Therefore, it may be advantageous to improve the encoding of transform coefficients of attributes in G-PCC from the complexity / memory and compression efficiency trade-off aspects through alphabet-partitioning and encoding of alphabet-partitioning information. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1A It is a diagram showing a method of generating LoD in G-PCC.

[0012] Figure 1B This is a diagram of the architecture used for P / U improvement in G-PCC.

[0013] Figure 2is a block diagram of a communication system according to an embodiment.

[0014] Figure 3 is a diagram of placement of a G-PCC compressor and a G-PCC decompressor in an environment according to an embodiment.

[0015] Figure 4 is a functional block diagram of a G-PCC compressor according to an embodiment.

[0016] Figure 5 is a functional block diagram of a G-PCC decompressor according to an embodiment.

[0017] Figure 6 is a flowchart illustrating a method of encoding point cloud coefficients according to an embodiment.

[0018] Figure 7 is a block diagram of an apparatus for point cloud coefficient encoding according to an embodiment.

[0019] Figure 8 is a diagram of a computer system suitable for implementing embodiments. DETAILED DESCRIPTION

[0020] Figure 1A is a diagram illustrating a method for generating level of detail (LoD) in geometry-based point cloud compression (G-PCC).

[0021] Reference Figure 1A In the current G-PCC attribute coding, a LoD (i.e., group) is generated for each 3D point (e.g., P0 to P9) based on the distance of each 3D point, and then the attribute values ​​of the 3D points in each LoD are encoded by applying prediction in the LoD-based order 110 of the 3D points instead of the original order 105 of the 3D points. For example, the attribute value of 3D point P2 is predicted by calculating the distance-based weighted average of 3D points P0, P5, and P4 that were encoded or decoded before 3D point P2.

[0022] The current anchoring method in G-PCC proceeds as follows.

[0023] First, the variability of the neighborhood of a 3D point is calculated to check how different the neighboring values ​​are, and if the variability is below a threshold, the predicted attribute value (a i ) i∈0...k-1 , a linear interpolation process based on the distance of the nearest neighbor of the current point i is used to calculate the distance-based weighted average prediction. Let is the set of k nearest neighbors of the current point i, let Its decoded / reconstructed attribute value, and the is the distance between it and the current point i. Then, the predicted attribute value It is given by:

[0024]

[0025] Note that when encoding attributes, the geometric positions of all point clouds are already available. In addition, both the neighboring points and their reconstructed attribute values ​​are available at both the encoder and decoder as a k-dimensional tree structure, which is used to facilitate the nearest neighbor search for each point in the same way.

[0026] Secondly, if the variability is higher than the threshold, rate-distortion optimization (RDO) prediction value selection is performed. Based on the results of the neighboring point search when generating the LoD, multiple prediction value candidates or candidate prediction values ​​are created. For example, when encoding the attribute value of the 3D point P2 by using the prediction, the weighted average of the distances from the 3D point P2 to the 3D points P0, P5 and P4 respectively is set to a prediction value index equal to 0. Then, the distance from the 3D point P2 to the nearest neighbor point P4 is set to a prediction value index equal to 1. In addition, as shown in Table 1 below, the distance from the 3D point P2 to the next nearest neighbor points P5 and P0 is set to a prediction value index equal to 2 and 3 respectively. Table 1: Sample of predicted value candidates for attribute encoding Prediction value index Predicted value 0 average value 1 P4 (first closest point) 2 P5 (second closest point) 3 P0 (third closest point)

[0027] After creating the predictor candidates, the best predictor is selected by applying a rate-distortion optimization process. The selected predictor index is then mapped to a truncated unary (TU) code, and the binary values ​​(bins) of the truncated unary (TU) code are arithmetic coded. Note that shorter TU codes are assigned to smaller predictor indices in Table 1.

[0028] The maximum number of prediction candidates, MaxNumCand, is limited and encoded in the attribute header. In the current implementation, the maximum number of prediction candidates, MaxNumCand, is set equal to the number of nearest neighbors in the prediction + 1 (numberOfNearestNeighborsInPrediction+1), and is used to encode and decode the prediction index via truncated unary binarization.

[0029] The lifting transform used for attribute coding in G-PCC is built on the prediction transform mentioned above. The main difference between the prediction scheme and the lifting scheme is the introduction of the update operator.

[0030] Figure 1BFigure 1 is a diagram of the architecture used for P / U (prediction / update) boosting in G-PCC. In order to facilitate the prediction step and the update step in boosting, the signal must be split into two sets with high correlation at the decomposition of each stage. In the boosting scheme in G-PCC, the segmentation is performed by utilizing the LoD structure, in which such high correlation between levels is expected, and each level is constructed by nearest neighbor search to organize the non-uniform point cloud into structured data. The P / U decomposition step at level N produces a detail signal D(N-1) and an approximation signal A(N-1), which are further decomposed into D(N-2) and A(N-2). This step is repeatedly applied until the base layer approximation signal A(1) is obtained.

[0031] Thus, in the lifting scheme, instead of encoding the input attribute signal itself consisting of LOD(N), ..., LOD(1), D(N-1), D(N-2), ..., D(1), A(1) is ultimately encoded. Note that the application of an efficient P / U step typically results in sparse subband "coefficients" in D(N-1), ..., D(1), thus providing a transform coding gain advantage.

[0032] Currently, the distance-based weighted average prediction for predicting transformations described above is used as the anchoring method in G-PCC for the prediction step during lifting.

[0033] In the prediction and boosting for attribute coding in G-PCC, the availability of neighboring attribute samples is crucial for compression efficiency, as more neighboring attribute samples can provide better predictions. In the absence of enough neighboring attribute samples to make predictions from, compression efficiency may suffer.

[0034] Another type of transform used for attribute coding in G-PCC can be a Region Adaptive Hierarchial Transform (RAHT). RAHT and its inverse can be performed on a hierarchy defined by the Morton code of the voxel position. The Morton code of the d-bit non-negative integer coordinates x, y, and z can be a 3d-bit non-negative integer that can be obtained by interleaving the bits of x, y, and z. Non-negative d-bit integer coordinates (in, It can be from (Advanced) to The Morton code M = morton(x, y, z) of the (low-order) bits of x, y, and z is a non-negative 3d-bit integer in, It can be from (Advanced) to (low-order) bits of M.

[0035] Can represent M Bit prefix. m can be such a prefix. Level The block at can be defined with prefix m as the set of all points (x, y, z) where If the level The two blocks at have the same bit prefix, they can be blocks of the same level. The union of two blocks of the same level at can be a hierarchy The block at which it is located is called its parent block.

[0036] Sequence A n , n=1, ..., N and its inverse region adaptive Haar transform can include base case and recursive function. n can be a property of the point, and T n can be its transformation, where T n =A n For a recursive function, there can be two sibling blocks and its parent block. and Can be points in the same block listed in increasing Morton order (x n ,y n , z n ) attributes, and and can be their respective transformations. Similarly, can be all points in its parent block listed in increasing Morton order (x n ,y n , x n ) attributes, and can be its transformation. Then, as well as in,

[0037] The transform of the parent block can be the concatenation of the two sibling blocks, except that the first (DC) components of the transforms of the two sibling blocks can be replaced by their weighted sum and difference, and the inverse transforms of the two sibling blocks can be copied from the first and last parts of the transform of the parent block, except that the DC components of the transforms of the two sibling blocks can be replaced by their weighted difference and sum. as well as

[0038] To efficiently encode the transformed attribute coefficients, an adaptive lookup table (A-LUT) can be used that tracks the N (e.g., 32) most frequent coefficient symbols and a cache that tracks the last observed different M (e.g., 16) coefficient symbols. The A-LUT can be initialized with N symbols provided by the user or calculated offline based on statistics of similar point clouds. The cache can be initialized with M symbols provided by the user or calculated offline based on statistics of similar point clouds. When a symbol S is encoded, binary information indicating whether S is in the A-LUT can be encoded. If S is in the A-LUT, the index of S in the A-LUT can be encoded using a binary arithmetic encoder. The number of occurrences of symbol S in the A-LUT can be incremented by 1. If S is not in the A-LUT, binary information indicating whether S is in the cache can be encoded. If S is in the cache, the binary representation of its index can be encoded using a binary arithmetic encoder. If S is not in the cache, the binary representation of S can be encoded using a binary arithmetic encoder. The symbol S may be added to the cache and the oldest symbol in the cache may be deleted.

[0039] The embodiments described herein provide methods and apparatus for point cloud coefficient encoding. In detail, encoding of transform coefficients from lifting, predictive transform, and RAHT can be performed by frequency-ordered lookup table index encoding, cache index encoding, and direct encoding of symbol values. In practice, these may require multiple lookup tables and caches with many (typically 32 to potentially up to 256) entries to cover a one-byte codeword. These lookup tables and caches may additionally need to be updated regularly, the frequency of which may imply different trade-offs in terms of computational requirements and coding efficiency. Therefore, it may be advantageous to improve the encoding of transform coefficients for attributes in G-PCC in terms of complexity / memory and compression efficiency trade-offs by alphabet-partitioning and encoding of alphabet-partition information.

[0040] Figure 2is a block diagram of a communication system 200 according to an embodiment. The communication system 200 may include at least two terminals 210 and 220 interconnected via a network 250. For one-way data transmission, the first terminal 210 may encode a point cloud at a local location for transmission to the second terminal 220 via the network 250. The second terminal 220 may receive the encoded point cloud data from the first terminal 210 from the network 250, decode the encoded point cloud data, and display the decoded point cloud data. One-way data transmission may be common in media service applications, etc.

[0041] Figure 2 Also shown are a second pair of terminals 230 and 240, which are provided to support bidirectional transmission of encoded point cloud data, such as might occur during a video conference. For bidirectional transmission of data, each terminal 230 or 240 can encode point cloud data captured at a local location for transmission to the other terminal via network 250. Each terminal 230 or 240 can also receive encoded point cloud data transmitted by the other terminal, can decode the encoded point cloud data, and can display the decoded point cloud data on a local display device.

[0042] exist Figure 2 In the embodiment, terminals 210 to 240 may be illustrated as servers, personal computers, and smart phones, but the principles of the embodiments are not limited thereto. The embodiments are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network 250 represents any number of networks that transmit encoded point cloud data between terminals 210 to 240, including, for example, wired and / or wireless communication networks. The communication network 250 can exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunication networks, local area networks, wide area networks, and / or the Internet. For the purposes of this discussion, the architecture and topology of network 250 may not be important to the operation of the embodiments unless otherwise specified below.

[0043] Figure 3 3 is a diagram illustrating the arrangement of a G-PCC compressor 303 and a G-PCC decompressor 310 in an environment, according to an embodiment. The disclosed subject matter is equally applicable to other point cloud-enabled applications, including, for example, video conferencing, digital television, and storage of compressed point cloud data on digital media including CDs, DVDs, memory sticks, and the like.

[0044] Streaming system 300 can include a capture subsystem 313, which can include a point cloud source 301, such as a digital camera, that creates, for example, uncompressed point cloud data 302. Point cloud data 302, which has a relatively high data volume, can be processed by a G-PCC compressor 303 coupled to point cloud source 301. G-PCC compressor 303 can include hardware, software, or a combination of hardware and software to implement or embody various aspects of the disclosed subject matter as described in greater detail below. Encoded point cloud data 304, which has a relatively low data volume, can be stored on a streaming server 305 for future use. One or more streaming clients 306 and 308 can access streaming server 305 to retrieve copies 307 and 309 of the encoded point cloud data 304. The client 306 may include a G-PCC decompressor 310 that decodes an incoming copy 307 of the encoded point cloud data and creates outgoing point cloud data 311 that can be presented on a display 312 or other presentation device (not depicted). In some streaming systems, the encoded point cloud data 304, 307, and 309 may be encoded according to a video encoding / compression standard. Examples of such standards include those developed by MPEG for G-PCC.

[0045] Figure 4 is a functional block diagram of the G-PCC compressor 303 according to an embodiment.

[0046] like Figure 4 As shown, the G-PCC compressor 303 includes a quantizer 405 , a point removal module 410 , an octree encoder 415 , an attribute transfer module 420 , an LoD generator 425 , a prediction module 430 , a quantizer 435 and an arithmetic encoder 440 .

[0047] The quantizer 405 receives the position of a point in the input point cloud. The position can be an (x, y, z) coordinate. The quantizer 405 also quantizes the received position using, for example, a scaling algorithm and / or a shifting algorithm.

[0048] The point removal module 410 receives the quantized positions from the quantizer 405 and removes or filters duplicate positions from the received quantized positions.

[0049] The octree encoder 415 receives the filtered positions from the point removal module 410 and encodes the received filtered positions into occupancy symbols of an octree representing the input point cloud using an octree encoding algorithm. The bounding box of the input point cloud corresponding to the octree can be any 3D shape, such as a cube.

[0050] The octree encoder 415 also reorders the received filtered positions based on their encodings.

[0051] The attribute transfer module 420 receives the attributes of the points in the input point cloud. The attributes may include, for example, the color or RGB value and / or reflectivity of each point. The attribute transfer module 420 also receives the reordered positions from the octree encoder 415.

[0052] Attribute transfer module 420 also updates the received attributes based on the received reordered positions. For example, attribute transfer module 420 may perform one or more pre-processing algorithms on the received attributes, including, for example, weighting and averaging the received attributes, and interpolating additional attributes from the received attributes. Attribute transfer module 420 also transfers the updated attributes to prediction module 430.

[0053] The LoD generator 425 receives the reordered positions from the octree encoder 415 and obtains the LoD of each point in the points corresponding to the received reordered positions. Each LoD can be considered as a set of points and can be obtained based on the distance of each point in the points. For example, Figure 1A As shown, points P0, P5, P4, and P2 may be in LoD LOD0, points P0, P5, P4, P2, P1, P6, and P3 may be in LoD LOD1, and points P0, P5, P4, P2, P1, P6, P3, P9, P8, and P7 may be in LoD LOD2.

[0054] The prediction module 430 receives the transferred attributes from the attribute transfer module 420 and receives the LoD of each of the obtained points from the LoD generator 425. The prediction module 430 applies a prediction algorithm to the received attributes in sequence based on the LoD of each of the received points to obtain prediction residuals (values) of the received attributes. The prediction algorithm may include any of various prediction algorithms such as interpolation, weighted average calculation, nearest neighbor algorithm, and RDO.

[0055] For example, Figure 1A As shown, before receiving the prediction residuals of the attributes of the points P1, P6, P3, P9, P8, and P7 respectively included in the LoD LOD1 and LOD2, the prediction residuals of the attributes of the points P0, P5, P4, and P2 included in the received LoD LOD0 can be obtained respectively. The prediction residual of the attribute of the received point P2 can be obtained by calculating the distance based on the weighted average of the points P0, P5, and P4.

[0056] The quantizer 435 receives the obtained prediction residual from the prediction module 430 and quantizes the received prediction residual using, for example, a scaling algorithm and / or a shifting algorithm.

[0057] The arithmetic encoder 440 receives the occupancy symbols from the octree encoder 415 and receives the quantized prediction residual from the quantizer 435. The arithmetic encoder 440 performs arithmetic coding on the received occupancy symbols and the quantized prediction residual to obtain a compressed bitstream. The arithmetic coding can include any of various entropy coding algorithms, such as context-adaptive binary arithmetic coding.

[0058] Figure 5 is a functional block diagram of the G-PCC decompressor 310 according to an embodiment.

[0059] like Figure 5 As shown, the G-PCC decompressor 310 includes an arithmetic decoder 505 , an octree decoder 510 , an inverse quantizer 515 , an LoD generator 520 , an inverse quantizer 525 , and an inverse prediction module 530 .

[0060] The arithmetic decoder 505 receives the compressed code stream from the G-PCC compressor 303 and performs arithmetic decoding on the received compressed code stream to obtain occupation symbols and quantized prediction residuals. The arithmetic decoding may include any of various entropy decoding algorithms, such as context-adaptive binary arithmetic decoding.

[0061] The octree decoder 510 receives the obtained occupation symbols from the arithmetic decoder 505 and decodes the received occupation symbols into quantized positions using an octree decoding algorithm.

[0062] The inverse quantizer 515 receives the quantized positions from the octree decoder 510 and inverse quantizes the received quantized positions using, for example, a scaling algorithm and / or a shifting algorithm to obtain reconstructed positions of points in the input point cloud.

[0063] The LoD generator 520 receives the quantized position from the octree decoder 510 and obtains LoD for each of the points corresponding to the received quantized position.

[0064] The inverse quantizer 525 receives the obtained quantized prediction residual and inversely quantizes the received quantized prediction residual using, for example, a scaling algorithm and / or a shift algorithm to obtain a reconstructed prediction residual.

[0065] The inverse prediction module 530 receives the obtained reconstructed prediction residual from the inverse quantizer 525 and the LoD of each of the obtained points from the LoD generator 520. The inverse prediction module 530 sequentially applies a prediction algorithm to the received reconstructed prediction residual based on the LoD of each of the received points to obtain reconstruction attributes of the received reconstructed prediction residual. The prediction algorithm may include any of various prediction algorithms such as interpolation, weighted average calculation, nearest neighbor algorithm, and RDO. The reconstructed attributes belong to the points in the input point cloud.

[0066] The method and apparatus for encoding point cloud coefficients will now be described in detail. Such a method and apparatus may be implemented in the aforementioned G-PCC compressor 303, i.e., the prediction module 430. The method and apparatus may also be implemented in the G-PCC decompressor 310, i.e., the inverse prediction module 530.

[0067] Alphabet partitioning of transform coefficients

[0068] The transformed coefficients or their 8-bit portions can be encoded using a lookup table (e.g., the A-LUT described above) or bypass coding with 256 symbols. The 8-bit coefficient value can be decomposed into a set-index and a symbol-index within the set, which can specify the exact position of the coefficient value in the set. For example, the index value can correspond to a position within a lookup table or a cache. As described in Table 2 below, the 256 possible coefficient values ​​can be grouped into N sets. Table 2. Example of alphabetical division of coefficient values

[0069] In one or more embodiments, offline training can be performed to design a partitioning of coefficient values ​​given a given number of partitions (N). The alphabet-partition boundary values ​​can be explicitly signaled. Alternatively, an index can be signaled to indicate a specific alphabet-partition with associated boundary values ​​given multiple alphabet-partition types shared between the encoder and decoder. It will be appreciated that the partitioning can be designed so that more frequent symbols belong to sets with lower indices and smaller sizes, and vice versa, to improve coding efficiency.

[0070] In one or more embodiments, a cache or a LUT sorted based on frequency can be used to track the frequency of coefficient values ​​in descending order. When forming the alphabet partition, lower set indices can be assigned to more frequent coefficient values, and vice versa, by using indices in the cache or LUT rather than the coefficient values ​​themselves. This process can be performed on the fly at both the encoder and decoder.

[0071] Alphabet - a code that divides information

[0072] The resulting set-index can be entropy coded in various ways, while the accompanying symbol-index can simply be bypass coded when the symbol distribution within the expected set is fairly uniform.

[0073] In one or more embodiments, the derived set-index is encoded by multi-symbol arithmetic coding or other types of context-based binary arithmetic coding.In order to better utilize different characteristics of the coefficients, different alphabet-partitionings may be used.

[0074] In one or more embodiments, different alphabet-partitions may be used for different level-of-detail (LOD) layers of lifting / prediction coefficients, since higher LOD layers may have smaller coefficients as a result of the lifting / prediction decomposition.

[0075] In one or more embodiments, different alphabet-partitions may be used for different quantization parameters (QPs), as higher QPs tend to produce smaller quantization coefficients, and smaller quantization coefficients tend to produce higher QPs.

[0076] In one or more embodiments, different alphabet partitionings may be used for different layers of granular scalability for SNR scalable coding, since enhancement layers (i.e., layers added to refine the reconstructed signal to smaller QP levels) may be noisy or have random properties in terms of correlation between coefficients.

[0077] In one or more embodiments, in the case of SNR scalable coding, different alphabet partitions may be used depending on the value or function of the value of the reconstructed samples from the corresponding position in the lower quantization level layer. For example, it is possible that a region with zero or very small reconstructed values ​​in the lower layer may have different coefficient characteristics than a region with the opposite trend.

[0078] In one or more embodiments, different alphabet partitions may be used based on the values ​​or functions of the values ​​of the reconstructed samples from the corresponding positions in the lower LOD at the same quantization level. These samples from the corresponding positions may be obtained as a result of the nearest neighbor search in the LOD construction in GPCC. It will be appreciated that these samples may be obtained at the decoder and as a result of LOD-by-LOD reconstruction in the transform technique in G-PCC.

[0079] Figure 6 is a flow chart illustrating a method 600 for encoding point cloud coefficients according to an embodiment. In some implementations, Figure 6 One or more processing blocks of may be performed by the G-PCC decompressor 310. In some implementations, Figure 6One or more processing blocks of may be performed by another device or a group of devices, such as the G-PCC compressor 303, separate from or including the G-PCC decompressor 310.

[0080] Reference Figure 6 In a first block 610 , method 600 includes decomposing transform coefficients associated with point cloud data into set-index values ​​and sign-index values, the sign-index value specifying a position of the transform coefficient within the set.

[0081] In a second block 620 , the method 600 includes partitioning the decomposed transform coefficients into one or more sets based on a set-index value and a symbol-index value.

[0082] In a third block 630, the method 600 includes entropy encoding the set-index values ​​of the partitioned transform coefficients.

[0083] In a fourth block 640, the method 600 includes bypass encoding the sign-index values ​​of the partitioned transform coefficients.

[0084] In a fifth block 650 , the method 600 includes compressing the point cloud data based on the entropy-encoded set-index value and the bypass-encoded sign-index value.

[0085] although Figure 6 Example blocks of method 600 are shown, but in some implementations, Figure 6 The method 600 may include additional blocks, fewer blocks, different blocks, or differently arranged blocks compared to the blocks described in . Additionally or alternatively, two or more of the blocks of the method 600 may be executed in parallel.

[0086] Furthermore, the proposed method may be implemented by a processing circuit system (e.g., one or more processors or one or more integrated circuits). In an example, one or more processors execute a program stored in a non-transitory computer-readable medium, the program being used to perform one or more of the proposed methods.

[0087] Figure 7 is a block diagram of an apparatus 700 for point cloud coefficient encoding according to an embodiment.

[0088] Reference Figure 7 , the apparatus 700 includes a decomposition code 710 , a partitioning code 720 , an entropy coding code 730 and a bypass coding code 740 .

[0089] The decomposition code 710 is configured to cause at least one processor to decompose transform coefficients associated with point cloud data into set-index values ​​and sign-index values, the sign-index value specifying a position of the transform coefficient within the set.

[0090] The partition code 720 is configured to cause at least one processor to partition the decomposed transform coefficients into one or more sets based on the set-index value and the symbol-index value.

[0091] The entropy encoding code 730 is configured to cause at least one processor to entropy encode the set-index values ​​of the divided transform coefficients.

[0092] The bypass encoding code 740 is configured to cause at least one processor to bypass encode the sign-index value of the divided transform coefficient.

[0093] The compression code 750 is configured to cause at least one processor to compress the point cloud data based on the entropy-encoded set-index value and the bypass-encoded symbol-index value.

[0094] Figure 8 is a diagram of a computer system 800 suitable for implementing embodiments.

[0095] Computer software may be encoded using any suitable machine code or computer language, and any suitable machine code or computer language may be assembled, compiled, linked, or the like to create code comprising instructions that may be executed directly by a computer central processing unit (CPU), graphics processing unit (GPU), or the like, or through interpretation, microcode execution, or the like.

[0096] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, IoT devices, and the like.

[0097] Figure 8 The components shown for computer system 800 are exemplary in nature and are not intended to suggest any limitation on the scope of use or functionality of computer software implementing the embodiments. Neither should the configuration of components be interpreted as having any dependency or requirement relating to any one or combination of components shown in the embodiments of computing system 800.

[0098] Computer system 800 may include certain human interface input devices. Such human interface input devices may respond to input from one or more human users through, for example, tactile input (such as keystrokes, swipes, data glove movements), audio input (such as voice, tapping), visual input (such as gestures), or olfactory input (not depicted). Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (such as voice, music, ambient sounds), images (such as scanned images, photographic images obtained from a still image camera), and video (such as two-dimensional video, three-dimensional video including stereoscopic video).

[0099] Input human interface devices may include one or more of the following (only one of each is depicted): keyboard 801 , mouse 802 , trackpad 803 , touch screen 810 , joystick 805 , microphone 806 , scanner 807 , and camera 808 .

[0100] The computer system 800 may also include certain human interface output devices. Such human interface output devices may stimulate one or more senses of a human user through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include: tactile output devices (e.g., tactile feedback via touch screen 810 or joystick 805, although tactile feedback devices that do not function as input devices may also be present), audio output devices (such as speakers 809, headphones (not depicted)), visual output devices (such as screen 810, including cathode ray tube (CRT) screens, liquid crystal display (LCD) screens, plasma screens, organic light-emitting diode (OLED) screens, each with or without touch screen input capabilities, each with or without tactile feedback capabilities - some of which may be capable of outputting two-dimensional visual output or more than three-dimensional output through means such as stereographic output; virtual reality glasses (not depicted), holographic displays and cigarette cans (not depicted), and printers (not depicted). Graphics adapter 850 generates images and outputs them to touch screen 810.

[0101] The computer system 800 may also include human-accessible storage devices and their associated media, such as optical media including a CD / DVD read-only memory (ROM) / read-write (RW) drive 820 having CD / DVD, etc. media 821, a thumb drive 822, a removable hard drive or solid-state drive 823, legacy magnetic media such as magnetic tapes and floppy disks (not depicted), dedicated ROM / application specific integrated circuit (ASIC) / programmable logic device (PLD) based devices such as security dongles (not depicted), etc.

[0102] Those skilled in the art will also understand that the term "computer-readable media" used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0103] The computer system 800 may also include an interface to one or more communication networks 855. The communication network 855 may be, for example, wireless, wired, or optical. The network 855 may also be a local area network, a wide area network, a metropolitan area network, an in-vehicle and industrial network, a real-time network, a delay-tolerant network, and the like. Examples of the network 855 include: a local area network such as Ethernet, a wireless local area network (LAN), a cellular network (including the global system for mobile communications (GSM), the third generation (3G), the fourth generation (4G), the fifth generation (5G), long term evolution (LTE), etc.), a television wired or wireless wide area digital network (including cable television, satellite television, and terrestrial broadcast television), an in-vehicle and industrial (including a controller area network (CAN) bus), and the like. The network 855 typically requires an external network interface adapter attached to some general-purpose data port or peripheral bus 849 (such as, for example, a universal serial bus (USB) port of the computer system 800); other networks are typically integrated into the core of the computer system 800 by attaching to the system bus as described below (for example, including the network interface 854 integrated into a PC computer system and / or the cellular network interface integrated into a smartphone computer system). Using any of these networks 855, the computer system 800 can communicate with other entities. Such communication can be one-way receive-only (for example, broadcast television), one-way send-only (for example, a CAN bus to certain CAN bus devices), or two-way (for example, using a local area digital network or a wide area digital network to other computer systems). Certain protocols and protocol stacks can be used on each of these networks 855 and network interfaces 854 as described above.

[0104] The above-mentioned human interface devices, human-accessible storage devices, and network interface 854 may be attached to the core 840 of the computer system 800 .

[0105] The core 840 may include one or more central processing units (CPUs) 841, graphics processing units (GPUs) 842, specialized programmable processing units in the form of field programmable gate arrays (FPGAs) 843, hardware accelerators 844 for certain tasks, and the like. These devices, along with read-only memory (ROM) 845, random-access memory (RAM) 846, and internal mass storage devices 847 such as internal non-user accessible hard drives, solid-state drives (SSDs), and the like, may be connected via a system bus 848. In some computer systems, the system bus 848 may be accessed in the form of one or more physical plugs to enable expansion with additional CPUs, GPUs, and the like. Peripheral devices may be attached to the core's system bus 848 directly or via a peripheral bus 849. Peripheral bus architectures include peripheral component interconnects (PCI), USB, and the like.

[0106] The CPU 841, GPU 842, FPGA 843, and hardware accelerator 844 can execute certain instructions, which, when combined, can constitute the aforementioned computer code. This computer code can be stored in ROM 845 or RAM 846. Transient data can also be stored in RAM 846, while permanent data can be stored, for example, in internal mass storage device 847. Fast storage and retrieval of any of the memory devices can be achieved through the use of cache memory, which can be closely associated with the CPU 841, GPU 842, internal mass storage device 847, ROM 845, RAM 846, and the like.

[0107] The computer-readable medium may have computer code thereon for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the embodiments, or they may be of a type well known and available to those skilled in the art of computer software.

[0108] As an example and not limitation, a computer system 800 having an architecture, and in particular, the core 840, can provide functionality due to a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software implemented in one or more tangible computer-readable media. Such computer-readable media can be media associated with a user-accessible mass storage device as described above, as well as certain storage devices of the core 840 with non-transitory properties, such as a core internal mass storage device 847 or ROM 845. Software implementing various embodiments can be stored in such a device and executed by the core 840. Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software can cause the core 840 and in particular the processor therein (including a CPU, GPU, FPGA, etc.) to perform specific processing or specific parts of specific processing described herein, including defining data structures stored in RAM 846 and modifying such data structures according to processing defined by the software. Additionally or alternatively, the computer system can provide functionality due to logic implemented in circuits (e.g., hardware accelerator 844) that is hardwired or otherwise, which can replace software or operate in conjunction with software to perform a specific process or a specific portion of a specific process described herein. Where appropriate, the software involved can encompass logic, and conversely, the logic involved can also encompass software. Where appropriate, the computer-readable medium involved can encompass circuits (e.g., integrated circuits (ICs)) storing software for execution, circuits implementing logic for execution, or both. Implementations encompass any suitable combination of hardware and software.

[0109] Although the present disclosure has described several embodiments, there are changes, permutations, and various substitute equivalents that fall within the scope of the present disclosure. It will therefore be understood that although not explicitly shown or described herein, those skilled in the art will be able to devise many systems and methods that implement the principles of the present disclosure and are therefore within the spirit and scope of the present disclosure.

Claims

1. A method for encoding point cloud coefficients, the method comprising: dividing the transform coefficients associated with the point cloud data into one or more sets based on the set-index value and the sign-index value; performing entropy encoding on the set-index values ​​of the divided transform coefficients, wherein the entropy encoding comprises aggregating the set-index values ​​to form a super-symbol; bypass encoding the sign-index values ​​of the divided transform coefficients; as well as The point cloud data is compressed based on the entropy-encoded set-index value and the bypass-encoded sign-index value.

2. The method according to claim 1, wherein The division of the transform coefficients includes: The transform coefficients associated with the point cloud data are decomposed into the set-index value and the sign-index value, the sign-index value specifying a position of the transform coefficient within a set.

3. The method according to claim 1, wherein Aggregating the set-index values ​​to form a super symbol includes: Determining a maximum set-index value from the set-index values ​​of child nodes of the same parent node in the octree associated with the point cloud data; Based on the maximum set-index value being non-zero, performing entropy encoding on the maximum set-index value; and Based on the fact that the maximum set-index value is smaller, a plurality of set-index values ​​are aggregated together to perform entropy coding on one or more set index value groups.

4. The method according to claim 1, wherein Aggregating the set-index values ​​to form a super symbol includes: Determining a maximum set-index value from the set-index values ​​of child nodes of the same parent node in the octree associated with the point cloud data; Based on the maximum set index value being non-zero, performing entropy encoding on the maximum set index value; entropy encoding the alpha value using an alphabet having 255 symbols; and Based on the sibling node occupancy value-index of the octree being less than the maximum set-index value, a plurality of set-index values ​​are aggregated together to perform entropy coding on one or more set index value groups.

5. The method according to any one of claims 1 to 4, wherein The frequency values ​​associated with the transform coefficients are stored in a cache or a lookup table sorted based on frequency in descending order, wherein the lowest set-index value is assigned to the transform coefficient having the largest frequency value.

6. The method according to any one of claims 1 to 5, wherein The symbol-index value and the set-index value are signaled to indicate an alphabet-partition with associated boundary values ​​based on one or more alphabet-partition types shared between an encoder and a decoder.

7. The method according to claim 6, wherein: The one or more alphabet-partitioning types are used for one or more level-of-detail layers corresponding to the transform coefficients.

8. The method according to claim 6, wherein: The one or more alphabet-partitioning types are used for one or more quantization parameters based on quantization of the transform coefficients.

9. The method according to claim 6, wherein: The one or more alphabet-partitioning types are used for one or more layers of scalability for signal-to-noise ratio scalable coding based on correlation between the transform coefficients.

10. The method according to any one of claims 1 to 5, wherein The set-index value is encoded by multi-symbol arithmetic coding.

11. A device for encoding point cloud coefficients, the device comprising: at least one memory configured to store computer program code; as well as At least one processor is configured to access the at least one memory and perform the method according to any one of claims 1 to 10 according to the computer program code. 12 . A non-transitory computer-readable storage medium storing instructions, the instructions being configured to cause at least one processor to perform the method according to claim 1 .

Images