Method and device for encoding and decoding point cloud on the basis of two-dimensional regularization planar projection

Abstract

To disclose a point cloud encoding and decoding method and device based on a two-dimensional regularization plane projection.SOLUTION: An encoding method includes: acquiring original point cloud data; performing two-dimensional regularization plane projection on the original point cloud data to obtain a two-dimensional projection plane structure; obtaining a plurality of pieces of two-dimensional image information according to the two-dimensional projection plane structure; and encoding the plurality of pieces of two-dimensional image information to obtain code stream information. A point cloud in a three-dimensional space is projected to a corresponding two-dimensional regularization projection plane structure, and regularization correction is performed on the point cloud in a vertical direction and a horizontal direction, to obtain a strong correlation representation of the point cloud on the two-dimensional projection plane structure, so that the spatial correlation of the point cloud is better reflected. When the plurality of pieces of two-dimensional image information are subsequently encoded, the spatial correlation of the point cloud can be greatly utilized, and spatial redundancy is reduced, thereby further improving encoding efficiency of the point cloud.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] This application claims priority to Chinese Patent Application No. 202110172795.4, entitled “Point Cloud Encoding and Decoding Method and Apparatus Based on Two-Dimensional Regularized Planar Projection,” filed with the China National Intellectual Property Office on February 8, 2021, which is incorporated herein by reference in its entirety.

[0002] The present invention relates to the technical field of encoding and decoding, and in particular to a point cloud encoding and decoding method and apparatus based on 2D regularized planar projection. [Background technology]

[0003] With the improvement of hardware processing power and the rapid development of computer vision, 3D point cloud has become a new generation of immersive multimedia after audio, image and video, and is widely applied in virtual reality, augmented reality, autonomous driving, environment modeling and the like. However, 3D point cloud usually has a relatively large amount of data, which is not conducive to the transmission and storage of point cloud data. Therefore, it is of great significance to research efficient point cloud encoding and decoding techniques.

[0004] In the existing Geometry-based Point Cloud Compression (G-PCC) encoding framework, the geometry information and attribute information of the point cloud are encoded separately. Currently, G-PCC geometric encoding and decoding can be divided into octree-based geometric encoding and decoding and predictive tree-based geometric encoding and decoding.

[0005] Octree-based geometric encoding and decoding: At the encoder side, first, the geometric information of the point cloud is preprocessed, which includes coordinate transformation and voxelization process of the point cloud. Then, in the breadth-first traversal order, tree partitioning (octree / quadtree / binary tree) is successively performed on the bounding box where the point cloud is located. Finally, the placeholder code of each node is encoded, and the number of points contained in each leaf node is encoded to generate a binary code stream. At the decoder side, first, the placeholder code of each node is successively obtained by parsing in the breadth-first traversal order. Then, the tree partitioning is successively performed in order until a 1×1×1 unit cube is obtained through partitioning, and then the partitioning stops. Finally, the number of points contained in each leaf node is obtained by parsing, and finally the reconstructed point cloud geometric information is obtained.

[0006] Prediction Tree Based Geometric Encoding and Decoding: At the encoder side, the original point cloud is firstly sorted. Then, a prediction tree structure is constructed. The prediction tree structure is constructed according to multiple different laser scanners by classifying each point into the laser scanner it belongs to. Then, each node in the prediction tree is traversed, and a prediction residual is obtained by selecting different prediction modes to predict the geometry information of the node, and the prediction residual is quantized using the quantization parameter. Finally, the prediction tree structure, the quantization parameter, and the prediction residual of the geometry information of the node, etc. are encoded to generate a binary code stream. At the decoder side, the code stream is firstly analyzed. Then, the prediction tree structure is reconstructed, and then, based on the prediction residual of the geometry information of each node obtained by parsing and the quantization parameter, the prediction residual is dequantized, and finally, the reconstructed geometry information of each node is restored. That is, the reconstruction of the point cloud geometry information is completed.

[0007] However, due to the relatively strong spatial sparseness of the point cloud, in the point cloud encoding technique using the octree structure, the structure leads to a relatively large proportion of empty nodes being obtained by division, which cannot fully reflect the spatial correlation of the point cloud, which is not conducive to point cloud prediction and entropy coding. In the predictive tree-based point cloud encoding and decoding technique, a tree structure is constructed using some parameters of a lidar device, and the tree structure is used for predictive coding based on the tree structure. However, the tree structure does not fully reflect the spatial correlation of the point cloud, which is not conducive to point cloud prediction and entropy coding. Therefore, both of the above two point cloud encoding and decoding techniques have the problem of insufficient coding efficiency. Summary of the Invention

[0008] In order to solve the above problems in the existing technology, the present invention provides a point cloud encoding and decoding method and device based on 2D regularized planar projection. The technical problem solved by the present invention is realized by the following technical solutions:

[0009] A method for encoding a point cloud based on a two-dimensional regularized planar projection is provided, the method comprising: Obtaining original point cloud data; performing a two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure; Obtaining a plurality of two-dimensional image information according to a two-dimensional projection plane structure; and encoding the plurality of two-dimensional image information to obtain codestream information.

[0010] In one embodiment of the present invention, performing a two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure includes: Initializing a two-dimensional projection plane structure; determining a mapping relationship between the original point cloud data and the two-dimensional projected planar structure, and projecting the original point cloud data onto the two-dimensional projected planar structure.

[0011] In one embodiment of the present invention, the plurality of two-dimensional image information includes geometry information maps, which include a placeholder information map, a depth information map, a projection residual information map, and a coordinate transformation error information map.

[0012] In one embodiment of the present invention, the step of encoding the plurality of two-dimensional image information to obtain code stream information includes: encoding the placeholder information map, the depth information map, the projection residual information map, and the coordinate transformation error information map to obtain a placeholder information code stream, a depth information code stream, a projection residual information code stream, and a coordinate transformation error information code stream, respectively; Obtaining a geometry information codestream according to the placeholder information codestream, the depth information codestream, the projection residual information codestream, and the coordinate transformation error information codestream.

[0013] In one embodiment of the present invention, after obtaining the geometry information codestream, the method further comprises: Performing geometry reconstruction according to the geometry information code stream to obtain reconstructed point cloud geometry information; encoding attribute information of the original point cloud data based on the reconstructed point cloud geometry information to obtain an attribute information codestream.

[0014] In one embodiment of the present invention, the plurality of two-dimensional image information further includes an attribute information map.

[0015] In one embodiment of the present invention, the step of encoding the plurality of two-dimensional image information to obtain code stream information further comprises: encoding the attribute information map to obtain an attribute information codestream.

[0016] Another embodiment of the present invention further provides a point cloud encoding device based on 2D regularized planar projection, the device comprising: a first data acquisition module configured to acquire original point cloud data; a projection module configured to perform a two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure; a data processing module configured to obtain a plurality of two-dimensional image information according to a two-dimensional projection plane structure; and an encoding module configured to encode the plurality of two-dimensional image information to obtain codestream information.

[0017] Yet another embodiment of the present invention further provides a point cloud decoding method based on 2D regularized planar projection, the method comprising: obtaining codestream information and decoding the codestream information to obtain parsed data; Reconstructing a plurality of two-dimensional image information according to the analyzed data; Obtaining a two-dimensional projection plane structure according to the plurality of two-dimensional image information; and reconstructing the point cloud using the two-dimensional projection plane structure.

[0018] Yet another embodiment of the present invention further provides a point cloud decoding device based on 2D regularized planar projection, the device comprising: a second data acquisition module configured to acquire codestream information and decode the codestream information to obtain parsed data; a first reconstruction module configured to reconstruct a plurality of two-dimensional image information according to the analyzed data; a second reconstruction module configured to obtain a two-dimensional projection plane structure according to the plurality of two-dimensional image information; and a point cloud reconstruction module configured to reconstruct a point cloud using the two-dimensional projection plane structure.

[0019] The beneficial effects of the present invention are as follows: According to the present invention, by projecting a point cloud in a three-dimensional space onto a corresponding two-dimensional regularized projection plane structure, and performing regularization correction on the point cloud in the vertical and horizontal directions to obtain a strongly correlated representation of the point cloud on the two-dimensional projection plane structure, the sparseness in the three-dimensional representation structure is avoided, and the spatial correlation of the point cloud is better reflected, so that when multiple two-dimensional image information obtained for the two-dimensional regularized projection plane structure is subsequently encoded, the spatial correlation of the point cloud can be greatly utilized, and the spatial redundancy is reduced, thereby further improving the encoding efficiency of the point cloud.

[0020] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. [Brief description of the drawings]

[0021] [Figure 1] FIG. 2 is a schematic diagram of a point cloud encoding method based on 2D regularized planar projection, according to one embodiment of the present invention; [Diagram 2] FIG. 2 is a schematic diagram of the correspondence between cylindrical coordinates of a point and a pixel in a two-dimensional projection plane, according to an embodiment of the present invention; [Diagram 3] FIG. 2 is a schematic diagram of a two-dimensional projection plane structure of a point cloud, according to an embodiment of the present invention; [Figure 4] FIG. 4 is a schematic diagram of a projection residual according to an embodiment of the present invention; [Diagram 5] FIG. 2 is a diagram of a framework for encoding point cloud geometry information according to one embodiment of the present invention. [Figure 6] FIG. 2 is a framework diagram of performing encoding of attribute information based on reconstructed geometry information according to one embodiment of the present invention. [Figure 7]FIG. 2 is a framework diagram for simultaneously encoding point cloud geometry information and attribute information according to one embodiment of the present invention. [Figure 8] FIG. 1 is a schematic diagram of a point cloud encoding device based on 2D regularized planar projection, according to an embodiment of the present invention. [Figure 9] FIG. 2 is a schematic diagram of a point cloud decoding method based on 2D regularized planar projection according to an embodiment of the present invention; [Figure 10] FIG. 1 is a schematic block diagram of a point cloud decoding device based on 2D regularized planar projection according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The present invention will be described in more detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0023] EMBODIMENT 1 FIG. 1 is a schematic diagram of a point cloud encoding method based on 2D regularized planar projection according to an embodiment of the present invention, which includes the following steps:

[0024] S1: Get the original point cloud data.

[0025] Specifically, the original point cloud data typically includes a group of three-dimensional spatial points, each of which records its geometric position information and additional attribute information, such as color, reflectance, and normal. The geometric position information of the point cloud is generally represented based on the Cartesian coordinate system, i.e., using the coordinates x, y, and z of the point. The original point cloud data may be acquired through a 3D scanning device, such as a lidar, or alternatively, may be acquired based on public datasets provided by various platforms. In this embodiment, it is assumed that the geometric position information of the acquired original point cloud data is represented based on the Cartesian coordinate system. It should be noted that the representation method of the geometric position information of the original point cloud data is not limited to Cartesian coordinates.

[0026] S2: Perform 2D regularized planar projection on the original point cloud data to obtain a 2D projected planar structure.

[0027] Specifically, in this embodiment, before performing the 2D regularized planar projection on the original point cloud, further pre-processing, such as voxelization, is performed on the original point cloud data to facilitate subsequent encoding.

[0028] First, a 2D projection plane structure is initialized.

[0029] The initialization of the 2D regularized projection plane structure of the point cloud requires the use of regularization parameters. The regularization parameters are usually precisely measured by the manufacturer and provided to the consumer as one of the required data, such as the acquisition range of the lidar, the sampling angular resolution of the horizontal azimuth angle or the number of sampling points, the distance correction coefficient of each laser scanner, the offset information of the laser scanner in the vertical and horizontal directions V o and H o , and the offset information of the laser scanner along the pitch angle and horizontal azimuth angle, θ and α.

[0030] It should be noted that the regularization parameters are not limited to the parameters given above. Given calibration parameters of the lidar may be used as the regularization parameters, or when the calibration parameters of the lidar are not given, the regularization parameters may be obtained by a method such as estimation optimization and data fitting.

[0031] The 2D regularized projected plane structure of a point cloud is a data structure that contains M rows and N columns of pixels, where a point in the 3D point cloud corresponds to a pixel in the projected data structure. Furthermore, pixel (i,j) in the data structure may be associated with cylindrical coordinate components (θ,φ). For example, pixel (i,j) corresponding to cylindrical coordinates (r,θ,φ) can be expressed as:

number

[0032] Specifically, FIG. 2 is a schematic diagram of the correspondence between cylindrical coordinates of a point and a pixel in a two-dimensional projection plane according to an embodiment of the present invention.

[0033] It should be noted that the pixel correspondence here is not limited to cylindrical coordinates.

[0034] In addition, the resolution of the 2D regularized projection plane can be obtained using the regularization parameter. For example, the resolution of the 2D regularized projection plane is assumed to be M×N, and M is initialized using the number of laser scanners in the regularization parameter, and the sampling angle resolution of the horizontal azimuth angle is:

number

number

[0035] Furthermore, in order to project the original point cloud data onto the two-dimensional projection plane structure, a mapping relationship between the original point cloud data and the two-dimensional projection plane structure is determined.

[0036] In this part, the point cloud, which is originally randomly distributed in the Cartesian coordinate system, is mapped onto a uniformly distributed 2D regularized projected plane structure by determining the position of the original point cloud in the 2D projected plane structure for each point. Specifically, for each point in the original point cloud, the corresponding pixel in the 2D projected plane structure is determined. For example, the pixel with the shortest spatial distance from the projected position of that point in the 2D plane may be selected as the corresponding pixel of that point.

[0037] When the cylindrical coordinate system is used for the 2D projection, the specific process of determining the pixels corresponding to the original point cloud is as follows:

[0038] a. The cylindrical coordinate component r of the current point of the original point cloud data is determined, specifically, as follows:

number

[0039] b. The search area of ​​the current point in the two-dimensional projection plane structure is determined. Specifically, the entire two-dimensional projection plane structure may be directly selected as the search area. In addition, in order to reduce the amount of calculation, the pitch angle θ and the azimuth angle φ of the cylindrical coordinate components of the current point may be further used to determine the search area of ​​the corresponding pixel in the two-dimensional projection plane structure, thereby reducing the search area.

[0040] c. After the search area is determined, for each pixel (i,j) in the search area, the calibration parameters θ0, V o , H o , and a regularization parameter α to calculate the position (xl,yl,zl) of the current pixel in the Cartesian coordinate system. The specific calculation formula is:

number

[0041] d. After obtaining the position (xl,yl,zl) of the current pixel in the Cartesian coordinate system, the spatial distance between that position and the current point (x,y,z) is calculated and the error Err, i.e.,

number

[0042] If the error Err is smaller than the current minimum error minErr, the error Err is used to update the minimum error minErr, and the i and j corresponding to the current pixel are used to update the i and j of the pixel corresponding to the current point; if the error Err is larger than the minimum error minErr, the above update process will not be performed.

[0043] e. After traversing all the pixels in the search area, the corresponding pixel (i,j) of the current point in the 2D projected plane structure can be determined.

[0044] Once the above process is completed for all points in the original point cloud, the 2D regularized planar projection of the point cloud is completed. Specifically, Figure 3 is a schematic diagram of a 2D projected planar structure of a point cloud according to an embodiment of the present invention. Each point of the original point cloud data is mapped to a corresponding pixel in this structure.

[0045] It should be noted that in the two-dimensional regularized plane projection of the point cloud, multiple points in the point cloud may correspond to the same pixel in the two-dimensional projection plane structure. To avoid this situation, these spatial points may be selected to be projected to different pixels during projection. For example, during the projection of a certain point, if the pixel corresponding to the point already has a corresponding point, the point is projected to an empty pixel adjacent to the pixel. Furthermore, if multiple points in the point cloud are projected to the same pixel in the two-dimensional projection plane structure, the number of corresponding points in each pixel should be additionally encoded during encoding based on the two-dimensional projection plane structure, and the information of each corresponding point in the pixel is encoded according to the number of points.

[0046] S3: Obtain multiple 2D image information according to the 2D projection plane structure.

[0047] In this embodiment, the plurality of two-dimensional image information may include a geometry information map, which may be one or more of a placeholder information map, a depth information map, a projection residual information map, and a coordinate transformation error information map, or may be another geometry information map.

[0048] In this embodiment, the above four geometry information maps are specifically used as examples for a detailed description.

[0049] a. Placeholder Information Map The placeholder information map is used to identify whether each pixel in the two-dimensional regularized projection plane structure is occupied, i.e., whether each pixel corresponds to a point in the point cloud. If each pixel is occupied, the pixel is referred to as being non-empty, otherwise, the pixel is referred to as being empty. For example, 0 and 1 may be used to represent, where 1 indicates that the pixel is currently occupied, and 0 indicates that the pixel is currently unoccupied. Thus, the placeholder information map of the point cloud can be obtained according to the two-dimensional projection plane structure of the point cloud.

[0050] b. Depth information map The depth information map is used to represent the distance between the corresponding point of each occupied pixel in the 2D regularized projected plane structure and the coordinate origin. For example, the cylindrical coordinate component r of the point corresponding to the pixel can be used as the depth of the pixel. Assuming that the Cartesian coordinate of the point corresponding to the pixel is (x,y,z), the cylindrical coordinate component r of the point, i.e., the depth of the pixel, can be expressed by the formula:

number

[0051] c. Projection residual information map The projection residual information map is used to represent the residual between the corresponding position of each occupied pixel in the two-dimensional regularized projection plane structure and the actual projected position. Figure 4 is a schematic diagram of the projection residual according to one embodiment of the present invention.

[0052] Specifically, the projection residual of a pixel can be calculated as follows: Assuming that the current pixel is (i,j) and the Cartesian coordinates of the corresponding point of the current pixel are (x,y,z), the actual projected position of the point can be expressed as (φ′,i′), which can be expressed as:

number

[0053] The corresponding position of the current pixel can be represented as (φ,i), which is expressed as:

number

[0054] Therefore, the projection residual (Δφ, Δi) corresponding to the current pixel is given by:

number

[0055] Based on the above calculation, each occupied pixel in the 2D regularized projection plane has a projection residual, resulting in a projection residual information map corresponding to the point cloud.

[0056] d. Coordinate conversion error information map The coordinate transformation error information map is used to represent the residual between the spatial location obtained through backprojection of each occupied pixel in the 2D regularized projected plane structure and the spatial location of the original point corresponding to that pixel.

[0057] For example, the coordinate transformation error of a pixel can be calculated as follows: Assuming the current pixel is (i,j) and the Cartesian coordinates of the corresponding point of the current pixel are (x,y,z), we can transform the pixel back to the Cartesian coordinate system to get the corresponding Cartesian coordinate (xl,yl,zl) using the regularization parameter and the following formula:

number

[0058] Then, the coordinate transformation error (Δx, Δy, Δz) of the current pixel is calculated by the following formula:

number

[0059] Based on the above calculation, each occupied pixel in the 2D regularized projected plane structure has a coordinate transformation error, and as a result, a coordinate transformation error information map corresponding to the point cloud is obtained.

[0060] According to the present invention, by projecting a point cloud in a 3D space onto a corresponding 2D regularized projected plane structure, and performing regularization correction on the point cloud in the vertical and horizontal directions to obtain a highly correlated representation of the point cloud on the 2D projected plane structure, the spatial correlation of the point cloud can be better reflected, and thus the encoding efficiency of the point cloud can be further improved.

[0061] S4: Encode the plurality of two-dimensional image information to obtain code stream information.

[0062] FIG. 5 is a diagram of a framework for encoding point cloud geometry information according to one embodiment of the present invention.

[0063] In this embodiment, the placeholder information map, depth information map, projection residual information map, and coordinate transformation error information map obtained in step S3 are encoded to obtain a placeholder information code stream, a depth information code stream, a projection residual information code stream, and a coordinate transformation error information code stream, respectively.

[0064] Specifically, in this embodiment, the pixels in the placeholder information map, the depth information map, the projection residual information map, and the coordinate transformation error information map are traversed separately in a certain scanning order, such as a Z-shaped scan.

[0065] For a current pixel in the placeholder information map, the reconstructed placeholder information of the encoded and decoded pixel can be used for prediction. Specifically, various existing neighborhood prediction techniques can be used. After obtaining the corresponding prediction residual, the existing entropy coding technique can be used for coding to obtain the placeholder information code stream.

[0066] For a current pixel in the depth information map, the reconstructed placeholder information map and the reconstructed depth information of the encoded and decoded pixel can be used for prediction. Specifically, prediction can be performed based on an existing neighborhood prediction technique combined with placeholder information of neighboring pixels, i.e., only neighboring pixels with non-empty placeholder information are used to predict the depth information of the current pixel. The predicted value can be calculated in a manner such as weighted average. After obtaining the corresponding prediction residual, an existing entropy coding technique can be used for coding to obtain a depth information code stream.

[0067] For a current pixel in the projection residual information map, the reconstructed placeholder information map and depth information map, as well as the reconstructed projection residual information of the encoded and decoded pixel, can be used for prediction. Specifically, prediction can be performed based on existing neighborhood prediction technology combined with placeholder information and depth information of neighboring pixels, i.e., only neighboring pixels that have non-empty placeholder information and depth information close to the depth information of the current pixel are used to predict the projection residual information of the current pixel. The predicted value can be calculated in a manner such as weighted average. After obtaining the corresponding prediction residual, an existing entropy coding technology can be used for coding to obtain a prediction residual information code stream.

[0068] For a current pixel in the coordinate transformation error information map, the reconstructed placeholder information map, depth information map, and projection residual information map, as well as the reconstructed coordinate transformation error information of the encoded and decoded pixel, may be used for prediction. Specifically, prediction may be performed based on an existing neighborhood prediction technique combined with placeholder information, depth information, and projection residual information of neighboring pixels, i.e., only neighboring pixels having non-empty placeholder information and having depth information and projection residual information close to that of the current pixel are used to predict the coordinate transformation error information of the current pixel. The predicted value may be calculated in a manner such as weighted average. After obtaining the corresponding prediction residual, a coordinate transformation error information code stream may be obtained by encoding using an existing entropy coding technique.

[0069] The prediction residual may alternatively be quantized and then coded.

[0070] In this embodiment, when multiple two-dimensional map information obtained by two-dimensional regularized plane projection, such as a placeholder information map, a depth information map, a projection residual information map, and a coordinate transformation error information map, are encoded, the strong correlation in the two-dimensional image is effectively used to perform prediction and entropy encoding for the current pixel, so that the spatial correlation of the point cloud can be greatly utilized and the spatial redundancy can be reduced, thereby further improving the encoding efficiency of the point cloud.

[0071] A geometry information codestream is obtained according to the placeholder information codestream, the depth information codestream, the projection residual information codestream, and the coordinate transformation error information codestream.

[0072] In this embodiment, after all the geometry information maps are encoded, the geometry information codestream of the original point cloud data can be obtained.

[0073] In another embodiment of the present invention, the geometry information map may instead be compressed through image / video compression, including but not limited to JPEG, JPEG2000, HEIF, H.264|AVC, and H.265|HEVC.

[0074] It should also be noted that the change and adjustment of the projection residual of the point cloud will affect the magnitude of its coordinate transformation error. Therefore, in order to obtain higher coding efficiency, the projection residual and coordinate transformation error of the point cloud may be adjusted rather than fixed. For example, in lossless coding, when the two-dimensional projection accuracy is relatively high, the projection residual will be relatively small. Therefore, the projection residual may be set to 0. Although this operation slightly increases the coordinate transformation error, the coding efficiency of the projection residual can be significantly improved if the coding efficiency of the coordinate transformation error is allowed to have a certain decrease or remain unchanged. Alternatively, when the accuracy of the two-dimensional projection is relatively low, the projection residual will be relatively large. In this case, the projection residual may be appropriately adjusted, so that the coordinate transformation error changes accordingly. The adjusted projection residual and coordinate transformation error can then be coded to obtain higher coding efficiency. In lossy coding, when the two-dimensional projection accuracy is relatively high, the projection residual will be relatively small, and therefore the projection residual does not need to be coded. Alternatively, if the coordinate transformation error is relatively small, the coordinate transformation error does not need to be coded, thereby improving the coding efficiency. Alternatively, the projection residual and the coordinate transformation error may be adjusted appropriately, and then the adjusted projection residual and the adjusted coordinate transformation error may be coded to obtain higher coding efficiency.

[0075] EMBODIMENT 2 Based on completing the encoding of the geometry information in embodiment 1, the encoding of the attribute information can be further performed based on the reconstructed geometry information. Figure 6 is a framework diagram of performing encoding of the attribute information based on the reconstructed geometry information according to an embodiment of the present invention.

[0076] First, geometry reconstruction is performed according to the geometry information codestream obtained in the first embodiment to obtain reconstructed point cloud geometry information.

[0077] Then, the attribute information of the original point cloud data is encoded based on the reconstructed point cloud geometry information to obtain an attribute information codestream.

[0078] Specifically, the encoding of attribute information is generally performed on color information and reflectance information of spatial points. The attribute information of the original point cloud data can be encoded using existing techniques based on the geometric reconstruction information of the point cloud. For example, the color information in the attribute is first converted from RGB color space to YUV color space. Then, the point cloud is recolored based on the reconstructed geometry information, so that the attribute information that is not encoded corresponds to the reconstructed geometry information. After sorting the point cloud using Morton code or Hilbert code, the reconstructed attribute values ​​of the encoded points are used to perform interpolation prediction on the points to be predicted to obtain predicted attribute values, and then a difference is made between the actual attribute values ​​and the predicted attribute values ​​to obtain prediction residuals. Finally, the prediction residuals are quantized and encoded to generate a binary code stream.

[0079] EMBODIMENT 3 According to embodiment 1, by using the two-dimensional projection plane structure, the attribute information map can also be obtained simultaneously. Then, the geometry information map and the attribute information map are encoded simultaneously to obtain the geometry information code stream and the attribute information code stream. Figure 7 is a framework diagram of simultaneously encoding the point cloud geometry information and attribute information according to an embodiment of the present invention.

[0080] Specifically, the encoding process of the geometry information map may refer to embodiment 1. The encoding process of the attribute information map is as follows:

[0081] First, the pixels in the attribute information map are traversed in a certain scan order, for example, through a Z-shaped scan. Then, for a current pixel in the attribute information map, the reconstructed placeholder information map and the reconstructed attribute information of the encoded and decoded pixel can be used for prediction. Specifically, prediction can be performed based on an existing neighborhood prediction technique combined with placeholder information of neighboring pixels, i.e., only neighboring pixels having non-empty placeholder information are used to predict the attribute information of the current pixel. The predicted value can be calculated in a manner such as weighted average. After obtaining the corresponding prediction residual, an attribute information code stream can be obtained by encoding using an existing entropy coding technique.

[0082] In this embodiment, the geometry information map and the attribute information map are simultaneously obtained using a two-dimensional projection plane structure, and then the geometry information and the attribute information are simultaneously coded, thereby improving the coding efficiency.

[0083] EMBODIMENT 4 According to the first to third embodiments, this embodiment provides a point cloud encoding device based on two-dimensional regularized planar projection. Figure 8 is a schematic diagram of a point cloud encoding device based on two-dimensional regularized planar projection according to an embodiment of the present invention, which includes: a first data acquisition module 11 configured to acquire original point cloud data; a projection module 12 configured to perform a two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure; a data processing module 13 configured to obtain a plurality of two-dimensional image information according to a two-dimensional projection plane structure; and an encoding module 14 configured to encode the plurality of two-dimensional image information to obtain codestream information.

[0084] The encoding device provided in this embodiment can implement the encoding methods described in embodiments 1 to 3, and the detailed process will not be described again here.

[0085] EMBODIMENT 5 FIG. 9 is a schematic diagram of a point cloud decoding method based on two-dimensional regularized planar projection according to an embodiment of the present invention, the method including:

[0086] Step 1: Obtain codestream information, and decode the codestream information to obtain parsed data.

[0087] The decoder side obtains the compressed codestream information, and performs corresponding decoding on the codestream information using corresponding existing entropy decoding techniques to obtain parsed data.

[0088] Step 2: Reconstruct multiple 2D image information according to the analyzed data.

[0089] In this embodiment, the plurality of two-dimensional image information includes a geometry information map, which includes a placeholder information map, a depth information map, a projection residual information map, and a coordinate transformation error information map.

[0090] On this basis, the analyzed data mainly includes: a prediction residual of placeholder information, a prediction residual of depth information, a prediction residual of projection residual information, and a prediction residual of coordinate transformation error information.

[0091] Encoder side traverses the pixels in placeholder information map, depth information map, projection residual information map and coordinate transformation error information map separately in a certain scanning order, and encodes corresponding information, so that the prediction residual information of pixels obtained at decoder side is also in this order, and decoder side can obtain the resolution of 2D map by using regularization parameter.For details, refer to the part of initializing 2D projection plane structure in S2 of embodiment 1.Therefore, decoder side can know the position of the pixel to be currently reconstructed in 2D map.

[0092] For the pixel to be currently reconstructed in the placeholder information map, the reconstructed placeholder information of the coded and decoded pixel is used for prediction. The prediction method is consistent with that of the encoder side. Then, the placeholder information of the current pixel is reconstructed according to the obtained prediction value and the analyzed prediction residual.

[0093] For the pixel to be currently reconstructed in the depth information map, the reconstructed placeholder information map and the reconstructed depth information of the encoded and decoded pixels can be used for prediction. The prediction method is consistent with that of the encoder side, that is, only the neighboring pixels with non-empty placeholder information are used to predict the depth information of the current pixel, and the depth information of the current pixel is reconstructed according to the obtained prediction value and the analyzed prediction residual.

[0094] For the pixel to be reconstructed in the projection residual information map, the reconstructed placeholder information map, the reconstructed depth information map, and the reconstructed projection residual information of the coded and decoded pixel can be used for prediction. The prediction method is consistent with that of the encoder side, that is, only the neighboring pixels that have non-empty placeholder information and have depth information close to the depth information of the current pixel are used to predict the projection residual information of the current pixel, and then the projection residual information of the current pixel is reconstructed according to the obtained prediction value and the analyzed prediction residual.

[0095] For the pixel to be reconstructed in the coordinate transformation error information map currently, the reconstructed placeholder information map, the reconstructed depth information map, the reconstructed projection residual information map, and the reconstructed coordinate transformation error information of the encoded and decoded pixel can be used for prediction. The prediction method is consistent with that of the encoder side, that is, only the neighboring pixels that have non-empty placeholder information and have depth information and projection residual information close to that of the current pixel are used to predict the coordinate transformation error information of the current pixel, and then the coordinate transformation error information of the current pixel is reconstructed according to the obtained prediction value and the analyzed prediction residual.

[0096] After each pixel in the placeholder information map, the depth information map, the projection residual information map, and the coordinate transformation error information map is reconstructed, a reconstructed placeholder information map, a reconstructed depth information map, a reconstructed projection residual information map, and a reconstructed coordinate transformation error information map can be obtained.

[0097] Moreover, the analyzed data may further include prediction residuals of the attribute information, and correspondingly, the attribute information map may also be reconstructed based on the information.

[0098] Step 3: Obtain a two-dimensional projection plane structure according to the plurality of two-dimensional image information.

[0099] Since the resolution of the two-dimensional projection plane structure is consistent with the resolution of the placeholder information map, the depth information map, the projection residual information map, and the coordinate transformation error information map, and all the two-dimensional image information has been reconstructed, the placeholder information, depth information, projection residual information, and coordinate transformation error information of each pixel in the two-dimensional projection plane structure can be known, and the reconstructed two-dimensional projection plane structure can be obtained.

[0100] Correspondingly, the reconstructed 2D projected plane structure can further include attribute information of the point cloud.

[0101] Step 4: Reconstruct the point cloud using the 2D projection plane structure.

[0102] Specifically, by traversing the pixels in the reconstructed two-dimensional projection plane structure in a certain scanning order, the placeholder information, depth information, projection residual information, and coordinate transformation error information of each pixel can be known. If the placeholder information of a current pixel (i, j) is non-empty, the spatial point (x, y, z) corresponding to the pixel can be reconstructed according to the depth information of the current pixel, that is, the cylindrical coordinate component r of the corresponding point of the pixel, the projection residual information, that is, the residual (Δφ, Δi) between the corresponding position of the pixel and the actual projection position, and the coordinate transformation error information, that is, the residual (Δx, Δy, Δz) between the spatial position obtained by backprojection of the pixel and the spatial position of the original point corresponding to the pixel, as follows:

[0103] The corresponding position of the current pixel (i,j) can be denoted as (φj,i), then the actual projected position (φ',i') of the spatial point corresponding to the current pixel is:

number

[0104] We can transform the current pixel back to the Cartesian coordinate system using the regularization parameter and the following formula to get its corresponding Cartesian coordinate (xl,yl,zl):

number

[0105] The spatial point (x, y, z) corresponding to the current pixel is calculated according to the spatial position (xl, yl, zl) obtained by backprojecting the current pixel and the coordinate conversion error (Δx, Δy, Δz) as follows:

number

[0106] For each non-empty pixel in the two-dimensional projected structure, the corresponding spatial point can be reconstructed according to the above equation to obtain a reconstructed point cloud.

[0107] In addition, when the point cloud is reconstructed on the decoder side, the reconstruction method can be adaptively selected according to the encoding method of the geometry information and attribute information of the point cloud at the encoder end to obtain the corresponding reconstructed point cloud.

[0108] EMBODIMENT 6 According to embodiment 5, this embodiment provides a point cloud decoding device based on 2D regularized planar projection. Figure 10 is a schematic configuration diagram of a point cloud decoding device based on 2D regularized planar projection according to an embodiment of the present invention, the device includes: a second data acquisition module 21 configured to acquire codestream information and to decode the codestream information to obtain parsed data; a first reconstruction module 22 configured to reconstruct a plurality of two-dimensional image information according to the analyzed data; a second reconstruction module 23 configured to obtain a two-dimensional projection plane structure according to the plurality of two-dimensional image information; and a point cloud reconstruction module 24 configured to reconstruct a point cloud using the two-dimensional projection plane structure.

[0109] The decoding device provided in this embodiment can implement the decoding method in embodiment 5, and the detailed process is not described again here.

[0110] The above is a detailed description of the present invention with reference to specific exemplary embodiments, and the specific implementation of the present invention should not be considered as being limited to these descriptions. Those skilled in the art to which the present invention belongs can further make some simple deductions or substitutions without departing from the concept of the present invention, and all such deductions or substitutions should be considered as being within the protection scope of the present invention.

Claims

1. Obtaining original point cloud data; performing a two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure; obtaining a plurality of two-dimensional image information according to the two-dimensional projection plane structure, the plurality of two-dimensional image information having a geometry information map; encoding the plurality of two-dimensional image information to obtain codestream information; A point cloud encoding method based on a two-dimensional regularized planar projection with

2. performing a two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure, initializing the two-dimensional projection plane structure; determining a mapping relationship between the original point cloud data and the two-dimensional projection planar structure, and projecting the original point cloud data onto the two-dimensional projection planar structure; The method of claim 1 , further comprising:

3. The method of claim 1 , wherein the geometry information map comprises a placeholder information map.

4. Encoding the plurality of two-dimensional image information to obtain code stream information includes: encoding the placeholder information map to obtain a placeholder information codestream; obtaining a geometry information codestream according to the placeholder information codestream; The method of claim 3 , further comprising:

5. After obtaining the geometry information codestream, the method further comprises: performing geometry reconstruction according to the geometry information codestream to obtain reconstructed point cloud geometry information; encoding attribute information of the original point cloud data based on the reconstructed point cloud geometry information to obtain an attribute information codestream; The method of claim 4 , comprising:

6. The method for encoding point clouds based on two-dimensional regularized plane projection according to claim 3 , wherein the plurality of two-dimensional image information further comprises an attribute information map.

7. The step of encoding the plurality of two-dimensional image information to obtain codestream information further comprises: encoding the attribute information map to obtain an attribute information codestream; The method of claim 6 , comprising:

8. a first data acquisition module (11) configured to acquire original point cloud data; a projection module (12) configured to perform a two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure; a data processing module (13) configured to obtain a plurality of two-dimensional image information according to the two-dimensional projection plane structure, the plurality of two-dimensional image information having a geometry information map; an encoding module (14) configured to encode the plurality of two-dimensional image information to obtain codestream information; A point cloud encoding device based on a 2D regularized planar projection with

9. obtaining codestream information and decoding the codestream information to obtain parsed data; reconstructing a plurality of two-dimensional image information according to the analyzed data; Obtaining a two-dimensional projection plane structure according to the plurality of two-dimensional image information, the plurality of two-dimensional image information having a geometry information map; reconstructing a point cloud using the two-dimensional projected plane structure; A point cloud decoding method based on a two-dimensional regularized planar projection with

10. a second data acquisition module (21) configured to acquire codestream information and decode the codestream information to obtain parsed data; a first reconstruction module (22) configured to reconstruct a plurality of two-dimensional image information according to the analyzed data; a second reconstruction module (23) configured to obtain a two-dimensional projection plane structure according to the plurality of two-dimensional image information, the plurality of two-dimensional image information having a geometry information map; a point cloud reconstruction module (24) configured to reconstruct a point cloud using the two-dimensional projection plane structure; A point cloud decoder based on a 2D regularized planar projection with

11. A computer readable storage medium having stored thereon computer executable program instructions which, when executed on a computer, enable the computer to carry out a method according to any one of claims 1 to 7.

12. A computer readable storage medium having stored thereon computer executable program instructions which, when executed on a computer, enable the computer to carry out the method of claim 9.

Images