Point group encoding and decoding method and device based on two-dimensional regularization plane projection

Abstract

To disclose point-group encoding methods and devices based on two-dimensional regularization plane projection.SOLUTION: An encoding method according to the present invention has a step of acquiring original point group data, a step of carrying out two-dimensional regularization plane projection to acquire a two-dimensional projection plane structure, a step of acquiring a plurality of two-dimensional image information in accordance with the two-dimensional projection plane structure, and a step of acquiring code stream information. According to the present invention, the two-dimensional regularization plane projection technique is used in order that spatial correlation of the point group should be well reflected, thereby acquiring strong correlation expression of the point group on the two-dimensional projection plane structure. When a projection residual information map acquired for the two-dimensional regularization projection plane structure is successively encoded, the point group correlation can be largely used, thereby improving point group encoding efficiency. Further, to support encoding of the projection residual information map in order to further improve the encoding efficiency, a place holder information map and a depth information map are used.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] This application claims priority to Chinese Patent Application No. 202110181878.X, entitled "POINT CLOUD ENCODING AND DECODING METHOD AND DEVICE BASED ON TWO-DIMENSIONAL REGULARIZATION PLANE PROJECTION," filed with the China State Intellectual Property Office on February 8, 2021, the entire contents of which are incorporated herein by reference.

[0002] The present invention relates to the field of encoding and decoding, and in particular to a point cloud encoding and decoding method and device based on two-dimensional regularized planar projection. [Background technology]

[0003] With the improvement of hardware processing power and the rapid development of computer vision, 3D point clouds have become a new generation of immersive multimedia after audio, image, and video, and have been widely applied in virtual reality, augmented reality, autonomous driving, environmental modeling, etc. However, 3D point clouds usually have a relatively large amount of data, which is not conducive to the transmission and storage of point cloud data. Therefore, it is very important to research efficient point cloud encoding and decoding technologies.

[0004] In the Geometry-based Point Cloud Compression (G-PCC) coding framework, the geometric information and attribute information of a point cloud are coded separately. Currently, G-PCC geometric coding and decoding can be divided into octree-based geometric coding and decoding and predictive tree-based geometric coding and decoding.

[0005] Octree-based Geometric Encoding and Decoding: On the encoder side, firstly, the geometric information of the point cloud is preprocessed, which includes coordinate transformation and voxelization process of the point cloud. Then, tree partitioning (octree / quadtree / binary tree) is successively performed on the bounding box in which the point cloud is arranged in the order of breadth-first traversal. Finally, the placeholder code of each node is encoded and the number of points contained in each leaf node is encoded to generate a binary code stream. On the decoder side, firstly, the placeholder code of each node is successively obtained by parsing in the order of breadth-first traversal. Then, the tree partitioning is successively performed in sequence, and the partitioning stops until a 1x1x1 unit cube is obtained through the partitioning. Finally, the number of points contained in each leaf node is obtained by parsing, and finally the reconstructed point cloud geometric information is obtained.

[0006] Prediction tree-based geometric coding and decoding: At the encoder side, the input point cloud is first sorted. Then, a prediction tree structure is established. By classifying each point to the laser scanner to which it belongs, a prediction tree structure is established according to different laser scanners. Then, each node in the prediction tree is traversed, the geometric information of the node is predicted by selecting different prediction modes to obtain a prediction residual, and the prediction residual is quantized using a quantization parameter. Finally, the prediction tree structure, the quantization parameter, the prediction residual of the geometric information of the node, etc. are coded to generate a binary code stream. At the decoder side, the code stream is first analyzed, then the prediction tree structure is reconstructed, and then the prediction residual is dequantized based on the prediction residual of the geometric information obtained by the analysis and the quantization parameter, and finally the reconstructed geometric information of each node is restored. That is, the reconstruction of the point cloud geometric information is completed.

[0007] However, due to the relatively strong spatial sparsity of the point cloud, in the point cloud encoding technology using the octree structure, this structure results in a relatively large proportion of empty nodes obtained by division, and the spatial correlation of the point cloud cannot be fully reflected, which is not conducive to point cloud prediction and entropy coding. In the predictive tree-based point cloud encoding and decoding technology, some parameters of the lidar device are used to establish a tree structure, and the tree structure is used for predictive coding based on this. However, the tree structure does not fully reflect the spatial correlation of the point cloud, which is not conducive to point cloud prediction and entropy coding. Therefore, the above two point cloud encoding and decoding technologies have the problem that the encoding efficiency is not high enough. Summary of the Invention [Means for solving the problem]

[0008] In order to solve the above problems in the existing technology, the present invention provides a point cloud encoding and decoding method and device based on two-dimensional regularized planar projection. The technical problem to be solved in the present invention is implemented by the following technical solutions:

[0009] A point cloud encoding method based on two-dimensional regularized planar projection is provided, which includes: Obtaining original point cloud data; performing a two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure; Obtaining a plurality of two-dimensional image information according to a two-dimensional projection plane structure; Encoding the plurality of two-dimensional image information to obtain codestream information.

[0010] In one embodiment of the present invention, the plurality of two-dimensional image information includes a projection residual information map.

[0011] In one embodiment of the present invention, the step of encoding a plurality of two-dimensional image information to obtain codestream information comprises: The method includes encoding the projected residual information map to obtain a projected residual information codestream.

[0012] In one embodiment of the present invention, the step of encoding the projected residual information map to obtain a projected residual information codestream comprises: performing a prediction of pixels in the projection residual information map based on the placeholder information map and the depth information map to obtain a prediction residual of the projection residual; or performing a prediction of pixels in the projection residual information map based on the reconstructed projection residual information of the encoded and decoded pixels to obtain a prediction residual; encoding the prediction residual of the projection residual to obtain a projection residual information codestream.

[0013] In one embodiment of the present invention, the step of performing prediction of pixels in the projection residual information map based on the placeholder information map and the depth information map to obtain a prediction residual of the projection residual comprises: traversing pixels in the projected residual information map in a particular scan order and identifying encoded and decoded non-empty pixels in a neighboring region of a current non-empty pixel according to the placeholder information map; Using the encoded and decoded non-empty pixels, establishing a relationship between the depth information and the reconstructed projection residual information, and estimating the projection residual information corresponding to the current pixel according to the relationship to obtain an estimate of the projection residual of the current pixel; using the estimate as a prediction value of the projection residual of the current pixel to obtain a prediction residual of the projection residual of the current pixel.

[0014] In one embodiment of the present invention, the step of encoding the projected residual information map to obtain a projected residual information codestream comprises: performing a prediction of pixels in the projection residual information map based on the placeholder information map to obtain a prediction residual of the projection residual; or performing a prediction of pixels in the projection residual information map based on the depth information map to obtain a prediction residual of the projection residual; encoding the prediction residual of the projection residual to obtain a projection residual information codestream.

[0015] Another embodiment of the present invention further provides a point cloud encoding device based on two-dimensional regularized planar projection, including: a first data acquisition module configured to acquire original point cloud data; A projection module configured to perform a two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure; a data processing module configured to acquire a plurality of two-dimensional image information according to a two-dimensional projection plane structure; An encoding module configured to encode the plurality of two-dimensional image information to obtain codestream information.

[0016] Yet another embodiment of the present invention further provides a point cloud decoding method based on two-dimensional regularized planar projection, including: obtaining codestream information; and decoding the codestream information to obtain analysis data; Reconstructing a plurality of two-dimensional image information according to the analysis data; Obtaining a two-dimensional projection plane structure according to the plurality of two-dimensional image information; Reconstructing the point cloud using the two-dimensional projection plane structure.

[0017] In one embodiment of the present invention, the step of reconstructing the plurality of two-dimensional image information according to the analysis data includes: The method includes a step of reconstructing the projection residual information map according to a prediction residual of the projection residual information map of the analysis data to obtain a reconstructed projection residual information map.

[0018] Yet another embodiment of the present invention further provides a point cloud decoding device based on a two-dimensional regularized planar projection, including: a second data acquisition module configured to acquire codestream information and decode the codestream information to acquire analysis data; a first reconstruction module configured to reconstruct a plurality of two-dimensional image information according to the analysis data; a second reconstruction module configured to obtain a two-dimensional projection plane structure according to the plurality of two-dimensional image information; A point cloud reconstruction module configured to reconstruct the point cloud using the two-dimensional projected plane structure.

[0019] The beneficial effects of the present invention are as follows:

[0020] 1. According to the present invention, a point cloud in a three-dimensional space is projected onto a corresponding two-dimensional regularized projection plane structure, and a regularization correction is performed vertically and horizontally on the point cloud to obtain a strongly correlated representation of the point cloud on the two-dimensional projection plane structure, so that the sparsity in the three-dimensional representation structure is avoided and the spatial correlation of the point cloud is better reflected; when the multiple two-dimensional image information obtained for the two-dimensional regularized projection plane structure is subsequently encoded, the correlation of the point cloud can be greatly utilized and the spatial redundancy is reduced, thereby further improving the encoding efficiency of the point cloud.

[0021] 2. According to the present invention, a placeholder information map and a depth information map are used to assist in encoding the projection residual information map so that the encoding efficiency is improved.

[0022] 3. According to the present invention, the projected residual information map can be further used to assist in coding other two-dimensional maps to improve coding efficiency.

[0023] Hereinafter, the present invention will be further described in detail with reference to the accompanying drawings and embodiments. [Brief description of the drawings]

[0024] [Figure 1]FIG. 2 is a schematic diagram of a point cloud encoding method based on two-dimensional regularized planar projection according to an embodiment of the present invention; [Diagram 2] 2 is a schematic diagram of the correspondence between cylindrical coordinates of points and pixels of a two-dimensional projection plane according to an embodiment of the present invention; FIG. [Diagram 3] FIG. 2 is a schematic diagram of a two-dimensional projection plane structure of a point cloud, according to an embodiment of the present invention; [Figure 4] FIG. 4 is a schematic diagram of a projection residual according to an embodiment of the present invention; [Diagram 5] FIG. 1 is a block diagram of an encoding of a projection residual information map according to an embodiment of the present invention; [Figure 6] FIG. 2 is a schematic diagram of a prediction of projected residual information according to an embodiment of the present invention; [Figure 7] 4 is a flowchart of entropy coding of a prediction residual of a projection residual according to an embodiment of the present invention. [Figure 8] FIG. 2 is a schematic structural diagram of a point cloud encoding device based on two-dimensional regularized planar projection according to an embodiment of the present invention; [Figure 9] FIG. 2 is a schematic diagram of a point cloud decoding method based on two-dimensional regularized planar projection according to an embodiment of the present invention; [Figure 10] FIG. 2 is a block diagram of a placeholder information map decoding according to one embodiment of the present invention; [Figure 11] FIG. 2 is a schematic structural diagram of a point cloud decoding device based on two-dimensional regularized planar projection according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] The present invention is further described in detail below with reference to specific embodiments, but implementations of the present invention are not limited thereto.

[0026] EMBODIMENT 1 FIG. 1 is a schematic diagram of a point cloud encoding method based on two-dimensional regularized planar projection according to an embodiment of the present invention, including:

[0027] S1: Obtain the original point cloud data. Specifically, the original point cloud data typically includes a group of three-dimensional spatial points, each of which records geometric position information and additional attribute information such as color, reflectance, and normal. The geometric position information of the point cloud is generally represented based on the Cartesian coordinate system, i.e., using the coordinates x, y, and z of the point. The original point cloud data may be acquired through a 3D scanning device such as a lidar, or may be acquired based on public datasets provided by various platforms. In this embodiment, it is assumed that the geometric position information of the acquired original point cloud data is represented based on the Cartesian coordinate system. It should be noted that the representation method of the geometric position information of the original point cloud data is not limited to Cartesian coordinates.

[0028] S2: Perform two-dimensional regularized plane projection on the original point cloud data to obtain a two-dimensional projected plane structure. Specifically, in this embodiment, before the 2D regularized planar projection is performed on the original point cloud, further pre-processing such as a voxelization process may be performed on the original point cloud data to facilitate subsequent encoding.

[0029] First, a two-dimensional projection plane structure is initialized.

[0030] The initialization of the two-dimensional regularized projection plane structure of the point cloud requires the use of regularization parameters. The regularization parameters are usually carefully measured by the manufacturer and provided to the consumer as one of the required data, such as the acquisition range of the lidar, the sampling angular resolution Δφ or the number of sampling points in the horizontal azimuth angle, the distance correction coefficient of each laser scanner, the offset information V0 and H0 of the laser scanner in the vertical and horizontal directions, and the offset information θ0 and α of the laser scanner along the pitch angle and horizontal azimuth angle.

[0031] It should be noted that the regularization parameters are not limited to the parameters listed above: the given calibration parameters of the lidar may be used as the regularization parameters, or the regularization parameters may be obtained in a manner such as optimizing estimation and data fitting if the calibration parameters of the lidar are not given.

[0032] The two-dimensional regularized projected plane structure of the point cloud is a data structure that includes M rows and N columns of pixels, and the points of the three-dimensional point cloud correspond to pixels in the projected data structure. In addition, pixel (i,j) in the data structure may be associated with cylindrical coordinate components (θ,φ). For example, pixel (i,j) corresponding to cylindrical coordinates (r,θ,φ) may be found using the following formula:

number

[0033] Specifically, Figure 2 is a schematic diagram of the correspondence between the cylindrical coordinates of points and the pixels of a two-dimensional projection plane according to an embodiment of the present invention. It should be noted that the pixel correspondence in this specification is not limited to cylindrical coordinates.

[0034] Furthermore, the resolution of the two-dimensional regularized projection plane can be obtained using the regularization parameter. For example, if the resolution of the two-dimensional regularized projection plane is assumed to be M×N, the number of laser scanners in the regularization parameter can be used to initialize M, and the sampling angle resolution Δφ of the horizontal azimuth angle (or the number of sampling points of the laser scanner) is used to initialize N. For example, to obtain a plane structure including M×N pixels, the following formula can be used, and the initialization of the two-dimensional projection plane structure can be finally completed.

number

[0035] In addition, a mapping relationship between the original point cloud data and the two-dimensional projection planar structure is determined, so as to project the original point cloud data onto the two-dimensional projection planar structure.

[0036] In this part, the point cloud, which is originally irregularly distributed in the Cartesian coordinate system, is mapped onto the evenly distributed two-dimensional regularized projected plane structure by determining the position of the original point cloud in the two-dimensional projected plane structure point by point. Specifically, for each point of the original point cloud, a corresponding pixel is determined in the two-dimensional projected plane structure. For example, the pixel having the shortest spatial distance from the projected position of the point in the two-dimensional plane may be selected as the corresponding pixel of the point.

[0037] When a cylindrical coordinate system is used for two-dimensional projection, the specific process of determining the pixels corresponding to the original point cloud is as follows:

[0038] a. The cylindrical coordinate component r of the current point in the original point cloud data is determined, and specifically, the following formula is used for calculation:

number

[0039] b. The search area of ​​the current point in the two-dimensional projection plane structure is determined. Specifically, the entire two-dimensional projection plane structure can be directly selected as the search area. In addition, in order to reduce the amount of calculation, the pitch angle θ and the azimuth angle φ of the cylindrical coordinate components of the current point can be further used to determine the search area of ​​the corresponding pixel in the two-dimensional projection plane structure to reduce the search area.

[0040] c. After the search area is determined, for each pixel (i,j) in the search area, the regularization parameters, i.e., the calibration parameters θ0, V0, H0, and α of the i-th laser scanner of the lidar are used to calculate the position (xl, yl, zl) of the current pixel in the Cartesian coordinate system, and the specific calculation formula is as follows: θ i =θ0 φ j =-180°+j×Δφ xl = r sin(φ j -α)-H0·cos(φ j -α) yl=r cos(φj -α)-H0·sin(φ j -α) zl=r tan θ i +V0

[0041] d. After the position (xl, yl, zl) of the current pixel in the Cartesian coordinate system is obtained, the spatial distance between that position and the current point is calculated and used as the error Err, i.e., Err=dist{(x,y,z),(xl,yl,zl)}

[0042] If the error Err is smaller than the current minimum error minErr, the error Err is used to update the minimum error minErr, and the i and j corresponding to the current pixel are used to update the i and j of the pixel corresponding to the current point; if the error Err is larger than the minimum error minErr, the above update process is not performed.

[0043] e. After all pixels in the search area have been traversed, the corresponding pixel of the current point in the two-dimensional projected planar structure can be determined.

[0044] Once the above operations are completed for all points in the original point cloud, the two-dimensional regularized planar projection of the point cloud is completed. Specifically, Figure 3 is a schematic diagram of a two-dimensional projected planar structure of a point cloud according to an embodiment of the present invention. Each point in the original point cloud data is mapped to a corresponding pixel in the structure.

[0045] It should be noted that during the two-dimensional regularized plane projection of the point cloud, multiple points of the point cloud may correspond to the same pixel in the two-dimensional projection plane structure. To avoid this situation, these spatial points may be selected to be projected to different pixels during projection. For example, during the projection of a certain point, if the pixel corresponding to that point already has a corresponding point, the point is projected to an empty pixel adjacent to that pixel. In addition, if multiple points of the point cloud are projected to the same pixel in the two-dimensional projection plane structure, during encoding based on the two-dimensional projection plane structure, the number of corresponding points in each pixel should be additionally encoded, and the information of each corresponding point in a pixel is encoded according to the number of points.

[0046] S3: Obtain multiple pieces of two-dimensional image information according to the two-dimensional projection plane structure.

[0047] In this embodiment, the plurality of two-dimensional image information includes a projection residual information map.

[0048] Specifically, a projection residual information map is used to represent the residual between the corresponding position of each occupied pixel in the two-dimensional regularized projection plane structure and the actual projected position. Figure 4 is a schematic diagram of the projection residual according to an embodiment of the present invention.

[0049] In this embodiment, the projection residual of a pixel may be calculated as follows: Assuming that the current pixel is (i,j) and the Cartesian coordinates of the corresponding point of the current pixel are (x,y,z), the actual projected position of the point may be represented as (φ,i), which may be calculated using the following formula:

number

[0050] The corresponding position of the current pixel is (φ j , i), which can be calculated using the following formula: φ j =-180°+j×Δφ

[0051] Therefore, the projection residual (Δφ, Δi) corresponding to the current pixel may be calculated using the following equation: Δφ=φ'-φ j Δi=i'-i

[0052] Based on the above calculation, each occupied pixel in the two-dimensional regularized projection plane has a projection residual, so that a projection residual information map corresponding to the point cloud is obtained.

[0053] S4: Encode the multiple two-dimensional image information to obtain codestream information.

[0054] Correspondingly, the step of encoding a plurality of two-dimensional image information to obtain code stream information includes the step of encoding a projection residual information map to obtain a projection residual information code stream. Specifically, first, prediction needs to be performed on the projection residual information map to obtain a prediction residual of the projection residual information, and then the prediction residual is encoded.

[0055] In this embodiment, to obtain a prediction residual, a prediction may be made for a pixel in the projection residual information map using the placeholder information map, the depth information map, and the reconstructed projection residual information of the encoded and decoded pixel.

[0056] The placeholder information map is used to identify whether each pixel in the two-dimensional regularized projection plane structure is occupied or not, i.e., whether each pixel corresponds to a point of the point cloud or not. If each pixel is occupied, the pixel is called non-empty. Otherwise, the pixel is called empty. In this way, the placeholder information map of the point cloud can be obtained according to the two-dimensional projection plane structure of the point cloud. The depth information map is used to represent the distance between the corresponding point of each occupied pixel in the two-dimensional regularized projection plane structure and the coordinate origin. For example, the cylindrical coordinate component r of the point corresponding to the pixel may be used as the depth of the pixel. Based on this, each occupied pixel in the two-dimensional regularized projection plane structure has a depth value, so that a corresponding depth information map is obtained. Both the placeholder information map and the depth information map may be obtained directly from the two-dimensional projection plane structure.

[0057] FIG. 5 is a block diagram of an encoding of a projection residual information map according to an embodiment of the present invention, which specifically includes:

[0058] 41) Predict the projection residual for the pixel.

[0059] In this embodiment, the projection residual of the current pixel may be predicted based on the placeholder information map, the depth information map, and the reconstructed projection residual information of the encoded and decoded pixels of the point cloud.

[0060] Specifically, the encoded and decoded non-empty pixels in the neighboring region of the current non-empty pixel may be identified according to the placeholder information map, and then these encoded and decoded non-empty pixels may be used to establish depth information and reconstructed projection residual information, and finally, the projection residual information corresponding to the current pixel may be estimated based on the depth information of the current pixel and used as a predicted value of the projection residual of the current pixel.

[0061] More specifically, each pixel in the projection residual information map may be traversed in a specific scanning order, for example, Z-shaped scanning. Then, whether the current pixel is non-empty is determined according to the placeholder information map, and the encoded and decoded non-empty pixels in the neighboring region of the current non-empty pixel are identified according to the placeholder information map. Then, a number of reference pixels having depth information close to that of the current pixel are selected from the encoded and decoded non-empty pixels in the neighboring region of the current non-empty pixel, and the average value of the reconstructed projection residuals of all the reference pixels is used as the estimate of the projection residual of the current pixel.

[0062] FIG. 6 is a schematic diagram of a prediction of projected residual information according to an embodiment of the present invention, where ☆ represents the current pixel;

number

number

number

[0063] During prediction of the projection residual of the current pixel, the placeholder information map is first used to determine the occupancy of the coded and decoded pixels in the neighboring region of the current pixel, i.e., the dashed box, and then the non-empty pixels therein are identified, and then the coded and decoded non-empty pixels are used to simply establish a relationship between the depth information and the reconstructed projection residual information. For example, the following relationship may be established. If the depth information of two pixels is close, their projection residuals are also close, so that a pixel with depth information close to that of the current pixel can be selected from these coded and decoded non-empty pixels as a reference pixel, and the reconstructed projection residual information of the reference pixel is averaged as a prediction value of the projection residual information of the current pixel. The prediction value of the projection residual information of the current pixel is denoted as (Δφ_pred, Δi_pred), and the calculation formula is as follows:

number

[0064] Here, (Δφ j ,Δi j ), j=1,2...N are the reconstructed projection residuals of the neighboring reference pixels of the current pixel, and N is the number of reference pixels in the neighboring region. After the predicted value of the projection residual of the current pixel is obtained, the difference between the original projection residual of the current pixel and the predicted projection residual is calculated to obtain the predicted residual of the projection residual of the current pixel.

[0065] In this embodiment, alternatively, prediction may be performed separately on the pixels in the projection residual information map based on the placeholder information map to obtain the prediction residual of the projection residual, or prediction is performed on the pixels in the projection residual information map based on the depth information map to obtain the prediction residual of the projection residual. The detailed process is not described again in this specification.

[0066] According to the present invention, during the encoding of the projected residual information, the placeholder information map and the depth information map are used to assist in the prediction of the projected residual information map, thereby improving the encoding efficiency.

[0067] In another embodiment of the present invention, a conventional encoding method may alternatively be used to perform direct prediction on pixels in the projection residual information map according to the reconstructed projection residual information of the encoded and decoded pixels to obtain the prediction residual.

[0068] In addition, an optimal prediction mode may alternatively be selected from a number of pre-set prediction modes through a rate-distortion optimization model to perform prediction on pixels in the projection residual information map to obtain a prediction residual.

[0069] For example, the following six prediction modes may be set. Mode0: Direct mode, skip prediction and compression are performed directly; Mode1: Left prediction, using the left non-empty pixel as reference pixel; Mode2: Upward prediction, using the non-empty pixel above as the reference pixel; Mode3: Top-left prediction, using the top-left non-empty pixel as the reference pixel; Mode4: Top-right prediction, using the top-right non-empty pixel as the reference pixel; Mode5: Use the left, top, top-left, and top-right non-empty pixels as reference pixels. The optimal mode is selected for prediction through a rate-distortion model to obtain a prediction residual.

[0070] 42) Encode the prediction residual to obtain a projection residual information codestream.

[0071] After the prediction of the projection residual information is completed, the prediction residual needs to be coded. It should be noted that when lossy coding is performed on the projection residual information map, the prediction residual of the projection residual information needs to be quantized before coding. When lossless coding is performed on the projection residual information map, the prediction residual does not need to be quantized.

[0072] Specifically, this embodiment is implemented in a context-based entropy coding manner. For example, the entropy coding process shown in Figure 7 may be used to code the prediction residual. The specific coding process is as follows: a. First, determine whether the prediction residual information of the projection residual of the current pixel is 0; if the prediction residual information is 0, code a 0 identifier, and skip the execution of subsequent coding. b. Otherwise, determine whether the prediction residual information of the projection residual of the current pixel is 1, and if the prediction residual information is 1, encode a 1 identifier, and skip performing subsequent encoding. c. Otherwise, determine whether the prediction residual of the projection residual of the current pixel is 2, and if the prediction residual is 2, code a 2 identifier, and skip performing subsequent encoding. d. Otherwise, subtract 3 from the current prediction residual value, then determine whether the prediction residual is greater than a certain threshold, if the prediction residual is less than the certain threshold, design a context model of the current prediction residual, otherwise, perform encoding as follows:

[0073] designing a context for prediction residual information of a portion where the prediction residual is smaller than a threshold for coding;

[0074] performing exponential-Golomb coding on the prediction residual information where the prediction residual is greater than a threshold value;

[0075] At this point, the encoding of the projection residual information map is completed.

[0076] In addition, in another embodiment of the present invention, the projection residual information map may alternatively be encoded through image / video compression. Encoding solutions that may be used herein include, but are not limited to, JPEG, JPEG2000, HEIF, H.264\AVC, H.265\HEVC, etc.

[0077] In another embodiment of the invention, other information maps obtained according to the two-dimensional projection plane structure, such as a placeholder information map, a depth information map, a coordinate transformation error information map, and an attribute information map, may be further encoded to obtain corresponding code stream information.

[0078] According to the present invention, the point cloud in three-dimensional space is projected into a corresponding two-dimensional regularized projection plane structure, and a regularization correction is performed vertically and horizontally on the point cloud to obtain a strongly correlated representation of the point cloud on the two-dimensional projection plane structure, so that the sparsity in the three-dimensional representation structure is avoided and the spatial correlation of the point cloud is better reflected; when the projection residual information map and other two-dimensional image information are subsequently encoded, the correlation of the point cloud can be greatly utilized and the spatial redundancy is reduced, thereby further improving the encoding efficiency of the point cloud.

[0079] EMBODIMENT 2 According to embodiment 1, this embodiment provides a point cloud encoding device based on two-dimensional regularized planar projection. Figure 8 is a schematic structural diagram of a point cloud encoding device based on two-dimensional regularized planar projection according to an embodiment of the present invention, including: a first data acquisition module 11 configured to acquire original point cloud data; a projection module 12 configured to perform a two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure; a data processing module 13 configured to obtain a plurality of two-dimensional image information according to a two-dimensional projection plane structure; An encoding module 14 configured to encode the plurality of two-dimensional image information to obtain codestream information.

[0080] The encoding device provided in this embodiment can implement the encoding method described in embodiment 1, and the detailed process will not be described again in this specification.

[0081] EMBODIMENT 3 FIG. 9 is a schematic diagram of a point cloud decoding method based on two-dimensional regularized planar projection according to an embodiment of the present invention, the method including:

[0082] Step 1: Obtain code stream information, decode the code stream information, and obtain analysis data.

[0083] The decoder side uses corresponding existing entropy decoding techniques to obtain the compressed codestream information and perform corresponding decoding on the codestream information to obtain the analysis data.

[0084] The specific decoding process is as follows. a. First, analyze whether the prediction residual information of the projection residual of the current pixel is 0. If the prediction residual information is 0, the prediction residual of the current pixel is 0, and skip the subsequent decoding execution. b. Otherwise, analyze whether the prediction residual information of the projection residual of the current pixel is 1, and if the prediction residual information is 1, the prediction residual of the current pixel is 1, and skip performing subsequent decoding. c. Otherwise, analyze whether the prediction residual information of the projection residual of the current pixel is 2, and if the prediction residual information is 2, the prediction residual of the current pixel is 2, and skip the execution of subsequent decoding. d. Otherwise, design the corresponding context model for the prediction residual of the projection residual of the current pixel for decoding; then determine whether the prediction residual obtained by analysis is greater than a certain threshold; if the prediction residual is less than the certain threshold, skip the execution of subsequent decoding; otherwise, decode the prediction residual value of the part where the prediction residual is greater than the threshold by exponential-Golomb decoding method. Finally, add 3 to the prediction residual value as the final prediction residual of the projection residual information obtained by analysis.

[0085] It should be noted that, when the encoder side quantizes the prediction residual of the projection residual information, in this specification, the prediction residual obtained by analysis needs to be quantized.

[0086] Step 2: Reconstruct multiple two-dimensional image information according to the analysis data.

[0087] In this embodiment, step 2 may include the following steps:

[0088] The method includes a step of reconstructing the projection residual information map according to a prediction residual of the projection residual information map of the analysis data to obtain a reconstructed projection residual information map.

[0089] Specifically, at the encoder side, the two-dimensional image information includes a projection residual information map, i.e., the projection residual information map is encoded, so that the code stream information at the decoder side includes a projection residual information code stream correspondingly. More specifically, the analysis data obtained by decoding the code stream information includes a prediction residual of depth information.

[0090] In embodiment 1, the encoder side traverses the pixels in the projection residual information map in a specific operation order and encodes the projection residual information of the non-empty pixels therein, so that the prediction residual of the pixel projection residual information obtained by the decoder side is also in this order, and the decoder side can use the regularization parameter to obtain the resolution of the projection residual information map.For details, please refer to the part of S2 in embodiment 1 that initializes the two-dimensional projection plane structure.Therefore, the decoder side can know the position of the pixel that is currently to be reconstructed in the two-dimensional map according to the resolution of the projection residual information map and the placeholder information map.

[0091] Specifically, Fig. 10 is a block diagram of a depth information map according to an embodiment of the present invention. The projection residual information of the pixel currently to be reconstructed is predicted according to the placeholder information map, the depth information map, and the reconstructed projection residual information of the coded and decoded pixel, which is consistent with the prediction method on the encoder side. First, the placeholder information map is used to determine the occupancy rate of the coded and decoded pixel in the adjacent area of ​​the pixel currently to be reconstructed, and then the non-empty pixel therein is identified. Then, the relationship between the depth information of the coded and decoded non-empty pixel established by the encoder and the reconstructed projection residual information is used. If the depth information of two pixels is close, their projection residuals are also close, and a pixel having depth information close to that of the pixel currently to be reconstructed may be selected from the coded and decoded non-empty pixels as a reference pixel, and the reconstructed projection residual information of the reference pixel is averaged as a predicted value of the projection residual information of the current pixel. Then, the projection residual information of the current pixel is reconstructed according to the obtained predicted value and the predicted residual obtained by analysis. After the projection residuals of all pixels are reconstructed, a reconstructed projection residual information map is obtained.

[0092] Step 3: Obtain the two-dimensional projection plane structure according to the two-dimensional image information.

[0093] The resolution of the two-dimensional projection plane structure matches that of the projection residual information map, and the projection residual information map has been reconstructed, so that the projection residual information of each non-empty pixel in the two-dimensional projection plane structure is known to obtain the reconstructed two-dimensional projection plane structure.

[0094] Step 4: Reconstruct the point cloud using the 2D projected plane structure.

[0095] By traversing the pixels in the reconstructed 2D projection plane structure in a specific operation order, the projection residual information of each non-empty pixel is found. If the current pixel (i,j) is non-empty and its projection residual is (Δφ,Δi), other information such as depth information and coordinate transformation error information is used to reconstruct the spatial point (x,y,z) corresponding to that pixel. Specifically, the corresponding position of the current pixel (i,j) is (φ j , i), and then the regularization parameter and other information such as depth information and coordinate transformation error information (Δx, Δy, Δz) can be used to reconstruct the spatial point (x, y, z) corresponding to the current pixel. The specific calculation is as follows: φ j =-180°+j×Δφ θ i =θ0 xl = r sin(φ j -α)-H0·cos(φ j -α) yl=r cos(φ j -α)-H0·sin(φ j -α) zl=r tan θ i +V0 (x,y,z)=(xl,yl,zl)+(Δx,Δy,Δz)

[0096] Finally, for each non-empty pixel in the two-dimensional projection structure, the corresponding spatial point can be reconstructed according to the above calculation to obtain a reconstructed point cloud.

[0097] EMBODIMENT 4 According to embodiment 3, this embodiment provides a point cloud decoding device based on two-dimensional regularized planar projection. Figure 11 is a schematic structural diagram of a point cloud decoding device based on two-dimensional regularized planar projection according to an embodiment of the present invention, including: a second data acquisition module 21 configured to acquire codestream information and to decode the codestream information to acquire analysis data; a first reconstruction module 22 configured to reconstruct a plurality of two-dimensional image information according to the analysis data; a second reconstruction module 23 configured to obtain a two-dimensional projection plane structure according to the plurality of two-dimensional image information; A point cloud reconstruction module 24 configured to reconstruct the point cloud using the two-dimensional projection plane structure.

[0098] The decoding device provided in this embodiment can implement the decoding method of embodiment 5, and the detailed process will not be described again in this specification.

[0099] The above is a detailed description of the present invention with reference to specific exemplary embodiments, and the specific embodiments of the present invention should not be considered as being limited to these descriptions. Those skilled in the art to which the present invention belongs may further make some simple estimations or substitutions without departing from the concept of the present invention, and all such estimations or substitutions should be considered as being within the protection scope of the present invention. [Explanation of symbols]

[0100] 11 First Data Acquisition Module 12 Projection Module 13 Data Processing Module 14 Encoding Module 21 Second Data Acquisition Module 22 First Reconstruction Module 23 Second Reconstruction Module 24 Point Cloud Reconstruction Module

Claims

1. A point cloud encoding method based on two-dimensional regularized planar projection, comprising: Obtaining original point cloud data; performing a two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure; obtaining a plurality of two-dimensional image information according to the two-dimensional projection plane structure; encoding the plurality of two-dimensional image information to obtain codestream information.

2. The method of claim 1 , wherein the plurality of two-dimensional image information comprises a projection residual information map.

3. The step of encoding the plurality of two-dimensional image information and obtaining code stream information includes: The two-dimensional regularized planar projection based point cloud encoding method of claim 2 , further comprising: encoding the projected residual information map to obtain a projected residual information code stream.

4. The step of encoding the projected residual information map to obtain a projected residual information codestream comprises: performing a prediction of pixels in the projection residual information map based on the placeholder information map and the depth information map to obtain a prediction residual of the projection residual; or performing a prediction of pixels in said projection residual information map based on the reconstructed projection residual information of the encoded and decoded pixels to obtain a prediction residual; and encoding the prediction residual of the projection residual to obtain the projection residual information codestream.

5. The step of performing prediction of pixels in the projection residual information map based on a placeholder information map and a depth information map to obtain a prediction residual of the projection residual comprises: traversing pixels in the projection residual information map in a particular scan order and identifying encoded and decoded non-empty pixels in a neighboring region of a current non-empty pixel according to the placeholder information map; using the encoded and decoded non-empty pixels to establish a relationship between depth information and the reconstructed projection residual information, and estimating the projection residual information corresponding to the current pixel according to the relationship to obtain an estimate of the projection residual of the current pixel; and using the estimated value as a predicted value of the projection residual of the current pixel to obtain a predicted residual of the projection residual of the current pixel.

6. The step of encoding the projected residual information map to obtain a projected residual information codestream comprises: performing a prediction of pixels in said projection residual information map based on a placeholder information map to obtain a prediction residual of the projection residual; or performing a prediction of pixels in the projection residual information map based on a depth information map to obtain a prediction residual of the projection residual; and encoding the prediction residual of the projection residual to obtain the projection residual information codestream.

7. A two-dimensional regularized planar projection based point cloud encoding device, comprising: a first data acquisition module (11) configured to acquire original point cloud data; a projection module (12) configured to perform a two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure; a data processing module (13) configured to acquire a plurality of two-dimensional image information according to the two-dimensional projection plane structure; and an encoding module (14) configured to encode the plurality of two-dimensional image information to obtain codestream information.

8. A point cloud decoding method based on two-dimensional regularized planar projection, comprising: obtaining codestream information and decoding the codestream information to obtain analysis data; reconstructing a plurality of two-dimensional image information according to the analysis data; obtaining a two-dimensional projection plane structure according to the plurality of two-dimensional image information; and reconstructing a point cloud using the two-dimensional projection plane structure.

9. The step of reconstructing a plurality of two-dimensional image information according to the analysis data includes: The point cloud decoding method based on two-dimensional regularized planar projection as claimed in claim 8, further comprising: reconstructing the projection residual information map according to a prediction residual of the projection residual information map of the analysis data to obtain a reconstructed projection residual information map.

10. A two-dimensional regularized planar projection based point cloud decoding device, comprising: a second data acquisition module (21) configured to acquire codestream information and to decode said codestream information to acquire analysis data; a first reconstruction module (22) configured to reconstruct a plurality of two-dimensional image information according to the analysis data; a second reconstruction module (23) configured to obtain a two-dimensional projection plane structure according to the plurality of two-dimensional image information; and a point cloud reconstruction module (24) configured to reconstruct a point cloud using the two-dimensional projected planar structure.

Images