Point cloud encoding and decoding method and device based on two-dimensional regularized planar projection

Abstract

To disclose point-group encoding methods and devices based on two-dimensional regularization plane projection.SOLUTION: An encoding method according to the present invention has a step of acquiring original point group data, a step of carrying out two-dimensional regularization plane projection to acquire a two-dimensional projection plane structure, a step of acquiring a plurality of two-dimensional image information in accordance with the two-dimensional projection plane structure, and a step of acquiring code stream information. According to the present invention, the two-dimensional regularization plane projection technique is used in order that spatial correlation of the point group should be well reflected, thereby acquiring strong correlation expression of the point group on the two-dimensional projection plane structure. When a projection residual information map acquired for the two-dimensional regularization projection plane structure is successively encoded, the point group correlation can be largely used, thereby improving point group encoding efficiency. Further, to support encoding of the projection residual information map in order to further improve the encoding efficiency, a place holder information map and a depth information map are used.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This application claims priority from Chinese Patent Application No. 202110181878.X, entitled "POINT CLOUD ENCODING AND DECODING METHOD AND DEVICE BASED ON TWO-DIMENSIONAL REGULARIZATION PLANE PROJECTION," filed with the State Intellectual Property Office of the People's Republic of China on February 8, 2021, which is incorporated herein by reference in its entirety.

[0002] The present invention relates to the field of encoding and decoding, and in particular to a point cloud encoding and decoding method and device based on two-dimensional regularized planar projection. [Background technology]

[0003] With the improvement of hardware processing power and the rapid development of computer vision, 3D point clouds have become a new generation of immersive multimedia after audio, images, and videos, and are widely applied in virtual reality, augmented reality, autonomous driving, environmental modeling, etc. However, 3D point clouds usually contain a relatively large amount of data, which makes transmission and storage of point cloud data unconvenient. Therefore, research into efficient point cloud encoding and decoding techniques is of great importance.

[0004] In the geometry-based point cloud compression (G-PCC) coding framework, the geometry and attribute information of a point cloud are coded separately. Currently, G-PCC geometry coding and decoding can be divided into octree-based geometry coding and decoding and predictive tree-based geometry coding and decoding.

[0005] Octree-based Geometry Encoding and Decoding: On the encoder side, the geometric information of the point cloud is first preprocessed, which includes coordinate transformation and voxelization of the point cloud. Then, tree partitioning (octree / quadtree / binary tree) is sequentially performed on the bounding box of the point cloud arranged in the order of breadth-first traversal. Finally, the placeholder code of each node is encoded, and the number of points contained in each leaf node is encoded to generate a binary code stream. On the decoder side, the placeholder code of each node is first sequentially obtained by parsing in the order of breadth-first traversal. Then, tree partitioning is sequentially performed sequentially, and the partitioning stops until a 1x1x1 unit cube is obtained through the partitioning. Finally, the number of points contained in each leaf node is obtained through the parsing, and finally, the reconstructed point cloud geometric information is obtained.

[0006] Prediction tree-based geometry coding and decoding: At the encoder side, the input point cloud is first sorted. Then, a prediction tree structure is established. The prediction tree structure is established according to the different laser scanners by classifying each point to the laser scanner to which it belongs. Next, each node in the prediction tree is traversed, and the geometric information of the node is predicted by selecting different prediction modes to obtain a prediction residual. The prediction residual is then quantized using a quantization parameter. Finally, the prediction tree structure, quantization parameter, and prediction residual of the geometric information of the node are coded to generate a binary code stream. At the decoder side, the code stream is first analyzed, then a prediction tree structure is reconstructed, and then the prediction residual is dequantized based on the geometric information prediction residual and quantization parameter obtained by the analysis. Finally, the reconstructed geometric information of each node is restored. That is, the reconstruction of the point cloud geometric information is completed.

[0007] However, due to the relatively strong spatial sparsity of point clouds, point cloud coding techniques using an octree structure result in a relatively large proportion of empty nodes obtained by division, and the spatial correlation of the point cloud cannot be fully reflected, which is not useful for point cloud prediction and entropy coding. In predictive tree-based point cloud coding and decoding techniques, several parameters of the lidar device are used to establish a tree structure, which is then used for predictive coding based on this. However, the tree structure does not fully reflect the spatial correlation of the point cloud, which is not useful for point cloud prediction and entropy coding. Therefore, the two point cloud coding and decoding techniques mentioned above have the problem of insufficient coding efficiency. Summary of the Invention [Means for solving the problem]

[0008] In order to solve the above-mentioned problems in the existing technology, the present invention provides a point cloud encoding and decoding method and device based on two-dimensional regularized planar projection. The technical problem to be solved in the present invention is implemented by the following technical solutions:

[0009] A point cloud encoding method based on two-dimensional regularized planar projection is provided, including: obtaining original point cloud data; performing two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure; acquiring a plurality of two-dimensional image information according to a two-dimensional projection plane structure; Encoding the plurality of two-dimensional image information to obtain codestream information.

[0010] In one embodiment of the present invention, the plurality of two-dimensional image information includes a projection residual information map.

[0011] In one embodiment of the present invention, the step of encoding a plurality of pieces of two-dimensional image information to obtain codestream information comprises: Encoding the projection residual information map to obtain a projection residual information codestream.

[0012] In one embodiment of the present invention, the step of encoding the projection residual information map to obtain a projection residual information codestream comprises: performing a prediction of pixels in the projection residual information map based on the placeholder information map and the depth information map to obtain a prediction residual of the projection residual; or performing prediction of pixels in the projection residual information map based on the reconstructed projection residual information of the coded and decoded pixels to obtain a prediction residual; encoding the prediction residual of the projection residual to obtain a projection residual information codestream.

[0013] In one embodiment of the present invention, the step of performing prediction of pixels in the projection residual information map based on the placeholder information map and the depth information map to obtain a prediction residual of the projection residual comprises: traversing pixels in the projection residual information map in a specific scan order and identifying coded and decoded non-empty pixels in a neighboring region of a current non-empty pixel according to the placeholder information map; Using the encoded and decoded non-empty pixels, establishing a relationship between the depth information and the reconstructed projection residual information, and estimating the projection residual information corresponding to the current pixel according to the relationship to obtain an estimate of the projection residual of the current pixel; using the estimated value as a prediction value of the projection residual of the current pixel to obtain a prediction residual of the projection residual of the current pixel.

[0014] In one embodiment of the present invention, the step of encoding the projection residual information map to obtain a projection residual information codestream comprises: performing prediction of pixels in the projection residual information map based on the placeholder information map to obtain a prediction residual of the projection residual; or performing prediction of pixels in the projection residual information map based on the depth information map to obtain a prediction residual of the projection residual; encoding the prediction residual of the projection residual to obtain a projection residual information codestream.

[0015] Another embodiment of the present invention further provides a point cloud encoding device based on two-dimensional regularized planar projection, including: a first data acquisition module configured to acquire original point cloud data; a projection module configured to perform two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure; a data processing module configured to acquire a plurality of two-dimensional image information according to a two-dimensional projection plane structure; An encoding module configured to encode the plurality of two-dimensional image information to obtain codestream information.

[0016] Yet another embodiment of the present invention further provides a point cloud decoding method based on two-dimensional regularized planar projection, including: obtaining codestream information; and decoding the codestream information to obtain analysis data; Reconstructing a plurality of two-dimensional image information according to the analysis data; Obtaining a two-dimensional projection plane structure according to the plurality of two-dimensional image information; Reconstructing the point cloud using a two-dimensional projection plane structure.

[0017] In one embodiment of the present invention, the step of reconstructing a plurality of two-dimensional image information according to the analysis data includes: The method includes reconstructing the projection residual information map according to a prediction residual of the projection residual information map of the analysis data to obtain a reconstructed projection residual information map.

[0018] Yet another embodiment of the present invention further provides a point cloud decoding device based on two-dimensional regularized planar projection, including: a second data acquisition module configured to acquire codestream information and decode the codestream information to acquire analysis data; a first reconstruction module configured to reconstruct a plurality of two-dimensional image information according to the analysis data; a second reconstruction module configured to obtain a two-dimensional projection plane structure according to the plurality of two-dimensional image information; A point cloud reconstruction module configured to reconstruct the point cloud using a two-dimensional projected plane structure.

[0019] The beneficial effects of the present invention are as follows:

[0020] 1. According to the present invention, a point cloud in a three-dimensional space is projected onto a corresponding two-dimensional regularized projection plane structure, and regularization correction is performed vertically and horizontally on the point cloud to obtain a strongly correlated representation of the point cloud on the two-dimensional projection plane structure, so that sparsity in the three-dimensional representation structure is avoided and the spatial correlation of the point cloud is better reflected. When the multiple two-dimensional image information obtained for the two-dimensional regularized projection plane structure is subsequently encoded, the correlation of the point cloud can be greatly utilized and spatial redundancy is reduced, thereby further improving the encoding efficiency of the point cloud.

[0021] 2. According to the present invention, a placeholder information map and a depth information map are used to assist in encoding the projection residual information map so that the encoding efficiency is improved.

[0022] 3. According to the present invention, the projection residual information map can be further used to assist in coding other two-dimensional maps to improve coding efficiency.

[0023] In the following, the present invention will be further described in detail with reference to the accompanying drawings and embodiments. [Brief explanation of the drawings]

[0024] [Figure 1]FIG. 1 is a schematic diagram of a point cloud encoding method based on two-dimensional regularized planar projection according to an embodiment of the present invention; [Figure 2] 2 is a schematic diagram of the correspondence between cylindrical coordinates of points and pixels of a two-dimensional projection plane according to an embodiment of the present invention; [Figure 3] FIG. 2 is a schematic diagram of a two-dimensional projection plane structure of a point cloud, according to an embodiment of the present invention; [Figure 4] FIG. 4 is a schematic diagram of a projection residual, according to an embodiment of the present invention; [Figure 5] FIG. 1 is a block diagram of a projection residual information map encoding according to an embodiment of the present invention; [Figure 6] FIG. 1 is a schematic illustration of a prediction of projection residual information according to an embodiment of the present invention; [Figure 7] 1 is a flowchart of entropy coding of a prediction residual of a projection residual according to one embodiment of the present invention. [Figure 8] FIG. 1 is a schematic structural diagram of a point cloud encoding device based on two-dimensional regularized planar projection according to an embodiment of the present invention; [Figure 9] FIG. 1 is a schematic diagram of a point cloud decoding method based on two-dimensional regularized planar projection according to an embodiment of the present invention; [Figure 10] FIG. 10 is a block diagram of a placeholder information map decoding according to one embodiment of the present invention. [Figure 11] FIG. 1 is a schematic structural diagram of a point cloud decoding device based on two-dimensional regularized planar projection according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0025] Although the present invention is further described in detail below with reference to specific embodiments, implementations of the present invention are not limited thereto.

[0026] Embodiment 1 FIG. 1 is a schematic diagram of a point cloud encoding method based on two-dimensional regularized planar projection according to an embodiment of the present invention, including:

[0027] S1: Obtain the original point cloud data. Specifically, the original point cloud data typically includes a group of three-dimensional spatial points, each of which records geometric position information and additional attribute information such as color, reflectance, and normal. The geometric position information of the point cloud is generally represented based on a Cartesian coordinate system, i.e., using point coordinates x, y, and z. The original point cloud data may be acquired through a 3D scanning device such as a lidar, or may be acquired based on public datasets provided by various platforms. In this embodiment, it is assumed that the geometric position information of the acquired original point cloud data is represented based on a Cartesian coordinate system. It should be noted that the method of expressing the geometric position information of the original point cloud data is not limited to Cartesian coordinates.

[0028] S2: Perform two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure. Specifically, in this embodiment, before the two-dimensional regularized planar projection is performed on the original point cloud, further pre-processing such as voxelization processing may be performed on the original point cloud data to facilitate subsequent encoding.

[0029] First, a two-dimensional projection plane structure is initialized.

[0030] The initialization of the two-dimensional regularized projection plane structure of the point cloud requires the use of regularization parameters, which are usually carefully measured by the manufacturer and provided to the consumer as one of the required data, such as the lidar acquisition range, the sampling angular resolution Δφ or the number of sampling points in the horizontal azimuth angle, the distance correction coefficients of each laser scanner, the laser scanner offset information in the vertical and horizontal directions V0 and H0, and the laser scanner offset information along the pitch angle and horizontal azimuth angle θ0 and α.

[0031] It should be noted that the regularization parameters are not limited to the parameters listed above: given calibration parameters of the lidar may be used as the regularization parameters, or if the calibration parameters of the lidar are not given, the regularization parameters may be obtained in a manner such as by optimizing estimation and data fitting.

[0032] The two-dimensional regularized projected plane structure of the point cloud is a data structure containing M rows and N columns of pixels, and the points of the three-dimensional point cloud correspond to the pixels in the projected data structure. In addition, pixel (i,j) in the data structure may be associated with cylindrical coordinate components (θ,φ). For example, pixel (i,j) corresponding to cylindrical coordinates (r,θ,φ) may be found using the following equation:

number

[0033] Specifically, Figure 2 is a schematic diagram of the correspondence between cylindrical coordinates of points and pixels of a two-dimensional projection plane according to an embodiment of the present invention. Note that the pixel correspondence herein is not limited to cylindrical coordinates.

[0034] Furthermore, the resolution of the two-dimensional regularized projection plane can be obtained using a regularization parameter. For example, if the resolution of the two-dimensional regularized projection plane is assumed to be M×N, the number of laser scanners in the regularization parameter may be used to initialize M, and the sampling angle resolution Δφ of the horizontal azimuth angle (or the number of sampling points of the laser scanner) is used to initialize N. For example, to obtain a plane structure including M×N pixels, the following equation may be used, and the initialization of the two-dimensional projection plane structure can be finally completed.

number

[0035] In addition, a mapping relationship between the original point cloud data and the two-dimensional projection plane structure is determined so as to project the original point cloud data onto the two-dimensional projection plane structure.

[0036] In this step, the points that are originally irregularly distributed in the Cartesian coordinate system are mapped onto the evenly distributed two-dimensional regularized projection plane structure by determining the positions of the original points in the two-dimensional projection plane structure one by one. Specifically, for each point in the original point cloud, a corresponding pixel is determined in the two-dimensional projection plane structure. For example, the pixel with the shortest spatial distance from the projected position of the point in the two-dimensional plane may be selected as the corresponding pixel of the point.

[0037] When a cylindrical coordinate system is used for two-dimensional projection, the specific process of determining the pixels corresponding to the original point cloud is as follows:

[0038] a. The cylindrical coordinate component r of the current point in the original point cloud data is determined, and specifically, the following formula is used for calculation:

number

[0039] b. The search area of ​​the current point in the two-dimensional projection plane structure is determined. Specifically, the entire two-dimensional projection plane structure can be directly selected as the search area. In addition, to reduce the amount of calculation, the pitch angle θ and the azimuth angle φ of the cylindrical coordinate components of the current point can be further used to determine the search area of ​​the corresponding pixel in the two-dimensional projection plane structure to reduce the search area.

[0040] c. After the search area is determined, for each pixel (i, j) in the search area, the regularization parameters, i.e., the calibration parameters θ0, V0, H0, and α of the i-th laser scanner of the lidar are used to calculate the current pixel position (xl, yl, zl) in the Cartesian coordinate system, and the specific calculation formula is as follows: θ i =θ0 φ j =-180°+j×Δφ xl=r·sin(φ j -α)-H0·cos(φ j -α) yl=r·cos(φj -α)-H0·sin(φ j -α) zl=r tanθ i +V0

[0041] d. After obtaining the current pixel position (xl, yl, zl) in the Cartesian coordinate system, the spatial distance between that position and the current point is calculated and used as the error Err, i.e., Err=dist{(x,y,z),(xl,yl,zl)}

[0042] If the error Err is smaller than the current minimum error minErr, the error Err is used to update the minimum error minErr, and the i and j corresponding to the current pixel are used to update the i and j of the pixel corresponding to the current point; if the error Err is larger than the minimum error minErr, the above update process is not performed.

[0043] e. After all pixels in the search area have been traversed, the corresponding pixel of the current point in the two-dimensional projection plane structure can be determined.

[0044] Once the above operations have been completed for all points in the original point cloud, the two-dimensional regularized planar projection of the point cloud is complete. Specifically, Figure 3 is a schematic diagram of a two-dimensional projected planar structure of a point cloud according to one embodiment of the present invention. Each point in the original point cloud data is mapped to a corresponding pixel in the structure.

[0045] It should be noted that during the two-dimensional regularized plane projection of the point cloud, multiple points of the point cloud may correspond to the same pixel in the two-dimensional projection plane structure. To avoid this situation, these spatial points may be selected to be projected to different pixels during projection. For example, during the projection of a specific point, if the pixel corresponding to that point already has a corresponding point, the point is projected to an empty pixel adjacent to that pixel. In addition, if multiple points of the point cloud are projected to the same pixel in the two-dimensional projection plane structure, the number of corresponding points in each pixel should be additionally encoded during encoding based on the two-dimensional projection plane structure, and the information of each corresponding point in the pixel is encoded according to the number of points.

[0046] S3: Obtain multiple pieces of two-dimensional image information according to the two-dimensional projection plane structure.

[0047] In this embodiment, the plurality of two-dimensional image information includes a projection residual information map.

[0048] Specifically, a projection residual information map is used to represent the residual between the corresponding position of each occupied pixel in the two-dimensional regularized projection plane structure and its actual projected position. Figure 4 is a schematic diagram of the projection residual according to one embodiment of the present invention.

[0049] In this embodiment, the projection residual of a pixel may be calculated as follows: Assuming the current pixel is (i,j) and the Cartesian coordinates of the corresponding point of the current pixel are (x,y,z), the actual projected position of the point may be represented as (φ,i), which may be calculated using the following formula:

number

[0050] The corresponding position of the current pixel is (φ j , i), which can be calculated using the following formula: φ j =-180°+j×Δφ

[0051] Therefore, the projection residual (Δφ, Δi) corresponding to the current pixel may be calculated using the following equation: Δφ=φ'-φ j Δi=i'-i

[0052] Based on the above calculation, each occupied pixel in the two-dimensional regularized projection plane has a projection residual, so that a projection residual information map corresponding to the point cloud is obtained.

[0053] S4: Encode the plurality of two-dimensional image information to obtain codestream information.

[0054] Correspondingly, the step of encoding the plurality of 2D image information to obtain the codestream information includes the step of encoding the projection residual information map to obtain the projection residual information codestream. Specifically, first, prediction needs to be performed on the projection residual information map to obtain the prediction residual of the projection residual information, and then the prediction residual is encoded.

[0055] In this embodiment, predictions may be made for pixels in the projection residual information map using the placeholder information map, the depth information map, and the reconstructed projection residual information of the encoded and decoded pixels to obtain prediction residuals.

[0056] The placeholder information map is used to identify whether each pixel in the two-dimensional regularized projection plane structure is occupied, i.e., whether each pixel corresponds to a point in the point cloud. If each pixel is occupied, the pixel is called non-empty. Otherwise, the pixel is called empty. In this way, the placeholder information map of the point cloud can be obtained according to the two-dimensional projection plane structure of the point cloud. The depth information map is used to represent the distance between the corresponding point of each occupied pixel in the two-dimensional regularized projection plane structure and the coordinate origin. For example, the cylindrical coordinate component r of the point corresponding to the pixel may be used as the depth of the pixel. Based on this, each occupied pixel in the two-dimensional regularized projection plane structure has a depth value, so that a corresponding depth information map is obtained. Both the placeholder information map and the depth information map may be obtained directly from the two-dimensional projection plane structure.

[0057] FIG. 5 is a block diagram of a projection residual information map encoding method according to an embodiment of the present invention, which specifically includes:

[0058] 41) Predict the projection residual of the pixel.

[0059] In this embodiment, the projection residual of the current pixel may be predicted based on the placeholder information map, the depth information map, and the reconstructed projection residual information of the encoded and decoded pixels of the point cloud.

[0060] Specifically, the coded and decoded non-empty pixels in the neighboring area of ​​the current non-empty pixel may be identified according to the placeholder information map, and then these coded and decoded non-empty pixels may be used to establish depth information and reconstructed projection residual information, and finally, the projection residual information corresponding to the current pixel may be estimated based on the depth information of the current pixel and used as a predicted value of the projection residual of the current pixel.

[0061] More specifically, each pixel in the projection residual information map may be traversed in a specific scanning order, for example, a Z-shaped scan. Then, whether the current pixel is non-empty is determined according to the placeholder information map, and coded and decoded non-empty pixels in the neighboring region of the current non-empty pixel are identified according to the placeholder information map. Then, multiple reference pixels having depth information close to that of the current pixel are selected from the coded and decoded non-empty pixels in the neighboring region of the current non-empty pixel, and the average value of the reconstructed projection residuals of all the reference pixels is used as an estimate of the projection residual of the current pixel.

[0062] FIG. 6 is a schematic diagram of a projection residual information prediction according to an embodiment of the present invention, where ☆ represents the current pixel;

number

number

number

[0063] During the prediction of the projection residual of the current pixel, the placeholder information map is first used to determine the occupancy of the coded and decoded pixels in the neighboring region of the current pixel, i.e., the dashed box, and then the non-empty pixels therein are identified. Then, the coded and decoded non-empty pixels are used to simply establish a relationship between the depth information and the reconstructed projection residual information. For example, the following relationship may be established: If the depth information of two pixels is close, their projection residuals are also close, so that a pixel having depth information close to that of the current pixel can be selected from these coded and decoded non-empty pixels as a reference pixel, and the reconstructed projection residual information of the reference pixel is averaged as a predicted value of the projection residual information of the current pixel. The predicted value of the projection residual information of the current pixel is denoted as (Δφ_pred, Δi_pred), and its calculation formula is as follows:

number

[0064] Here, (Δφ j ,Δi j ), j=1,2...N are the reconstructed projection residuals of the neighboring reference pixels of the current pixel, and N is the number of reference pixels in the neighboring region. After the predicted value of the projection residual of the current pixel is obtained, the difference between the original projection residual of the current pixel and the predicted projection residual is calculated to obtain the predicted residual of the projection residual of the current pixel.

[0065] In this embodiment, alternatively, prediction may be performed separately on pixels in the projection residual information map based on the placeholder information map to obtain prediction residuals of the projection residuals, or prediction is performed on pixels in the projection residual information map based on the depth information map to obtain prediction residuals of the projection residuals. The detailed process will not be described again in this specification.

[0066] According to the present invention, during the encoding of the projection residual information, the placeholder information map and the depth information map are used to assist in the prediction of the projection residual information map, thereby improving the encoding efficiency.

[0067] In another embodiment of the present invention, a conventional encoding method may alternatively be used to perform direct prediction on pixels in the projection residual information map according to the reconstructed projection residual information of the coded and decoded pixels to obtain the prediction residual.

[0068] In addition, an optimal prediction mode may alternatively be selected from a plurality of pre-set prediction modes through a rate-distortion optimization model to perform prediction on pixels in the projection residual information map to obtain a prediction residual.

[0069] For example, the following six prediction modes may be set. Mode0: Direct mode, skip prediction and compression directly performed; Mode1: Left prediction, using the left non-empty pixel as the reference pixel; Mode2: Upward prediction, using the non-empty pixel above as the reference pixel; Mode3: Top-left prediction, using the top-left non-empty pixel as the reference pixel; Mode4: Top-right prediction, using the top-right non-empty pixel as the reference pixel; Mode 5: Use the left, top, top-left, and top-right non-empty pixels as reference pixels. The optimal mode is selected for prediction through a rate-distortion model to obtain a prediction residual.

[0070] 42) Encode the prediction residual to obtain a projection residual information codestream.

[0071] After the prediction of the projection residual information is completed, the prediction residual needs to be coded. It should be noted that when lossy coding is performed on the projection residual information map, the prediction residual of the projection residual information needs to be quantized before coding. When lossless coding is performed on the projection residual information map, the prediction residual does not need to be quantized.

[0072] Specifically, this embodiment is implemented in a context-based entropy coding manner. For example, the entropy coding process shown in Figure 7 may be used to code the prediction residual. The specific coding process is as follows: a. First, determine whether the prediction residual information of the projection residual of the current pixel is 0. If the prediction residual information is 0, encode a 0 identifier and skip the subsequent encoding. b. Otherwise, determine whether the prediction residual information of the projection residual of the current pixel is 1, and if the prediction residual information is 1, encode a 1 identifier and skip the subsequent encoding execution. c. Otherwise, determine whether the prediction residual of the projection residual of the current pixel is 2; if the prediction residual is 2, encode a 2 identifier, and skip the subsequent encoding execution. d. Otherwise, subtract 3 from the current prediction residual value, then determine whether the prediction residual is greater than a certain threshold, if the prediction residual is less than the certain threshold, design a context model for the current prediction residual, otherwise, perform encoding as follows:

[0073] designing a context for prediction residual information of a portion where the prediction residual is smaller than a threshold for coding;

[0074] A step of performing exponential-Golomb coding on the prediction residual information where the prediction residual is greater than a threshold value.

[0075] At this point, the encoding of the projection residual information map is complete.

[0076] Additionally, in another embodiment of the present invention, the projection residual information map may alternatively be encoded through image / video compression. Encoding solutions that may be used herein include, but are not limited to, JPEG, JPEG2000, HEIF, H.264\AVC, H.265\HEVC, etc.

[0077] In another embodiment of the invention, other information maps obtained according to the two-dimensional projection plane structure, such as a placeholder information map, a depth information map, a coordinate transformation error information map, and an attribute information map, may be further encoded to obtain corresponding codestream information.

[0078] According to the present invention, the point cloud in three-dimensional space is projected onto a corresponding two-dimensional regularized projection plane structure, and vertical and horizontal regularization correction is performed on the point cloud to obtain a strongly correlated representation of the point cloud on the two-dimensional projection plane structure, so that the sparsity in the three-dimensional representation structure is avoided and the spatial correlation of the point cloud is better reflected; when the projection residual information map and other two-dimensional image information are subsequently encoded, the correlation of the point cloud can be greatly utilized and the spatial redundancy is reduced, thereby further improving the encoding efficiency of the point cloud.

[0079] Embodiment 2 According to embodiment 1, this embodiment provides a point cloud encoding device based on two-dimensional regularized planar projection. Figure 8 is a schematic structural diagram of a point cloud encoding device based on two-dimensional regularized planar projection according to an embodiment of the present invention, including: a first data acquisition module 11 configured to acquire original point cloud data; a projection module 12 configured to perform two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure; a data processing module 13 configured to acquire a plurality of two-dimensional image information according to a two-dimensional projection plane structure; An encoding module 14 configured to encode the plurality of two-dimensional image information to obtain codestream information.

[0080] The encoding device provided in this embodiment can implement the encoding method described in embodiment 1, and the detailed process will not be described again in this specification.

[0081] Embodiment 3 FIG. 9 is a schematic diagram of a point cloud decoding method based on two-dimensional regularized planar projection according to an embodiment of the present invention, the method including:

[0082] Step 1: Obtain codestream information, decode the codestream information, and obtain analysis data.

[0083] The decoder side uses corresponding existing entropy decoding techniques to obtain the compressed codestream information and perform corresponding decoding on the codestream information to obtain the analysis data.

[0084] The specific decoding process is as follows: a. First, analyze whether the prediction residual information of the projection residual of the current pixel is 0. If the prediction residual information is 0, the prediction residual of the current pixel is 0, and skip the subsequent decoding. b. Otherwise, analyze whether the prediction residual information of the projection residual of the current pixel is 1. If the prediction residual information is 1, the prediction residual of the current pixel is 1, and skip the subsequent decoding. c. Otherwise, analyze whether the prediction residual information of the projection residual of the current pixel is 2. If the prediction residual information is 2, the prediction residual of the current pixel is 2, and skip the subsequent decoding. d. Otherwise, the corresponding context model for the prediction residual of the projection residual of the current pixel is designed for decoding. Then, determine whether the prediction residual obtained by analysis is greater than a certain threshold. If the prediction residual is smaller than the certain threshold, skip the subsequent decoding execution. If not, decode the prediction residual value of the part where the prediction residual is greater than the threshold using the exponential-Golomb decoding method. Finally, add 3 to the prediction residual value as the final prediction residual of the projection residual information obtained by analysis.

[0085] It should be noted that, if the encoder side quantizes the prediction residual of the projection residual information, in this specification, the prediction residual obtained by analysis needs to be quantized.

[0086] Step 2: Reconstruct multiple two-dimensional image information according to the analysis data.

[0087] In this embodiment, step 2 may include the following steps:

[0088] The method includes reconstructing the projection residual information map according to a prediction residual of the projection residual information map of the analysis data to obtain a reconstructed projection residual information map.

[0089] Specifically, at the encoder side, the plurality of 2D image information includes a projection residual information map, i.e., the projection residual information map is encoded, so that the codestream information at the decoder side includes a projection residual information codestream corresponding thereto. More specifically, the analysis data obtained by decoding the codestream information includes a prediction residual of depth information.

[0090] In embodiment 1, the encoder side traverses the pixels in the projection residual information map in a specific operation order and encodes the projection residual information of the non-empty pixels therein, so the prediction residuals of the pixel projection residual information obtained by the decoder side are also in this order, and the decoder side can obtain the resolution of the projection residual information map using a regularization parameter. For details, see the part of S2 in embodiment 1 that initializes the two-dimensional projection plane structure. Therefore, the decoder side can know the position of the pixel currently to be reconstructed in the two-dimensional map according to the resolution of the projection residual information map and the placeholder information map.

[0091] Specifically, Figure 10 is a block diagram of a depth information map decoding method according to an embodiment of the present invention. The projection residual information of the pixel currently being reconstructed is predicted according to the placeholder information map, the depth information map, and the reconstructed projection residual information of the coded and decoded pixel, which is consistent with the prediction method on the encoder side. First, the placeholder information map is used to determine the occupancy rate of the coded and decoded pixels in the neighboring area of ​​the pixel currently being reconstructed, and then non-empty pixels therein are identified. Next, the relationship between the depth information of the coded and decoded non-empty pixels established by the encoder and the reconstructed projection residual information is used. If the depth information of two pixels is close, their projection residuals are also close. A pixel having depth information close to that of the pixel currently being reconstructed may be selected as a reference pixel from the coded and decoded non-empty pixels, and the reconstructed projection residual information of the reference pixel is averaged as a predicted value of the projection residual information of the current pixel. Then, the projection residual information of the current pixel is reconstructed according to the obtained predicted value and the prediction residual obtained by analysis. After the projection residuals of all pixels are reconstructed, a reconstructed projection residual information map is obtained.

[0092] Step 3: Obtain the two-dimensional projection plane structure according to the two-dimensional image information.

[0093] The resolution of the two-dimensional projection plane structure is consistent with that of the projection residual information map, and the projection residual information map is reconstructed, so that the projection residual information of each non-empty pixel in the two-dimensional projection plane structure is known to obtain the reconstructed two-dimensional projection plane structure.

[0094] Step 4: Reconstruct the point cloud using the 2D projection plane structure.

[0095] By traversing the pixels in the reconstructed 2D projection plane structure in a specific operation order, the projection residual information of each non-empty pixel is found. If the current pixel (i,j) is non-empty and its projection residual is (Δφ,Δi), other information such as depth information and coordinate transformation error information is used to reconstruct the spatial point (x,y,z) corresponding to that pixel. Specifically, the corresponding position of the current pixel (i,j) is (φ j , i), and then the regularization parameter, as well as other information such as depth information and coordinate transformation error information (Δx, Δy, Δz), can be used to reconstruct the spatial point (x, y, z) corresponding to the current pixel. The specific calculation is as follows: φ j =-180°+j×Δφ θ i =θ0 xl=r·sin(φ j -α)-H0·cos(φ j -α) yl=r·cos(φ j -α)-H0·sin(φ j -α) zl=r tanθ i +V0 (x,y,z)=(xl,yl,zl)+(Δx,Δy,Δz)

[0096] Finally, for each non-empty pixel in the two-dimensional projection structure, the corresponding spatial point can be reconstructed according to the above calculation to obtain a reconstructed point cloud.

[0097] Embodiment 4 According to embodiment 3, this embodiment provides a point cloud decoding device based on two-dimensional regularized planar projection. Figure 11 is a schematic structural diagram of a point cloud decoding device based on two-dimensional regularized planar projection according to an embodiment of the present invention, including: a second data acquisition module 21 configured to acquire codestream information and decode the codestream information to acquire analysis data; a first reconstruction module 22 configured to reconstruct a plurality of two-dimensional image information according to the analysis data; a second reconstruction module 23 configured to obtain a two-dimensional projection plane structure according to the plurality of two-dimensional image information; a point cloud reconstruction module 24 configured to reconstruct the point cloud using the two-dimensional projection plane structure;

[0098] The decoding device provided in this embodiment is 1 can be implemented, and the detailed process will not be described again in this specification.

[0099] The above content is a detailed description of the present invention with reference to specific exemplary embodiments, and the specific embodiments of the present invention should not be considered limited to these descriptions. Those skilled in the art to which the present invention pertains may further make some simple deductions or substitutions without departing from the concept of the present invention, and all such deductions or substitutions should be considered to be within the protection scope of the present invention. [Explanation of symbols]

[0100] 11 First Data Acquisition Module 12 Projection Module 13 Data Processing Module 14 Encoding Module 21 Second Data Acquisition Module 22 First Reconstruction Module 23 Second Reconstruction Module 24 Point Cloud Reconstruction Module

Claims

1. A point cloud encoding method, comprising: obtaining original point cloud data; acquiring a plurality of pieces of two-dimensional image information according to the original point cloud data, the plurality of pieces of two-dimensional image information including azimuth angle projection residual information, the azimuth angle being φ among coordinate components obtained by transforming coordinates (x, y, z) in a Cartesian coordinate system into coordinates (r, θ, φ) in a cylindrical coordinate system, the azimuth angle projection residual information being used to represent a residual between an azimuth angle corresponding to a position of each occupied pixel in a two-dimensional projected planar structure and the azimuth angle corresponding to an actual projected position of a point corresponding to the pixel; performing a prediction on the projection residual information of the azimuth angle to obtain a prediction residual of the projection residual of the azimuth angle; encoding the prediction residual of the azimuthal projection residual to obtain codestream information.

2. The point cloud encoding method according to claim 1 , wherein the plurality of pieces of two-dimensional image information include projection residual information, and the projection residual information includes projection residual information of the azimuth angle.

3. The step of encoding the prediction residual of the azimuthal projection residual to obtain codestream information comprises: The point cloud encoding method of claim 2 , comprising: encoding the prediction residual of the azimuthal projection residual to obtain a projection residual information codestream.

4. The step of performing a prediction on the projection residual information of the azimuth angle to obtain a prediction residual of the projection residual of the azimuth angle comprises: performing prediction of pixels in the projection residual information based on placeholder information and depth information to obtain a prediction residual of a projection residual, wherein the prediction residual of the projection residual comprises the prediction residual of the projection residual of the azimuthal angle; or performing prediction of pixels in the projection residual information map based on the reconstructed projection residual information of the coded and decoded pixels to obtain a prediction residual of the projection residual; The point cloud encoding method of claim 3, comprising:

5. The step of performing prediction of pixels in the projection residual information based on placeholder information and depth information to obtain a prediction residual of the projection residual includes: traversing pixels in the projection residual information map in a specific scan order and identifying coded and decoded non-empty pixels in a neighboring region of a current non-empty pixel according to a placeholder information map; using the encoded and decoded non-empty pixels to establish a relationship between depth information and reconstructed projection residual information, and estimating the projection residual information corresponding to the current pixel according to the relationship to obtain an estimate of the projection residual of the current pixel; and using the estimated value as a predicted value of the projection residual of the current pixel to obtain a prediction residual of the projection residual of the current pixel.

6. The step of performing a prediction on the projection residual information of the azimuth angle to obtain a prediction residual of the projection residual of the azimuth angle, comprising: performing a prediction on the projection residual information for the azimuth angle based on placeholder information to obtain a prediction residual of the projection residual; or performing a prediction on the azimuth projection residual information based on depth information to obtain a prediction residual of the azimuth projection residual; Including, The point cloud encoding method according to claim 3.

7. The step of performing a prediction on the projection residual information of the azimuth angle based on depth information to obtain a prediction residual of the projection residual of the azimuth angle comprises: performing a prediction on the projection residual information of the azimuth angle based on depth information to obtain a predicted projection residual for a current pixel; and obtaining a predicted residual of the projection residual of the current pixel according to a difference between an original projection residual of the current pixel and the predicted projection residual.

8. The method comprises: performing a two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure; The step of acquiring a plurality of pieces of two-dimensional image information according to the original point cloud data includes: acquiring a plurality of two-dimensional image information according to the two-dimensional projection plane structure; The point cloud encoding method according to any one of claims 1 to 7.

9. The step of performing two-dimensional regularized planar projection on the original point cloud data to obtain a two-dimensional projected planar structure, comprising: performing two-dimensional regularized planar projection on the original point cloud data according to a regularization parameter to obtain a two-dimensional projected planar structure, wherein the regularization parameter includes at least one of the following parameters: number of laser scanners, and horizontal sampling angular resolution; The point cloud encoding method of claim 8, comprising:

10. A point cloud encoding device, comprising: Get the original point cloud data, obtaining a plurality of pieces of two-dimensional image information according to the original point cloud data, wherein the plurality of pieces of two-dimensional image information includes azimuth angle projection residual information, the azimuth angle being φ among coordinate components obtained by transforming coordinates (x, y, z) in a Cartesian coordinate system into coordinates (r, θ, φ) in a cylindrical coordinate system, and the azimuth angle projection residual information is used to represent a residual between an azimuth angle corresponding to a position of each occupied pixel in the two-dimensional projected planar structure and the azimuth angle corresponding to an actual projected position of a point corresponding to the pixel; performing a prediction on the projection residual information of the azimuth angle to obtain a prediction residual of the projection residual of the azimuth angle; A point cloud encoding device configured to encode the prediction residual of the azimuthal projection residual to obtain codestream information.

11. A point cloud decoding method, comprising: obtaining codestream information and decoding the codestream information to obtain analysis data, the analysis data including a prediction residual of a projection residual of an azimuth angle, the azimuth angle being φ among coordinate components obtained by transforming coordinates (x, y, z) in a Cartesian coordinate system to coordinates (r, θ, φ) in a cylindrical coordinate system; a step of reconstructing a plurality of two-dimensional image information according to the analysis data, wherein the plurality of two-dimensional image information includes azimuth angle projection residual information, which is used to represent a residual between an azimuth angle corresponding to a position of each occupied pixel in a two-dimensional projection planar structure and the azimuth angle corresponding to an actual projection position of a point corresponding to the pixel; and reconstructing a point cloud using the plurality of two-dimensional image information.

12. The step of reconstructing a plurality of two-dimensional image information according to the analysis data includes: The point cloud decoding method according to claim 11, comprising: reconstructing the projection residual information according to a prediction residual of the projection residual information of the analysis data to obtain reconstructed projection residual information.

13. The method comprising: reconstructing the projection residual information according to a predicted residual of the projection residual information in the analysis data and depth information to obtain reconstructed projection residual information; or reconstructing the projection residual information according to the analysis data and a predicted residual of the projection residual information in the placeholder information to obtain reconstructed projection residual information. further comprising: The point cloud decoding method according to claim 12.

14. The plurality of two-dimensional image information includes placeholder information, The method comprises: reconstructing the projection residual information according to the analysis data, depth information, and predicted residual of the projection residual information in placeholder information to obtain reconstructed projection residual information. The point cloud decoding method according to claim 12.

15. A point cloud decoding device, comprising: obtaining codestream information and decoding the codestream information to obtain analysis data, wherein the analysis data includes a prediction residual of a projection residual of an azimuth angle, the azimuth angle being φ among coordinate components obtained by transforming coordinates (x, y, z) in a Cartesian coordinate system into coordinates (r, θ, φ) in a cylindrical coordinate system; reconstructing a plurality of two-dimensional image information according to the analysis data, wherein the plurality of two-dimensional image information includes azimuth angle projection residual information, which is used to represent a residual between an azimuth angle corresponding to a position of each occupied pixel in the two-dimensional projection planar structure and the azimuth angle corresponding to an actual projection position of a point corresponding to the pixel; reconstructing a point cloud using the plurality of two-dimensional image information; 1. A point cloud decoding device configured to:

Images