Large-scale point cloud oriented 2D regularized planar projection and encoding and decoding methods

The two-dimensional regularized plane projection method addresses the inefficiencies of existing point cloud encoding by transforming point clouds into a 2D structure, enhancing spatial correlation representation and reducing redundancy for improved encoding efficiency.

JP7584182B2Active Publication Date: 2024-11-15HONOR DEVICE CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2023544076
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2021-02-08
Filing Date
2022-02-07
Publication Date
2024-11-15
Estimated Expiration
2042-02-07

AI Technical Summary

Technical Problem

Existing point cloud encoding and decoding techniques, such as octree-based and prediction tree-based methods, fail to adequately represent the spatial correlation of point clouds, leading to inefficient encoding and decoding due to high spatial sparsity and unsuitable representation of geometric information.

Method used

A large-scale point cloud-oriented two-dimensional regularized plane projection method is employed, using regularization parameters to transform point clouds into a 2D projection plane structure, establishing a mapping relationship and encoding data based on this structure to exploit spatial correlation and reduce redundancy.

Benefits of technology

This approach provides a strongly correlated representation of point clouds, reducing spatial redundancy and improving encoding efficiency by projecting point clouds onto a 2D regularized plane, facilitating better data processing and compression.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007584182000010
    Figure 0007584182000010
  • Figure 0007584182000011
    Figure 0007584182000011
  • Figure 0007584182000012
    Figure 0007584182000012
Patent Text Reader

Abstract

A large-scale point cloud-oriented two-dimensional regularized plane projection method and an encoding and decoding method are disclosed. The two-dimensional regularized plane projection method includes: obtaining original point cloud data, initializing a two-dimensional projection plane structure of the point cloud, and determining a mapping relationship between the original point cloud data and the two-dimensional projection plane structure to obtain a two-dimensional regularized projection plane structure corresponding to the point cloud. The encoding and decoding method includes: predicting based on the two-dimensional regularized projection plane structure of the point cloud to obtain data to be encoded, dividing the data to be encoded into a first type of data to be encoded and a second type of data to be encoded, separately encoding different data in the first type of data to be encoded according to a preset encoding method, and encoding the second type of data to be encoded accordingly to obtain a geometric information bit stream.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to the field of point cloud data processing technology, in particular to large-scale point cloud oriented 2D regularized planar projection and encoding and decoding methods. [Background technology]

[0002] With the improvement of hardware processing power and the rapid development of computer vision, 3D point clouds have become a new generation of immersive multimedia following audio, image, and video, and are widely used in virtual reality, augmented reality, autonomous driving, environment modeling, etc. However, it has been reported that large-scale point cloud data is unevenly distributed due to noise, device jitter, device calibration, etc., which makes related data processing difficult and limits further improvement of coding efficiency. In addition, large-scale point clouds are generally very unsuitable for transmitting and storing point cloud data due to their relatively large data capacity.

[0003] In the existing Geometry-based Point Cloud Compression (G-PCC) framework, the geometric information and attribute information of a point cloud are coded separately. Currently, the geometry coding and decoding of G-PCC can be classified into octree-based geometry coding and decoding and predictive tree-based geometry coding and decoding.

[0004] In the case of octree-based geometry coding, first, the geometric information of the point cloud is preprocessed, which includes coordinate transformation and voxelization of the point cloud. Then, tree decomposition (octree / quadtree / binary tree) is successively performed on the bounding box in which the point cloud is located according to a breadth-first traversal order. Finally, the placeholders of each node and the number of points contained in each leaf node are encoded to generate a binary bitstream.

[0005] In the case of predictive tree-based geometry coding, first, the original point cloud data is sorted. Then, a predictive tree structure is established. By classifying each node to the laser scanner it belongs to, the predictive tree structure is established according to different laser scanners. Then, each node of the predictive tree is traversed, and the geometric information of the node is predicted by selecting different prediction modes to obtain a prediction residual, and the prediction residual is quantized by using a quantization parameter. Finally, the predictive tree structure, the quantization parameter, the prediction residual of the geometric information of the node, etc. are coded to generate a binary bitstream.

[0006] However, since the point cloud has a relatively strong spatial sparsity, in the case of the point cloud coding technology using the octree structure, the octree structure has a relatively high ratio of free nodes obtained by division, and the spatial correlation of the point cloud cannot be fully expressed, so it is not suitable for prediction and entropy coding of the point cloud. The predictive tree-based point cloud coding and decoding technology uses part of the parameters of the LiDAR device to establish a tree structure. On this basis, the predictive coding is performed by using a tree structure. However, the tree structure does not fully express the spatial correlation of the point cloud, so it is not suitable for prediction and entropy coding of the point cloud. Therefore, both of the above two point cloud coding and decoding technologies have the problem that the coding efficiency is not high enough. Summary of the Invention [Problem to be solved by the invention]

[0007] In order to solve the above problems existing in the prior art, the present invention provides a large-scale point cloud-oriented 2D regularized plane projection and encoding and decoding method. The technical problem to be solved by the present invention is addressed by the following technical solutions: [Means for solving the problem]

[0008] A large-scale point cloud oriented 2D regularized plane projection method is Obtaining original point cloud data; Initializing a two-dimensional projection structure of the point cloud by using a regularization parameter; determining a mapping relationship between the original point cloud data and the two-dimensional projection plane structure to obtain a two-dimensional regularized projection plane structure of the point cloud; Includes.

[0009] In an embodiment of the present invention, the regularization parameters include parameters obtained by optimization estimation or data fitting or calibration parameters of the LiDAR.

[0010] In an embodiment of the present invention, initializing the two-dimensional projection plane structure of the point cloud by using a regularization parameter includes: Initializing a two-dimensional projection plane structure of the point cloud by using the number of laser scanners, the sampling angular resolution in horizontal azimuth, or the number of sampling points of the laser scanner in a regularization parameter; M=laserNum, and N=360° / Δφ or N=pointNumPerLaser where M and N respectively represent the vertical and horizontal resolutions in the two-dimensional projected planar structure, laserNum is the number of laser scanners, Δφ represents the sampling angular resolution of the laser scanner in the horizontal azimuth, and pointNumPerLaser represents the number of sampling points of the laser scanner.

[0011] In an embodiment of the present invention, determining a mapping relationship between the original point cloud data and the two-dimensional projected plane structure includes: Solving a regularization parameter and a calibration equation to obtain corresponding positions of points of the original point cloud data on the two-dimensional projected planar structure through calculation to determine a mapping relationship between the original point cloud data and the two-dimensional projected planar structure; The calibration formula is:

number

[0012] In an embodiment of the present invention, determining a mapping relationship between the original point cloud data and the two-dimensional projected plane structure includes: determining cylindrical coordinate components of a current point of the original point cloud data; determining a search area for a current point in a two-dimensional projected plane structure; traversing the pixels in the search area, calculating a position of a current pixel in a Cartesian coordinate system, and calculating a spatial distance between that position and the current point; selecting the pixel with the shortest spatial distance as the corresponding pixel of the current point in the two-dimensional projected planar structure; Repeating the above steps until the corresponding pixels in the 2D projected plane structure are found for all points in the original point cloud data. Further includes.

[0013] In an embodiment of the present invention, determining a search area of ​​a current point in a two-dimensional projection plane structure includes: Determining a search area of ​​the current point in the two-dimensional projected plane structure according to the pitch angle θ and azimuth φ of the current point in the cylindrical coordinate system; or determining a search region for the current point in the two-dimensional projected plane structure according to a regularization parameter and a calibration equation; or Determining the search area for the current point according to a priori information Includes.

[0014] In an embodiment of the present invention, the formula for calculating the position of the current pixel in the Cartesian coordinate system is

number

[0015] Another embodiment of the present invention is a method for large scale point cloud oriented encoding, comprising the steps of: The method involves obtaining the original point cloud data, and performing a two-dimensional regularized plane projection to obtain the two-dimensional regularized projection plane structure of the point cloud, and the two-dimensional regularized plane projection is: Any one of the above large-scale point cloud-oriented 2D normalized plane projection methods and their embodiments and Predicting based on a two-dimensional regularized projection plane structure of the point cloud to obtain data to be encoded; Dividing encoding target data into a first type of encoding target data and a second type of encoding target data, the first type of encoding target data being two-dimensional regularized projection plane structure data, and the second type of encoding target data being the remaining encoding target data other than the two-dimensional regularized projection plane structure data; Separately encoding different data in the first type of encoding target data according to a preset encoding method, and encoding the second type of encoding target data to obtain a geometric information bit stream. The present invention provides a method comprising:

[0016] In an embodiment of the present invention, predicting based on a two-dimensional regularized projection plane structure of a point cloud to obtain data to be encoded includes: The method includes setting different prediction modes for different data according to a two-dimensional regularized projection plane structure of the point cloud, and predicting according to the corresponding prediction modes to obtain data to be encoded.

[0017] Yet another embodiment of the present invention is a large scale point cloud oriented decoding method, comprising the steps of: Obtaining a geometry information bit stream; decoding the geometry information bitstream to obtain parsed data; Reconstructing a two-dimensional regularized projected plane structure according to the parsed data; and performing geometric reconstruction according to the reconstructed two-dimensional regularized projection plane structure to obtain a reconstructed point cloud; The present invention provides a method comprising: Effect of the Invention

[0018] 1. In the present invention, by projecting a point cloud in a three-dimensional space onto a corresponding two-dimensional regularized projection plane structure, a regularization correction is performed on the point cloud in the vertical and horizontal directions, and a strongly correlated representation of the point cloud on the two-dimensional projection plane structure is obtained, thereby avoiding the sparsity existing in the three-dimensional representation structure, better reflecting the spatial correlation of the point cloud, and providing a representation form that facilitates data processing for point cloud applications.

[0019] 2. In the present invention, the encoding is performed based on the two-dimensional regularized projection plane structure of the point cloud, which can greatly utilize the spatial correlation of the point cloud and reduce the spatial redundancy, so that there is no need to use additional bitstream to compress other auxiliary information, thereby reducing the bitstream and improving the encoding efficiency.

[0020] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. [Brief description of the drawings]

[0021] [Figure 1] FIG. 2 is a schematic diagram of a large-scale point cloud oriented 2D regularized plane projection method according to an embodiment of the present invention. [Diagram 2] 4 is a schematic diagram of the correspondence between cylindrical coordinates of points and pixels in a two-dimensional projected planar structure according to an embodiment of the present invention; FIG. [Diagram 3] 4 is a flowchart for determining a mapping relationship between an original point cloud and a two-dimensional projected planar structure according to an embodiment of the present invention. [Figure 4] 1 is a schematic diagram of a two-dimensional projection plane structure of a point cloud according to an embodiment of the present invention. [Diagram 5] FIG. 1 is a block diagram of large scale point cloud oriented encoding according to an embodiment of the present invention. [Figure 6] 1 is a schematic structural diagram of three prediction modes according to an embodiment of the present invention; [Figure 7] 1 is a schematic structural diagram of five prediction modes according to an embodiment of the present invention; [Figure 8] FIG. 2 is a block diagram of large scale point cloud oriented decoding according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The present invention is further described in detail below with reference to specific embodiments, but practice of the present invention is not limited thereto. EXAMPLES

[0023] Please refer to FIG. 1, which is a schematic diagram of a large-scale point cloud oriented 2D regularized plane projection method according to an embodiment of the present invention.

[0024] The method includes step 1. acquiring original point cloud data.

[0025] Specifically, the original point cloud data generally includes a group of three-dimensional spatial points, each of which records its geometric position information and additional attribute information such as color, reflectance, or normal. Generally, the geometric position information of the point cloud is expressed based on a Cartesian coordinate system, that is, by the x, y, and z coordinates of the point. The original point cloud data can be acquired by LiDAR scanning or public datasets provided by various platforms. In this embodiment, it is assumed that the geometric position information of the acquired original point cloud data is expressed based on a Cartesian coordinate system. It should be noted that the method of expressing the geometric position information of the original point cloud data is not limited to Cartesian coordinates.

[0026] Step 2. Initialize the 2D projection plane structure of the point cloud by using the regularization parameter.

[0027] Specifically, in this embodiment, before the 2D regularized planar projection is performed on the original point cloud, pre-processing such as coordinate transformation, scale control, or voxelization may be performed on the original point cloud data to facilitate subsequent encoding.

[0028] Initializing the 2D projection plane structure of the point cloud requires the use of regularization parameters, which include parameters obtained by optimization estimation or data fitting or calibration parameters of the LiDAR.

[0029] In general, the calibration parameters of LiDAR are precisely measured by the manufacturer and provided to the consumer as one of the required data. For example, the collection range of the LiDAR, the sampling angle resolution Δφ in the horizontal azimuth or the number of sampling points, the distance correction coefficient of each laser scanner, the offset information V of the laser scanner along the horizontal and vertical directions, 0 and H 0 , or the offset information of the laser scanner along the pitch angle and horizontal azimuth θ 0 and α.

[0030] It should be noted that the regularization parameters are not limited to these calibration parameters of the LiDAR given above. When the calibration parameters of the LiDAR are not given, the regularization parameters may also be obtained by methods such as optimization estimation or data fitting.

[0031] The 2D regularized projected plane structure of the point cloud is a data structure that contains M rows and N columns of pixels. After being projected, the points of the original point cloud correspond to pixels of the data structure. Furthermore, pixel (i,j) of the data structure may be associated with cylindrical coordinate components (θ,φ). For example, pixel (i,j) corresponding to cylindrical coordinates (r,θ,φ) may be found by using the following formula:

number

[0032] Specifically, refer to FIG. 2, which is a schematic diagram of the correspondence between cylindrical coordinates of points and pixels in a two-dimensional projected plane structure according to an embodiment of the present invention.

[0033] It should be noted that pixel correspondence here is not restricted to cylindrical coordinates.

[0034] Furthermore, the resolution of the two-dimensional regularized projection plane can be obtained by the regularization parameter. For example, assuming that the resolution of the two-dimensional regularized projection plane is M×N, the number of laser scanners in the regularization parameter can be used to initialize M, and the sampling angle resolution Δφ in the horizontal azimuth or the number of sampling points of the laser scanner pointNumPerLaser can be used to initialize N. The specific formula is as follows, and finally, the initialization of the two-dimensional projection plane structure can be completed, and a plane structure including M×N pixels can be obtained. M=laserNum, and N=360° / Δφ or N=pointNumPerLaser

[0035] Step 3. Determine the mapping relationship between the original point cloud data and the 2D projection plane structure to obtain the 2D regularized projection plane structure of the point cloud.

[0036] In this embodiment, the location of the original point cloud in the two-dimensional projection plane structure is determined for each point, and the points originally randomly distributed in the Cartesian coordinate system are mapped onto the uniformly distributed two-dimensional regularized projection plane structure. Specifically, for each point of the original point cloud, a corresponding pixel is determined in the two-dimensional projection plane structure. For example, the pixel with the shortest spatial distance from the projected position of the point on the two-dimensional plane may be selected as the corresponding pixel of the point.

[0037] In this embodiment, determining the mapping relationship between the original point cloud data and the two-dimensional projected plane structure includes: Solving a regularization parameter and a calibration equation to obtain corresponding positions of points of the original point cloud data on the two-dimensional projected planar structure through calculation to determine a mapping relationship between the original point cloud data and the two-dimensional projected planar structure; The calibration formula is:

number

[0038] In another embodiment of the present invention, another method is provided for determining a mapping relationship between original point cloud data and a two-dimensional projected plane structure. Referring to Fig. 3, Fig. 3 is a flowchart of determining a mapping relationship between an original point cloud and a two-dimensional projected plane structure according to an embodiment of the present invention.

[0039] The flowchart includes 31) determining cylindrical coordinate components of the current point of the original point cloud data.

[0040] Specifically, the calculation of the cylindrical coordinate component r of the current point (x,y,z) is as follows, where r represents the radial distance between the current point and the coordinate origin: r=√(x 2 +y 2 )

[0041] 32) Determine the search area of ​​the current point in the 2D projected plane structure.

[0042] In this embodiment, the search region may include one pixel in the two-dimensional projected planar structure, or may include multiple pixels in the two-dimensional projected planar structure.

[0043] The entire two-dimensional projection plane structure may be directly selected as the search region.Furthermore, to reduce the amount of calculation, the search region of the corresponding pixel in the two-dimensional projection plane structure may be determined by the pitch angle θ and azimuth φ of the cylindrical coordinate components of the current point to reduce the search region.

[0044] In another embodiment of the present invention, the candidate corresponding positions of the current point on the two-dimensional projection plane structure may be determined by the regularization parameter and the above calibration formula, and the search area may be determined with reference to the candidate corresponding positions. Furthermore, the search area of ​​the current point may be determined according to a priori information.

[0045] 33) traverses the pixels in the search area, calculates the position of the current pixel in a Cartesian coordinate system, and calculates the spatial distance between that position and the current point.

[0046] After the search area is determined, for each pixel (i,j) in the search area, the regularization parameter of the i-th laser scanner of the LiDAR, i.e., the a priori calibration parameter θ 0 , V 0 , H 0 By using α, the position (xl, yl, zl) of the current pixel in the Cartesian coordinate system is calculated, and the specific calculation formula is as follows:

number

[0047] After the position of the current pixel in the Cartesian coordinate system (xl,yl,zl) is obtained, the spatial distance between it and the current point (x,y,z) is calculated and the error Err, i.e.: Err=dist{(x,y,z),(xl,yl,zl)} It is considered that.

[0048] 34 ) The pixel with the shortest spatial distance is selected as the corresponding pixel of the current point in the 2D projected plane structure.

[0049] Specifically, if the error Err is greater than the minimum error minErr, the above update process is skipped; if the error Err is less than the current minimum error minErr, the error Err is used to update the minimum error minErr, and the i and j corresponding to the current pixel are used to update the i and j corresponding pixels of the current point; and if the error Err is greater than the minimum error minErr, the above update process is skipped.

[0050] After all pixels in the search region have been traversed, the corresponding pixel (i,j) of the current point in the two-dimensional projected planar structure and the cylindrical coordinate component r of the current point may be determined.

[0051] 35 ) The above steps are repeated until the corresponding pixel in the 2D projected plane structure is found for every point in the original point cloud data.

[0052] After all the points of the original point cloud have completed the above operations, the 2D regularized plane projection of the point cloud is completed. Specifically, refer to Fig. 4, which is a schematic diagram of a 2D projection plane structure of a point cloud according to an embodiment of the present invention, and all the points of the original point cloud data are mapped to corresponding pixels in the structure.

[0053] It should be noted that during the two-dimensional regularization plane projection of the point cloud, multiple points of the point cloud may correspond to the same pixel in the two-dimensional projection plane structure. To avoid this situation, it may be selected to project these spatial points to different pixels during projection. For example, when a point is projected, if there is already a point corresponding to the pixel corresponding to the point, the point is projected to an empty pixel adjacent to the pixel. Furthermore, if multiple points of the point cloud have already been projected to the same pixel in the two-dimensional projection plane structure, when the encoding is performed based on the two-dimensional projection plane structure, the number of corresponding points of each pixel should be further encoded, and the information of each corresponding point in the pixel should be encoded according to the number of corresponding points.

[0054] In the present invention, by projecting the point cloud in three-dimensional space onto a corresponding two-dimensional regularized projection plane structure, regularization correction is performed on the point cloud in the vertical and horizontal directions, and a strongly correlated representation of the point cloud on the two-dimensional projection plane structure is obtained, thereby avoiding the sparsity existing in the three-dimensional representation structure, better reflecting the spatial correlation of the point cloud, and providing a representation format that facilitates data processing for point cloud applications. EXAMPLES

[0055] Based on the above Example 1, this embodiment provides a large-scale point cloud-oriented encoding method. Refer to Figure 5, which is a block diagram of a large-scale point cloud-oriented encoding method according to an embodiment of the present invention.

[0056] The method includes: Step 1. obtaining original point cloud data, and performing two-dimensional regularized planar projection to obtain a two-dimensional regularized projected planar structure of the point cloud.

[0057] Specifically, this embodiment uses the 2D regularized planar projection of Example 1 above to process the original point cloud to obtain a 2D regularized projected planar structure of the point cloud.

[0058] Furthermore, pre-processing such as coordinate transformation, scale control, or voxelization may be performed on the original point cloud first, and then a 2D regularized planar projection may be performed.

[0059] Step 2. Predict based on the 2D regularized projection plane structure of the point cloud to obtain the data to be encoded.

[0060] In general, when predictive coding is performed on large scale point cloud data, information of the large scale point cloud data, such as radial distance r, pitch angle information i, azimuth interval n, azimuth residual information Δφ, residuals (Δx, Δy, Δz) between each point and the corresponding origin in the Cartesian coordinate system after coordinate transformation of the point cloud, repetition point information, or quantization parameters, need to be specifically coded.

[0061] In this embodiment, when predictive coding is performed based on a two-dimensional regularized projection plane structure of a point cloud, information such as the position i and position j (alternatively, i is the position information in the pitch angle direction, and j is the position information in the azimuth direction) of each non-empty pixel in the two-dimensional regularized projection plane structure, the radial distance r of the corresponding point, the residual (Δx, Δy, Δz) between the space point obtained by backprojection of the pixel and the corresponding point in the Cartesian coordinate system, the iteration point information, or the quantization parameter, etc., need to be coded in particular. It should be noted that a residual called a projection residual may exist between the corresponding position of the non-empty pixel in the two-dimensional regularized projection plane structure and the actual projected position of the corresponding point of the non-empty pixel. During the actual coding, the projection residual may also be coded.

[0062] A 2D regularized projection planar structure of the point cloud is obtained after step 1. The planar structure exhibits the characteristics of uniform distribution, and the advantages of the structure cannot be highlighted if conventional predictive tree coding is used.

[0063] On this basis, this embodiment provides a more flexible method, namely, set different prediction modes for different data according to the two-dimensional regularized projection plane structure of point cloud, and perform prediction according to corresponding prediction modes to obtain data to be coded, thereby improving the effectiveness of prediction and reducing the residual error generated in prediction, which will be described in detail below.

[0064] 1. Position information in azimuth direction: In this embodiment, the position parameter j in the azimuth direction represents interval information along the azimuth direction in units of the minimum sampling interval. Since the two-dimensional regularized projected plane structure of the point cloud also shows a regularized distribution in the azimuth direction, the position j in the azimuth direction in this embodiment is uniformly distributed, compared with the non-uniform distribution in which the interval between the current point to be compressed and the reference point is n in the conventional G-PCC predictive tree coding. Therefore, the current position j can be predicted by the j component of the same position in the previous row.

[0065] More specifically, if the corresponding pixel position of the current point to be compressed in the 2D regularized projection plane structure is (i, j) and the position (i, j-1) is not free, the j component to be compressed can be obtained by subtracting the j component of the position of the (j-1)th row from the j of the current point; if the position (i, j-1) is free, the j component to be compressed can directly use the j component of the current point.

[0066] Compared with the conventional G-PCC method, the compression of j components is performed by referring to the information of the same column in the 2D regularized projection plane structure, which effectively reduces the value and reduces the bitstream overhead.

[0067] 2.Residuals in the azimuth direction φ: The two-dimensional regularized plane projection of the point cloud is corrected in two dimensions including pitch angle and azimuth, and the two-dimensional regularized plane projection structure of the point cloud shows the characteristics of uniform distribution in the azimuth direction. Therefore, the error of azimuth is very small, and there is no need to compress the residual in the azimuth direction φ, so there is no need to perform residual calculation.

[0068] 3. Position information in the pitch angle direction: The prediction may be performed according to the conventional G-PCC prediction tree method, and the specific process will not be described again here.

[0069] 4. Radial distance r: The prediction in this embodiment is performed based on a 2D regularized projected plane structure of the point cloud, which exhibits a regularized distribution in both two dimensions, i.e., pitch angle and azimuth, so that the prediction may be performed according to direction.

[0070] Specifically, a total of three prediction modes can be constructed according to the left and top directions as follows: Mode 0: Direct mode, compress directly without prediction; Mode 1: predict left and use the corresponding pixel on the left as the reference point; and Mode 2: Predict upwards and use the corresponding point of the pixel above as the reference point.

[0071] Specifically, referring to Figure 6, Figure 6 is a schematic structural diagram of three prediction modes according to an embodiment of the present invention, where "●" represents an encoded point, "○" represents an uncoded point, and "★" represents the current position to be coded. Prediction can be performed along the azimuth (mode 1) direction and pitch angle (mode 2) direction based on the radial distance residual Δr, and a point with a smaller Δr can be selected as the reference point.

[0072] The 2D regularized projection plane structure of the point cloud is traversed by using the order from top to bottom and the order from left to right. It is first determined during the traversal whether the corresponding point of the current pixel is the first point to be traversed. If it is, the current point is set as the first node, no prediction is performed, mode 0 is selected, and the radial distance r of the point is directly regarded as the data to be coded; otherwise, prediction is performed according to the prediction methods set in mode 1 and mode 2, the Δr obtained by mode 1 and mode 2 are compared, and the direction mode with the smaller Δr is selected as the prediction mode of the current point, and the current Δr is regarded as the data to be coded.

[0073] The above process is repeated until the corresponding points of all pixels in the 2D regularized projected plane structure, i.e., all points in the point cloud, have been predicted.

[0074] Furthermore, in another embodiment of the present invention, a total of five prediction modes may be established according to left direction, up direction, up-left direction, and up-right direction as follows: Mode 0: Direct mode, compress directly without prediction; Mode 1: predict left and use the corresponding point of the pixel on the left as the reference point; Mode 2: predict upwards and use the corresponding point of the pixel above as the reference point; Mode 3: predict the top left and use the corresponding point of the top left pixel as the reference point; and Mode 4: Predict the top right and use the corresponding point in the top right pixel as the reference point.

[0075] Specifically, referring to Figure 7, Figure 7 is a schematic structural diagram of five prediction modes according to an embodiment of the present invention, where "●" represents an encoded point, "○" represents an uncoded point, and "★" represents the current position to be coded. Prediction can be performed along the azimuth (mode 1) direction and pitch angle (mode 2) direction based on the radial distance residual Δr, and a point with a smaller Δr can be selected as the reference point.

[0076] 5.About the residuals of x, y and z: In order to reduce the number of iterations of the encoding end, a calculation process of residual (Δx, Δy, Δz) can be placed in the above prediction process to calculate the residual of radial distance r. Back projection is first performed on the pixel in the two-dimensional regularized projection plane structure by using the above calibration formula in this specification, during back projection, the radial distance r of the corresponding point of the pixel needs to be combined with the corresponding pitch angle and corresponding azimuth of the pixel, and then the residual (Δx, Δy, Δz) between the spatial position obtained by back projection and the original point cloud is calculated.

[0077] In this embodiment, when performing 2D regularized planar projection on the point cloud, regularization parameters in two dimensions including pitch angle and azimuth are used, so that the corresponding residual values ​​are smaller than those obtained by the traditional G-PCC method, which further reduces the bitstream and improves the coding efficiency to a certain extent.

[0078] Step 3. Divide the encoding target data into a first type of encoding target data and a second type of encoding target data, where the first type of encoding target data is two-dimensional regularized projection plane structure data, and the second type of encoding target data is the remaining encoding target data other than the two-dimensional regularized projection plane structure data.

[0079] The two-dimensional regularized projection plane structure of the point cloud most directly carries the geometric information of the original point cloud, and the amount of data is relatively large, so in this embodiment, the encoding target data obtained in the last step is divided into two types according to the characteristics of the two-dimensional regularized projection plane structure of the point cloud, where the first type of encoding target data is the two-dimensional regularized projection plane structure data of the point cloud, including the radial distance r, the position information i in the pitch angle direction, and the position information j in the azimuth direction. The second type of encoding target data is the remaining encoding target data other than the two-dimensional regularized projection plane structure data, including the residual (Δx, Δy, Δz) between each point after the two-dimensional regularized projection plane of the point cloud and the original point cloud in the Cartesian coordinate system, the repeating point information, or the quantization parameter, etc.

[0080] Step 4. Separately encode different data in the first type of encoding target data according to a preset encoding method, and simultaneously encode the second type of encoding target data to obtain a geometric information bit stream.

[0081] Specifically, according to the analysis in step 2, the residual in the azimuth direction φ in the first type of encoding target data is not encoded, and other information in the first type of encoding target data is encoded in an arithmetic encoding manner, and the second type of encoding target data is encoded to obtain a geometric information bit stream and complete the large-scale point cloud oriented encoding.

[0082] It should be further noted that in the large-scale point cloud-oriented coding method provided in the present invention, a 2D regularized plane projection is performed on the point cloud in a 3D space, a 2D regularized projection plane structure corresponding to the point cloud is obtained, and then predictive coding is performed based on the structure. The 2D regularized projection plane structure of the point cloud can visualize the information of the point cloud, so that the point cloud after the 2D regularized plane projection can be further compressed by picture / video compression methods, such as JPEG, JPEG2000, HEIF, H.264 / AVC, H.265 / HEVC, etc.

[0083] 4 To further illustrate the advantageous effects of the present invention, in this embodiment, by using KITTI data, which is the most popular in point cloud field, as test data, the lossless point cloud encoding efficiency of the method provided in the present invention and that of the traditional MPEG G-PCC method under four scenarios including campus (kitti_campus), residential area (kitti_residential), city (kitti_city) and road (kitti_road) are tested, and the results are as shown in the following table: [Table 1]

[0084] From the table, we can see that the method provided in the present invention shows better performance improvement, with an overall increase of 5.51%, and for scenes with few pedestrians such as campus or residential areas, a gain of more than 8% is obtained. EXAMPLES

[0085] Based on the above Example 2, this embodiment provides a large-scale point cloud-oriented decoding method. Refer to Figure 8, which is a block diagram of large-scale point cloud-oriented decoding according to an embodiment of the present invention.

[0086] The method includes: S1. Obtaining a geometry information bit stream.

[0087] S2. Decode the geometry information bitstream to obtain parsed data.

[0088] Specifically, first, the first type of encoding target data and the second type of encoding target data may be obtained by decoding the geometric information bit stream, where the first type of encoding target data is two-dimensional regularized projection plane structure data of the point cloud, including radial distance r, position information i in the pitch angle direction, and position information j in the azimuth direction. The second type of encoding target data is the remaining encoding target data other than the two-dimensional regularized projection plane structure data, including information such as residual (Δx, Δy, Δz) between each point after two-dimensional regularized plane projection of the point cloud and the original point cloud in the Cartesian coordinate system, iteration point information, or quantization parameter.

[0089] S3. Reconstruct the 2D regularized projection plane structure according to the parsed data.

[0090] Specifically, the parsed data includes the radial distance r of the point, the position information i in the pitch angle direction, and the position information j in the azimuth direction, so that the corresponding pixel of the point in the 2D regularized projection plane structure, i.e., the pixel in the i-th row and j-th column in the 2D regularized projection plane structure, can be determined through i and j. Furthermore, the radial distance r of the corresponding point of the pixel can be known by r. Thus, the reconstructed 2D projection plane structure is obtained.

[0091] S4. Perform geometric reconstruction according to the reconstructed 2D regularized projection plane structure to obtain a reconstructed point cloud.

[0092] For each pixel in the reconstructed 2D regularized projected plane structure, if the current pixel is not empty, the spatial point (x, y, z) corresponding to the current pixel may be reconstructed according to the radial distance r of the corresponding point of the current pixel (i, j) and the parsed residual (Δx, Δy, Δz) in the following manner:

[0093] The corresponding position of the current pixel (i,j) is (φ j ,i), where: φ j =-180°+j×Δφ It is.

[0094] The following uses the regularization parameter and the following formula to backproject the current pixel onto the Cartesian coordinate system to obtain the corresponding Cartesian coordinates (xl,yl,zl):

number

[0095] Finally, the spatial point (x,y,z) corresponding to the current pixel is reconstructed according to the spatial position (xl,yl,zl) obtained by backprojection of the current pixel and the residual (Δx,Δy,Δz) by using the following formula: x=xl+Δx y=yl+Δy z=zl+Δz

[0096] According to the above calculation, the spatial points corresponding to each non-empty pixel in the reconstructed two-dimensional regularized projected plane structure can be reconstructed, thereby obtaining a reconstructed point cloud.

[0097] The above further describes the present invention in detail with reference to specific exemplary embodiments, and the specification should not be considered as a limitation to the specific embodiments of the present invention. Those skilled in the art can further make simple inferences or substitutions without departing from the concept of the present invention, and all such inferences or substitutions should be considered as being within the protection scope of the present invention.

[0098] This application claims priority to Chinese Patent Application No. 202110172053.1, filed with the State Intellectual Property Office of the People's Republic of China on February 8, 2021, for the invention titled "LARGE-SCALE POINT CLOUD-ORIENTED TWO-DIMENSIONAL REGULARIZED PLANAR PROJECTION AND ENCODING AND DECODING METHOD", the entire text of which is incorporated herein by reference.

Claims

1. Obtaining original point cloud data and performing two-dimensional regularized planar projection to obtain a two-dimensional regularized projected planar structure of the point cloud; predicting based on the two-dimensional regularized projection plane structure of the point cloud to obtain data to be encoded; The encoding target data is divided into a first type of encoding target data and a second type of encoding target data, the first type of encoding target data being two-dimensional regularized projection plane structure data, and the second type of encoding target data being the remaining encoding target data other than the two-dimensional regularized projection plane structure data; separately encoding different data in the first type of encoding target data with a preset encoding method, and encoding the second type of encoding target data to obtain a geometric information bit stream; Performing the two-dimensional regularized planar projection includes: initializing a two-dimensional projection plane structure of the point cloud by using a regularization parameter; determining a mapping relationship between the original point cloud data and the two-dimensional projected plane structure to obtain the two-dimensional regularized projected plane structure of the point cloud; having Large-scale point cloud oriented coding method.

2. The regularization parameters include parameters obtained by optimization estimation or data fitting or LiDAR calibration parameters; The large scale point cloud oriented encoding method according to claim 1.

3. The step of initializing the two-dimensional projection plane structure of the point cloud by using a regularization parameter as described above includes the steps of: initializing the two-dimensional projection plane structure of the point cloud by using the number of laser scanners in the regularization parameters, the sampling angular resolution in horizontal azimuth Δφ, or the number of sampling points of the laser scanner; M=laserNum, and N=360° / Δφ or N=pointNumPerLaser where M and N respectively represent the vertical and horizontal resolutions in the two-dimensional projection planar structure, laserNum is the number of the laser scanners, Δφ represents the sampling angular resolution of the laser scanner in the horizontal azimuth, and pointNumPerLaser represents the number of sampling points of the laser scanner. The large scale point cloud oriented encoding method according to claim 1.

4. Determining the mapping relationship between the original point cloud data and the two-dimensional projected plane structure includes: solving the regularization parameters and a calibration equation to obtain corresponding positions of points of the original point cloud data on the two-dimensional projected planar structure by calculation to determine the mapping relationship between the original point cloud data and the two-dimensional projected planar structure; The calibration equation is: [0070] where (x, y, z) represents the Cartesian coordinates of the point in the original point cloud data, r is the cylindrical coordinate component of the point that represents the distance between the point and the coordinate origin of the LiDAR, laserNum represents the number of laser scanners, Δφ represents the sampling angular resolution of the laser scanner in the horizontal azimuth, and θ i and φ j are the pitch angle and azimuth of the corresponding pixel of the point in the two-dimensional projection planar structure, (i, j) represents the position of the corresponding pixel of the point in the two-dimensional projection planar structure, and θ 0 , V 0 , H 0 and α is the regularization parameter. The large scale point cloud oriented encoding method according to claim 1.

5. Determining the mapping relationship between the original point cloud data and the two-dimensional projected plane structure includes: a) determining cylindrical coordinate components of a current point of the original point cloud data; b) determining a search area for the current point in the two-dimensional projection plane structure; c) traversing pixels within said search area, calculating a position of a current pixel in a Cartesian coordinate system, and calculating a spatial distance between said position and said current point; d) selecting the pixel with the shortest spatial distance as the corresponding pixel of the current point in the two-dimensional projected planar structure; e) repeating a), b), c) and d) until a corresponding pixel in the two-dimensional projected planar structure is found for every point of the original point cloud data; Further comprising The large scale point cloud oriented encoding method according to claim 1.

6. Determining a search area for the current point in the two-dimensional projection plane structure includes: determining the search region of the current point in the two-dimensional projection plane structure by a pitch angle θ and an azimuth φ of the current point in a cylindrical coordinate system; or determining the search area of ​​the current point in the two-dimensional projected planar structure according to the regularization parameter and a calibration equation; or determining the search area of ​​the current point according to a priori information; having A large scale point cloud oriented encoding method according to claim 5.

7. The formula for calculating the position of the current pixel in the Cartesian coordinate system is [0080] where (i,j) represents the position of the current pixel in the two-dimensional projection plane structure, and the pitch angle and azimuth corresponding to the current pixel are θ i and φ j where Δφ represents the sampling angular resolution of the laser scanner in horizontal azimuth, x and y are the Cartesian coordinate components of the current point of the original point cloud data, r is the cylindrical coordinate components of the current point of the original point cloud data, (xl, yl, zl) represents the position of the current pixel in the Cartesian coordinate system, and θ 0 , V 0 , H 0 and α is the regularization parameter. A large scale point cloud oriented encoding method according to claim 5.

8. The prediction based on the two-dimensional regularized projection plane structure of the point cloud to obtain the encoding target data includes: setting different prediction modes for different data according to the two-dimensional regularized projection plane structure of the point cloud, and predicting according to the corresponding prediction modes to obtain the encoding target data; A method for large scale point cloud oriented encoding according to any one of claims 1 to 7.

9. Obtaining a geometry information bit stream; decoding the geometric information bitstream to obtain parsed data, the parsed data including a first type of to-be-encoded data and a second type of to-be-encoded data, the first type of to-be-encoded data being two-dimensional regularized projected plane structure data, and the second type of to-be-encoded data being remaining to-be-encoded data other than the two-dimensional regularized projected plane structure data; reconstructing a two-dimensional regularized projected plane structure according to the parsed data; and performing a geometric reconstruction according to the reconstructed two-dimensional regularized projected plane structure to obtain a reconstructed point cloud; A large-scale point cloud oriented decoding method having

10. 1. An electronic device having a processor and a memory, the memory is configured to store computer-executable instructions; The processor is configured to execute the computer-executable instructions stored in the memory to cause the electronic device to perform the method of any one of claims 1 to 8. Electronic devices.

11. 1. An electronic device having a processor and a memory, the memory is configured to store computer-executable instructions; The processor is configured to execute the computer-executable instructions stored in the memory to cause the electronic device to perform the method of claim 9. Electronic devices.

12. storing computer programs or instructions; The computer program or the instructions, when executed, perform the method according to any one of claims 1 to 8. A computer-readable storage medium.

13. storing computer programs or instructions; The computer program or instructions, when executed, perform the method according to claim 9. A computer-readable storage medium.

Citation Information

Patent Citations

  • Two-dimensional point cloud matching method

    CN105427239A

  • A method for converting a two-dimensional image into a three-dimensional point cloud

    CN109509143A

  • Point cloud occupancy map compression

    US20190156520A1

  • Visualization and storage algorithms associated with processing point cloud data

    US7804498B1