A hierarchical division method, encoder and storage medium for point cloud data

Abstract

The embodiment of the present application provides a hierarchical division method of point cloud data, an encoder and a storage medium. The above method includes: obtaining point cloud data to be processed; pre-analyzing and pre-fitting the point cloud data to determine fitting parameters; based on the fitting parameters, determining the number of detail level division layers with optimal encoding performance; hierarchically dividing the point cloud data according to the number of detail level division layers, and encoding the point cloud data.

Description

Technical Field

[0001] The embodiments of the present application relate to video coding and decoding technology in the field of communications, and in particular to a hierarchical division method, encoder and storage medium for point cloud data. Background Art

[0002] In the geometry-based point cloud compression (G-PCC) encoder framework, the geometric information of the point cloud and the attribute information corresponding to each point cloud are encoded separately. After the geometry encoding is completed, the geometry information will be reconstructed, and the encoding of the attribute information will depend on the reconstructed geometry information. Among them, the attribute information encoding is mainly for the encoding of color information. In the color information encoding, there are two main transformation methods, one is the lifting transformation based on the distance-based level of detail (LOD) division, and the other is the direct region adaptive hierarchical transformation (RAHT). Both methods convert the color information from the spatial domain to the frequency domain, obtain high-frequency coefficients and low-frequency coefficients through transformation, and finally quantize and encode the coefficients to generate a binary code stream.

[0003] In the related art, during the LOD division process of the point cloud, the user manually sets the number of LOD division layers. In the general test conditions of the G-PCC encoder framework of the point cloud based on the geometry of the point cloud, the number of LOD division layers is usually set to a larger number. Since different point clouds have different characteristics, if the number of division layers is too large, although the best encoding performance can be obtained, unnecessary LOD layers may be generated at the same time, which will increase the time complexity without improving the encoding performance. Summary of the invention

[0004] The embodiments of the present application hope to provide a hierarchical division method, encoder and storage medium for point cloud data, which can improve the accuracy of the number of detail division layers while ensuring the encoding performance.

[0005] The technical solution of the embodiment of the present application can be implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for hierarchical division of point cloud data, comprising:

[0007] Obtain the point cloud data to be processed;

[0008] Pre-analyzing and pre-fitting the point cloud data to determine fitting parameters;

[0009] Based on the fitting parameters, determining the number of detail level division layers with optimal coding performance;

[0010] The point cloud data is hierarchically divided according to the number of detail level division layers, and the point cloud data is encoded.

[0011] In a second aspect, an embodiment of the present application provides an encoder, including:

[0012] An acquisition part, configured to acquire point cloud data to be processed;

[0013] The determination part is configured to pre-analyze and pre-fit the point cloud data to determine fitting parameters; and determine the number of detail level division layers with optimal encoding performance based on the fitting parameters;

[0014] A division part is configured to divide the point cloud data into levels according to the number of levels of detail division;

[0015] The encoding part is configured to encode the point cloud data.

[0016] In a third aspect, an embodiment of the present application further provides an encoder, including:

[0017] Memory and processor;

[0018] The memory is used to store a computer program that can be run on the processor;

[0019] The processor is used to execute the hierarchical division method of point cloud data provided in the embodiment of the present application when running the computer program.

[0020] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing executable point cloud hierarchical division instructions, which are used to cause a processor to execute and implement the hierarchical division method of point cloud data provided by the embodiment of the present application.

[0021] The embodiment of the present application provides a method for hierarchical division of point cloud data, an encoder and a storage medium, including: obtaining point cloud data to be processed; pre-analyzing and pre-fitting the point cloud data to determine fitting parameters; determining the number of detail level division layers with optimal encoding performance based on the fitting parameters; hierarchically dividing the point cloud data according to the number of detail level division layers, and encoding the point cloud data. Using the above technical implementation scheme, since the encoder can use simulated division for the point cloud data and perform fitting processing in the process of encoding the point cloud data to be processed, it is determined that the fitting parameters that can guarantee the optimal encoding performance are obtained, so that the fitting parameters can be used to specifically confirm the number of detail level division layers that the point cloud data can be divided into. The number of detail level division layers is not necessarily the largest, but it can already guarantee the optimal encoding performance, thereby achieving the accuracy of the number of detail division layers while ensuring the encoding performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 An exemplary encoding flow chart provided for an embodiment of the present application;

[0023] Figure 2 An exemplary decoding flow chart provided for an embodiment of the present application;

[0024] Figure 3 A schematic diagram of an exemplary LOD division implementation process provided in an embodiment of the present application;

[0025] Figure 4 A flowchart of a method for hierarchical division of point cloud data provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of an exemplary structure of performing LOD division based on Morton code provided in an embodiment of the present application;

[0027] Figure 6 A schematic diagram of the structure of an encoder provided in an embodiment of the present application Figure 1 ;

[0028] Figure 7 A schematic diagram of the structure of an encoder provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0029] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0030] In an embodiment of the present application, in the point cloud G-PCC encoder framework, after the point cloud of the input three-dimensional image model is sliced, each slice is independently encoded.

[0031] like Figure 1In the flowchart of G-PCC coding shown in FIG. 1 , it is applied to the point cloud encoder. For the point cloud data to be encoded, the point cloud data is first divided into multiple slices by slice division. In each slice, the geometric information of the point cloud and the attribute information corresponding to each point cloud are encoded separately. In the process of geometric coding, the coordinates of the geometric information are transformed so that all the point clouds are contained in a bounding box, and then quantized. This step of quantization mainly plays a role in scaling. Due to the quantization rounding, the geometric information of a part of the point cloud is the same, so it is determined whether to remove duplicate points based on parameters. The process of quantization and removal of duplicate points is also called voxelization. Then the bounding box is divided into octrees. In the octree-based geometric information coding process, the bounding box is divided into 8 sub-cubes in eight equal parts, and the non-empty (containing points in the point cloud) sub-cubes are further divided into eight equal parts until the leaf node obtained by division is a 1x1x1 unit cube, and the division is stopped. The points in the leaf node are arithmetically encoded to generate a binary geometric bit stream, that is, a geometric code stream. In the process of geometric information encoding based on trisoup (triangle soup, triangle patch set), octree division must also be performed first, but different from geometric information encoding based on octree, this trisoup does not need to divide the point cloud into unit cubes with a side length of 1x1x1 step by step, but stops dividing when the block (sub-block) has a side length of W. Based on the surface formed by the distribution of point clouds in each block, at most twelve vertices (intersections) generated by the surface and the twelve edges of the block are obtained, and the vertices are arithmetically encoded (surface fitting based on intersections) to generate a binary geometric bit stream, that is, a geometric code stream. Vertex is also used to implement the process of geometric reconstruction, and the reconstructed set information is used when encoding the attributes of the point cloud.

[0032] In the attribute encoding process, after the geometric encoding is completed and the geometric information is reconstructed, color conversion is performed to convert the color information (i.e., attribute information) from the RGB color space to the YUV color space. Then, the point cloud is recolored using the reconstructed geometric information so that the uncoded attribute information corresponds to the reconstructed geometric information. In the color information encoding process, there are two main transformation methods. One is the distance-based lifting transformation that relies on the level of detail (LOD) division, and the other is the direct transformation of the region adaptive hierarchical transform (RAHT). Both methods will convert the color information from the spatial domain to the frequency domain, obtain high-frequency coefficients and low-frequency coefficients through the transformation, and finally quantize the coefficients (i.e., quantized coefficients). Finally, the geometric coded data after octree division and surface fitting and the attribute coded data processed by the quantized coefficients are sliced ​​and synthesized, and the vertex coordinates of each block are encoded in turn (i.e., arithmetic coding) to generate a binary attribute bit stream, i.e., the attribute code stream.

[0033] like Figure 2 The flowchart of G-PCC decoding shown in the figure is applied to the point cloud decoder. The decoder obtains the binary code stream and independently decodes the geometric bit stream and attribute bit stream in the binary code stream. When decoding the geometric bit stream, the geometric information of the point cloud is obtained through arithmetic decoding-octree synthesis-surface fitting-reconstruction of geometry-inverse coordinate transformation; when decoding the attribute bit stream, the attribute information of the point cloud is obtained through arithmetic decoding-inverse quantization-LOD-based inverse lifting or RAHT-based inverse transformation-inverse color conversion, and the three-dimensional image model of the point cloud data to be encoded is restored based on the geometric information and attribute information.

[0034] It should be noted that the encoding of attribute information is divided into three modes: prediction mode, enhancement mode and RAHT mode. For the prediction mode and enhancement mode, the point cloud needs to be divided into LOD. LOD is to divide the point cloud into a series of enhancement layers and detail layers in a certain way, such as Figure 3 As shown, Figure 3 This is a schematic diagram of the implementation process of LOD division according to an exemplary embodiment of the present application, combined with Figure 3 The following instructions are given:

[0035] For point set 401 (LOD1), it is divided into set A1 representing the detail information of the point cloud, and point set 402 (LOD2) representing the skeleton information of the point cloud; then, for point set 402, it is divided into set A2 representing the detail information of the point cloud, and point set 403 (LOD3) representing the skeleton information of the point cloud.

[0036] In the related art, there is no regulation on the number of LOD division layers, which is set manually by the user, but there is no regulation on the range of LOD division layers. Under the current general test conditions of G-PCC, a larger LOD generation layer number is set for all point clouds. However, for point clouds with different characteristics, different LOD layers should be set. If the number of layers is not selected properly, it is likely that the optimal coding performance will not be obtained. On the other hand, if the number of LOD layers is set too large, although the best coding performance can be obtained, unnecessary LOD layers may be generated at the same time, which will increase the time complexity without improving the coding performance.

[0037] Based on the above background, the following describes a method for hierarchical division of point cloud data provided by an embodiment of the present application, which is applied to a hierarchical division device for point cloud data, where the hierarchical division of point cloud data can be a point cloud encoder, that is, an encoder. The present application acts in the process of generating LOD.

[0038] like Figure 4 As shown, an embodiment of the present application provides a method for hierarchical division of point cloud data, which may include:

[0039] S101, obtaining point cloud data to be processed;

[0040] S102, pre-analyzing and pre-fitting the point cloud data to determine fitting parameters;

[0041] S103, determining the number of detail level division layers with optimal coding performance based on the fitting parameters;

[0042] S104, dividing the point cloud data into layers according to the number of detail levels, and encoding the point cloud data.

[0043] In S101, in an embodiment of the present application, during the encoding process of a three-dimensional image model, the original point cloud data of the three-dimensional image model to be encoded in space is obtained, and the original point cloud data may contain geometric information and attribute information of the three-dimensional image model. In the process of encoding the three-dimensional image model, the geometric information of the point cloud and the attribute information corresponding to each point are encoded separately. Among them, the geometric information includes: point cloud normal vector, curvature, point cloud density, point cloud surface roughness, point cloud center of gravity, point cloud weight center of gravity, point cloud covariance and point cloud mutual covariance, etc. The attribute information includes: the square of the division distance of the input point cloud, the total number of division layers, the information of the color space, the spatial resolution, the point position accuracy and the surface normal vector, etc. After the division of the strip (slice) during the geometric information encoding is completed, the point cloud data of each slice, that is, the point cloud data to be processed, is subjected to geometric encoding and attribute encoding. In the process of generating LOD of the attribute encoding, the hierarchical division method of the point cloud data provided in the embodiment of the present application is adopted.

[0044] In S102 , when performing LOD division for attribute coding, the encoder performs characteristic analysis and fitting on the point cloud data to be processed to determine fitting parameters for LOD division.

[0045] It should be noted that the purpose of the encoder performing pre-analysis and pre-fitting on the point cloud data to be processed is to obtain the best encoding performance, that is, to achieve fitting parameters for lossless compression.

[0046] In an embodiment of the present application, pre-analysis may include a pre-division method, and may also include other methods that can analyze the point cloud data to be processed and achieve the purpose of lossless compression, which is not limited by the embodiment of the present application.

[0047] In some embodiments of the present application, when the pre-analysis is pre-division, the encoder pre-divides the point cloud data to obtain enhancement layer division parameters, estimated enhancement layer, estimated detail layer and detail layer division parameters; predicts the estimated enhancement layer and the estimated detail layer to determine the enhancement layer prediction error and the detail layer prediction error; performs pre-fitting based on the enhancement layer prediction error, the enhancement layer division parameters, the detail layer prediction error and the detail layer division parameters to determine the fitting parameters.

[0048] It should be noted that pre-division is only a simulation division process, not the actual LOD division process.

[0049] In the embodiment of the present application, the pre-divided layers may be implemented based on Morton codes or based on sampling distances.

[0050] In the embodiment of the present application, the pre-division includes: division based on sampling distance. The enhancement layer division parameter is the sampling distance used by the current layer, and the detail layer division parameter is the distance between adjacent points of the current layer.

[0051] In an embodiment of the present application, the pre-division includes: division based on Morton code. The enhancement layer division parameter is the number of right-shift bits of the Morton code of the current layer, and the detail layer division parameter is the number of right-shift bits of the Morton code of the previous layer based on the division order; wherein the Morton code is determined according to the position parameter of the midpoint of the point cloud data.

[0052] It should be noted that the enhancement layer may be multiple layers, and each layer corresponds to an enhancement layer division parameter.

[0053] In some possible implementations, the process of pre-dividing the point cloud data into LODs based on the Morton code is as follows:

[0054] First, assume that the variable (P i ) i＝1...N Save the original point cloud data to be processed, variable (M i )i＝1....N is the Morton value associated with each point. And D0 and ρ are set as two initial parameters set by the user, which are used as parameters for pre-dividing the LOD distance threshold, where ρ>1. The specific method for calculating the Morton code is described as follows. For each component of the three-dimensional coordinate represented by a d-bit binary number, its three components can be expressed as shown in formula (1):

[0055]

[0056] Among them, x l ,y l ,z l ∈{0,1} are the binary values ​​corresponding to the highest bit (l=1) to the lowest bit (l=d) of x, y, and z respectively. The Morton code M is to arrange x, y, and z in a crosswise order starting from the highest bit. l ,y l ,z l To the lowest bit, the calculation formula of M is shown in the following formula (2):

[0057]

[0058] Among them, m l' ∈{0,1} are the values ​​of the highest bit (l'=1) to the lowest bit (l'=3d) of M. After obtaining the Morton code M of each point in the point cloud, the points in the point cloud are arranged in order of the Morton code from small to large.

[0059] In some embodiments, during the compression process of the point cloud, the size of the bounding box of the point cloud can be obtained according to the encoding of the geometric information, and a cube is constructed according to the longest side, which is the root node of the point cloud.

[0060] In some possible implementations, in the LOD generation process based on the Morton code right shift, abh.attr_dist2_delta is first calculated based on aps.dist2 and the input point cloud, and then the initial right shift bit shiftbits0 is calculated based on aps.dist2 and abh.attr_dist2_delta, as shown in formula (3):

[0061] shiftbits0=aps.dist2+abh.attr_dist2_delta (3)

[0062] The first right shift number shiftbits is shiftbits0, and the corresponding node size is 2. shifbits0 , expressed as Figure 5A in the LOD. When LOD is divided, shiftbits increases by 1 for each layer; and for each right shift of 1 bit, the edge length of the node increases to twice that of the previous layer node, expressed as Figure 5 2A in.

[0063] When divided into the Nth layer, shiftbits = shiftbits0 + (N-1), and the node size is 2 N-1+shifbits0 In an exemplary embodiment of the present application, the condition for terminating the LOD division is: when the division reaches the root node, the LOD division ends, and at this time the root node size is 2 maxRootNodeDimLog2 , we can get formula (4):

[0064] 2 N-1+shifbits0 ≤2 maxRootNodeDimLog2 (4)

[0065] When the encoder pre-division is completed, the first N-1 layers are estimated enhancement layers, the Nth layer is the estimated detail layer, and the enhancement layer division parameter is the current right shift number of each layer, and the detail layer division parameter is the right shift number of the N-1th layer.

[0066] In another implementation of the present application, the process of performing LOD pre-stratification on point cloud data based on sampling distance is as follows:

[0067] The LOD generation process is based on a set of different distance thresholds (d l ) l =0...L -1 The original input point cloud is divided into different refinement levels (R l ) l＝0...L-1 Note that the distance threshold (d l ) l＝0...L-1 The two conditions shown in formula (1) need to be met: that is,

[0068] The LOD pre-division process is determined after the point cloud geometry is reconstructed, at which time the geometric coordinate information of the point cloud can be obtained. The LOD division process is applied to both the encoding and decoding ends of the point cloud. The specific process of LOD division is as follows:

[0069] The variable I stores the index of all points. The algorithm is iterated continuously. At the kth iteration, the points in LODk will search for the nearest neighbors from LOD0 to LODk-1. The specific process of the algorithm is as follows:

[0070] First, initialize the sampling distance D = D0;

[0071] Then, at the kth iteration, k = 0, 1...N-1 (N is the total number of LOD divisions). The variable L(k) stores the points belonging to the kth LOD layer, and the variable O(k) belongs to the point set higher than the LODk layer. The calculation steps of L(k) and O(k) are as follows:

[0072] In the first step, O(k) and L(k) are initialized;

[0073] Among them, L(k) is initialized to the empty set; O(k) is initialized to the empty set.

[0074] In the second step, each iteration is traversed according to the index of the point saved in variable I. Each traversal will calculate the distance from the current point to all points within a certain range in the set O(k). Based on the Morton order value corresponding to the current point, a search is performed within a search range SR1 of the set O(k) (SR1 represents the search range based on the Morton order, generally 8, 16, 64). If a point with a distance less than the threshold dl from the current point is found within the range, the current point is added to the set L(k), otherwise it is added to the set O(k).

[0075] The third step is that in each iteration, the sets L(k) and O(K) are calculated separately, and the points in O(k) are used as the prediction points of the points in the set L(k), thus realizing the LOD pre-stratification of the point cloud data. Assume that the set R(k) = L(k)\L(k-1), where R(k) represents the point set that is the difference between the LOD(k-1) and LOD(k) sets.

[0076] The series of O(k) obtained by the final pre-division is the estimated enhancement layer, and the final L(k) obtained is the estimated detail layer. The sampling distance used in each layer is the enhancement layer division parameter corresponding to the layer, and the distance between adjacent points of the current layer in the detail layer is the detail layer division parameter.

[0077] By adopting the above method, the encoder can perform pre-division to obtain the enhancement layer division parameters, estimated enhancement layer, detail layer division parameters and estimated detail layer after division. The encoder performs a first prediction process on the estimated enhancement layer to determine the enhancement layer prediction error of the estimated enhancement layer; and performs a second prediction process on the estimated detail layer to determine the detail layer prediction error of the estimated detail layer.

[0078] In the embodiment of the present application, the first prediction process includes: inter-layer prediction and intra-layer prediction; the second prediction process is intra-layer prediction.

[0079] In some embodiments of the present application, the encoder performs inter-layer prediction and intra-layer prediction on the estimated enhancement layer to determine the enhancement layer prediction error of the estimated enhancement layer; and performs intra-layer prediction on the estimated detail layer to determine the detail layer prediction error of the estimated detail layer.

[0080] In some embodiments of the present application, the fitting parameters include: a first fitting parameter and a second fitting parameter; the encoder may use a first error model to perform pre-fitting processing based on the enhancement layer prediction error and the enhancement layer division parameters to determine the first fitting parameters; and use a second error model to perform pre-fitting processing based on the detail layer prediction error and the detail layer division parameters to determine the second fitting parameters.

[0081] In the embodiment of the present application, the first error model and the second error model may be power function models.

[0082] Exemplarily, the first error model may be formula (5), and the second error model may be formula (6). As follows:

[0083]

[0084]

[0085] Among them, f(d1) represents the detail layer prediction error, g(d2) represents the enhancement layer prediction error, d1 represents the detail layer division parameter, and d2 represents the enhancement layer division parameter.

[0086] The encoder can perform fitting processing according to formula (5), the enhancement layer prediction error and the enhancement layer division parameter to determine the first fitting parameters a2 and b2, and perform fitting processing according to formula (6), the detail layer prediction error and the detail layer division parameter to determine the second fitting parameters a1 and b1.

[0087] In some embodiments of the present application, the encoder sets the number of detail level division levels to a value not greater than a division level threshold, wherein the division level threshold is a preset value.

[0088] In some embodiments of the present application, the preset value is a value corresponding to a conformance parameter, wherein the conformance parameter includes at least one of the following parameters: profile, tier, and level.

[0089] In S103, the process of the encoder determining the number of detail level division layers with optimal coding performance based on the fitting parameters may be determining the number of detail level division layers with optimal coding performance based on the first fitting parameters and the second fitting parameters.

[0090] In some embodiments of the present application, the encoder performs optimal layer calculation based on the fitting parameters using a preset layer number model to obtain the number of detail level division layers; wherein the preset layer number model represents the correspondence between the fitting parameters and the number of detail level division layers when the encoding performance is optimal.

[0091] In the embodiment of the present application, the optimal coding performance corresponds to the minimum prediction error. Therefore, the preset layer number model can be obtained by solving the relationship between the fitting parameter, the partition parameter and the number of partition layers related to the coding error for the minimum coding error.

[0092] For example, in the embodiment of the present application, the division of LOD based on sampling distance is taken as an example for explanation. The preset layer number model can be shown in formula (7), as follows:

[0093]

[0094] Among them, N op is the number of detail levels, D0 is the preset initial sampling distance, C1 is the expression of a2 and b2, and C2 is the expression of a1 and b1.

[0095] in, Wherein P is the ratio of the points in the preset enhancement layer to the points in the corresponding detail layer at each division. The detailed process of presetting P will be described in the subsequent embodiments.

[0096] In the embodiment of the present application, a2, b2, a1 and b1 are substituted into formula (7) to obtain N op .

[0097] In some embodiments of the present application, the process of determining the preset layer number model is as follows:

[0098] S1. Determine a preset prediction error model, wherein the preset prediction error model is constructed by weighting the mean absolute error of the enhancement layer and the mean absolute error of the detail layer, and the preset prediction error model includes a weighted ratio parameter;

[0099] S2. Determine a correspondence between a partition parameter and a weighted ratio parameter, wherein the partition parameter includes at least one of an enhancement layer partition parameter and a detail layer partition parameter;

[0100] S3, determining a first prediction error model corresponding to the partitioning parameter based on the corresponding relationship and the preset prediction error model;

[0101] S4, constructing a first functional relationship between a partition parameter and a prediction error of an enhancement layer, and a second functional relationship between a partition parameter and a prediction error of a detail layer;

[0102] S5. Based on the first functional relationship and the second functional relationship, the first prediction error model is converted to determine a prediction error minimization model;

[0103] S6. According to the prediction error minimization model, determine the preset number of layers model corresponding to the minimum prediction error.

[0104] For S1, the encoder builds a model for the prediction error of LOD, and the point cloud is N (i.e., N op ) layer after LOD division, it is divided into {A1, A2, ..., A N-1 , LOD N}. Define the mean absolute error as E, the mean absolute error of the prediction error of the entire point cloud sequence E total It can be expressed as the weighted sum of the average absolute error of the prediction errors of each part after LOD division, that is, the enhancement layer (A i ) i＝1,…,N-1 The mean absolute error (i.e., mean absolute error of the enhancement layer) and detail layer LOD N The mean absolute error The weighted sum of (detail layer mean absolute error) is used to obtain the preset prediction error model. The weighted ratio parameter is p 1,i and p 2,N .

[0105] Exemplarily, the preset prediction error model is shown in formula (8):

[0106]

[0107] Among them, p 1,i A i The ratio of the number of points in the point cloud to the total number of points in the point cloud, p 2,N For LOD N The ratio of the number of points in the point cloud to the total number of points in the point cloud, and

[0108] For S2, in the LOD division process (taking the sampling distance as an example), the division parameter includes at least one of the enhancement layer division parameter and the detail layer division parameter. The encoder determines the corresponding relationship between the division parameter and the weighted ratio parameter as follows:

[0109] Assume r i Enhancement layer A i The number of midpoints accounts for the corresponding detail layer LOD i The ratio of the number of midpoints. i ,have

[0110]

[0111] For LOD i ,have

[0112] Among them, formula (9) and formula (10) are the corresponding relationship between the partition parameter and the weighted ratio parameter. i It is related to the partition parameter (here is the sampling distance). iFor the first layer, it is related to the sampling distance D0. For the second layer and above, r i It is related to the ratio of the distances between two adjacent samplings.

[0113] It should be noted that, assuming that the point cloud to be processed is uniformly distributed, then except for the first sampling, the other sampling distances are fixed, so r i It is related to the sampling distance ratio of two adjacent LOD layers. In G-PCC, the LOD sampling distance ratio of two adjacent layers is set to 2 (this application does not limit this distance ratio). When the LOD is divided to the second layer or above, the ratio of each LOD division is consistent. i =P, where the ratio r1 of the first LOD division is related to the initial division distance D0.

[0114] In the embodiment of the present application, in practice, all the points in the point cloud are located on the surface of the object, and the points within a certain range can be regarded as two-dimensional.

[0115] In S3, the encoder adopts the corresponding relationship to replace the weighted ratio parameter in the preset prediction error model to obtain a first prediction error model related to the partition parameter.

[0116] In the LOD division process (taking sampling distance as an example), when N=1, formula (1) is: The LOD1 layer is the point cloud itself. When N ≥ 2, the first prediction error model is shown in formula (11):

[0117]

[0118] In S4, the encoder constructs a first functional relationship between the partition parameter and the prediction error of the enhancement layer, and a second functional relationship between the partition parameter and the prediction error of the detail layer.

[0119] It should be noted that the first functional relationship represents the correspondence between the division parameter and the enhancement layer prediction error, and the second functional relationship represents the correspondence between the division parameter and the detail layer prediction error, wherein the division parameter in the first functional relationship can be the enhancement layer division parameter, and the division parameter in the second functional relationship can be the detail layer division parameter.

[0120] Exemplarily, the first functional relationship between the constructed partition parameter and the enhancement layer prediction error is shown in formula (12), and the second functional relationship between the constructed partition parameter and the detail layer prediction error is shown in formula (13):

[0121]

[0122]

[0123] Wherein, d1 represents the enhancement layer division parameter, and d2 represents the detail layer division parameter.

[0124] That is to say, in the prediction coding of LOD, for the case of only intra-layer prediction, let Where d1 is the distance between adjacent points in the layer; for the case of both inter-layer prediction and intra-layer prediction, let Where d2 is the sampling distance of the current enhancement layer.

[0125] In S5, the encoder performs power function fitting on the first functional relationship and the second functional relationship respectively to obtain a first error model and a second error model with the partition parameters as variables; the first error model and the second error model are used to transform the first prediction error model to determine a prediction error minimization model.

[0126] In an embodiment of the present application, the encoder performs power function fitting on the first functional relationship and the second functional relationship respectively, and obtains a first error model (for example, formula (5)) and a second error model (for example, formula (6)) with the partition parameter as a variable. The encoder then replaces the enhancement layer prediction error and the detail layer prediction error in the first prediction error model with the expressions of the first error model and the second error model, and obtains a prediction error minimization model.

[0127] For example, in the LOD division process (taking sampling distance as an example), the prediction error minimization model is shown in formula (14):

[0128]

[0129] Among them, d 2,1 is the LOD initial sampling distance D0, d 2,i To generate LOD i+1 The sampling distance of the layer (d 2,i =2 i-1 D0), d 1,N For LOD N The distance between adjacent points in a layer and the sampling distance of the previous layer (d 1,N =d 2,N-1 ).

[0130] It should be noted that the encoder can first substitute the first functional relationship and the second functional relationship into the first prediction error model to obtain formula (15), and then obtain the prediction error minimization model by fitting the first error model and the second error model.

[0131]

[0132] In S6, the encoder may perform a derivative operation on the prediction error minimization model with respect to the number of divided layers to determine a preset number of layers model.

[0133] In the present embodiment, E total (N) is a convex function, find E total (N) Partial derivative with respect to N We get formula (16):

[0134]

[0135] make

[0136] when When E total (N) obtains the minimum value, that is, the prediction error minimization model is derivatized according to the number of division layers to achieve the best coding performance. At this time, N = N op , thus obtaining a model with preset number of layers.

[0137] Exemplarily, the preset layer model is formula (7).

[0138] In the embodiment of the present application, the number of LOD division levels is N op When the prediction error of the point cloud is total Minimal, lossless compression for optimal performance.

[0139] In S104, after determining the number of detail level division layers that can achieve optimal encoding performance, the encoder can divide the point cloud data into layers according to the number of detail level division layers, and encode the point cloud data.

[0140] In some embodiments of the present application, the encoder can divide the point cloud data into layers according to the detail level, obtain an enhancement layer and a detail layer; perform inter-layer prediction and intra-layer prediction on the enhancement layer to obtain a high-frequency coefficient; perform intra-layer prediction on the detail layer to obtain a low-frequency coefficient; and encode the high-frequency coefficient and the low-frequency coefficient.

[0141] It should be noted that the process of dividing the point cloud data into layers according to the number of layers of detail levels can be consistent with the process of dividing based on sampling distance or Morton code described above, which will not be repeated here.

[0142] In an embodiment of the present application, the encoder divides the point cloud data into layers according to the number of detail level division layers, encodes the number of detail level division layers, and writes the encoded bits into the bit stream for use by the decoding end during decoding.

[0143] In the embodiment of the present application, the encoder fitting process needs to process the point cloud data twice:

[0144] The point cloud is subjected to lossless compression based on LOD that allows intra-layer prediction and inter-layer prediction, and the prediction error in each enhancement layer and the sampling distance of the current enhancement layer are counted for fitting to obtain the first fitting parameter; the point cloud is subjected to compression and fitting based on LOD that only allows intra-layer prediction to obtain the second fitting parameter; after fitting, the first fitting parameter, the second fitting parameter and formula (7) are used to calculate the optimal number of LOD division layers, that is, the number of detail level division layers.

[0145] It is understandable that, since the encoder can adopt simulated division of point cloud data and perform fitting processing in the process of encoding the point cloud data to be processed, and determine the fitting parameters that can ensure the optimal encoding performance, the fitting parameters can be used to further specifically confirm the number of detail level division layers that the point cloud data can be divided into. The number of detail level division layers is not necessarily the largest, but it can already ensure the optimal encoding performance, thereby achieving the accuracy of the number of detail division layers while ensuring the encoding performance.

[0146] Furthermore, the encoder first models the prediction error of the LOD, and then fits the relationship between the prediction error and the distance of each LOD layer. Finally, based on the fitting result and the prediction error model, the optimal number of LOD division layers can be calculated to minimize the attribute bit rate of the point cloud lossless compression. At the same time, unnecessary LOD divisions can be avoided, thereby saving encoding and decoding time.

[0147] The point cloud sequences of the A-class dataset from static objects and scenes, longdress_vox10_1300, redandblack_vox10_1550, soldier_vox10_0690, basketball_player_vox11_00000200, dancer_vox11_00000001, facade_00015_vox14 and frog_00067_vox12, are tested, and the optimal LOD division layer number N calculated by the proposed method is op The number of LOD division levels N set in the general test conditions of G-PCC G The comparison is shown in Table 1.

[0148] Table 1

[0149]

[0150] As can be seen from Table 1, the comparison of LOD division layers uses the N required by this application. op N G This minimizes the bit rate of the point cloud lossless compression attribute during the encoding process and avoids unnecessary LOD division, thus saving encoding and decoding time.

[0151] As shown in Table 2, N op The bit rate when LOD is divided is N G The ratio of the bit rate when the number of layers LOD is divided, and N op The complexity of LOD division accounts for N G The ratio of complexity when dividing the layers into LODs.

[0152] Table 2

[0153]

[0154]

[0155] From Table 2, we can see that N op The complexity is less than N G , and the bit rate after compression is consistent, indicating that the performance of lossless compression is consistent. At the same time, the number of LOD division layers calculated using the method of the present application will significantly reduce the time complexity.

[0156] Based on the implementation basis of the aforementioned embodiment, Figure 6 As shown, the embodiment of the present application provides an encoder 1, comprising:

[0157] An acquisition part 10 is configured to acquire point cloud data to be processed;

[0158] The determination part 11 is configured to pre-analyze and pre-fit the point cloud data to determine fitting parameters; and determine the number of detail level division layers with optimal encoding performance based on the fitting parameters;

[0159] A division part 12 is configured to divide the point cloud data into levels according to the number of levels of detail division;

[0160] The encoding part 13 is configured to encode the point cloud data.

[0161] In some embodiments of the present application, the determination part 11 is also configured to pre-divide the point cloud data to obtain enhancement layer division parameters, estimated enhancement layer, detail layer division parameters and estimated detail layer; the pre-analysis includes: the pre-division; predicting the estimated enhancement layer and the estimated detail layer to determine the enhancement layer prediction error and the detail layer prediction error; pre-fitting based on the enhancement layer prediction error, the enhancement layer division parameters, the detail layer prediction error and the detail layer division parameters to determine the fitting parameters.

[0162] In some embodiments of the present application, the determination part 11 is also configured to perform a first prediction processing on the estimated enhancement layer to determine the enhancement layer prediction error of the estimated enhancement layer; and perform a second prediction processing on the estimated detail layer to determine the detail layer prediction error of the estimated detail layer.

[0163] In some embodiments of the present application, the fitting parameters include: a first fitting parameter and a second fitting parameter; the determination part 11 is also configured to perform pre-fitting processing using a first error model based on the enhancement layer prediction error and the enhancement layer division parameters to determine the first fitting parameters; and perform pre-fitting processing using a second error model based on the detail layer prediction error and the detail layer division parameters to determine the second fitting parameters.

[0164] In some embodiments of the present application, the determination part 11 is also configured to perform optimal layer calculation based on the fitting parameters using a preset layer number model to obtain the number of detail level division layers; wherein the preset layer number model represents the correspondence between the fitting parameters and the number of detail level division layers when the encoding performance is optimal.

[0165] In some embodiments of the present application, the optimal coding performance corresponds to the minimum prediction error.

[0166] In some embodiments of the present application, the determination part 11 is also configured to determine a preset prediction error model, wherein the preset prediction error model is constructed by weighting the mean absolute error of the enhancement layer and the mean absolute error of the detail layer, and the preset prediction error model includes a weighted ratio parameter.

[0167] In some embodiments of the present application, the determination part 11 is also configured to determine the correspondence between the division parameter and the weighted ratio parameter after determining the preset prediction error model, wherein the division parameter includes at least one of the enhancement layer division parameter and the detail layer division parameter; based on the correspondence and the preset prediction error model, determine the first prediction error model corresponding to the division parameter; construct a first functional relationship between the division parameter and the enhancement layer prediction error, and a second functional relationship between the division parameter and the prediction error of the detail layer; based on the first functional relationship and the second functional relationship, transform the first prediction error model to determine a prediction error minimization model; and according to the prediction error minimization model, determine the preset layer number model corresponding to the minimum prediction error.

[0168] In some embodiments of the present application, the determination part 11 is further configured to perform a derivative operation on the prediction error minimization model with respect to the number of division layers to determine the preset number of layers model.

[0169] In some embodiments of the present application, the determination part 11 is also configured to perform power function fitting on the first functional relationship and the second functional relationship respectively to obtain a first error model and a second error model with the division parameter as a variable; use the first error model and the second error model to transform the first prediction error model to determine the prediction error minimization model.

[0170] In some embodiments of the present application, the pre-division includes: division based on sampling distance.

[0171] In some embodiments of the present application, the enhancement layer division parameter is the sampling distance adopted by the current layer, and the detail layer division parameter is the distance between adjacent points of the current layer.

[0172] In some embodiments of the present application, the pre-division includes: division based on Morton codes.

[0173] In some embodiments of the present application, the enhancement layer division parameter is the number of bit right shifts of the Morton code of the current layer, and the detail layer division parameter is the number of bit right shifts of the Morton code of the previous layer based on the division order; wherein the Morton code is determined based on the position parameters of the midpoint of the point cloud data.

[0174] In some embodiments of the present application, the determining part 11 is further configured to set the number of detail level division levels to a value not greater than a division level threshold.

[0175] In some embodiments of the present application, the division layer number threshold is a preset value.

[0176] In some embodiments of the present application, the preset value is a value corresponding to a compliance parameter, wherein the compliance parameter includes at least one of the following parameters: grade, level, and level.

[0177] In some embodiments of the present application, the encoding part 13 is further configured to hierarchically divide the point cloud data according to the number of layers of the detail level division to obtain an enhancement layer and a detail layer; perform inter-layer prediction and intra-layer prediction on the enhancement layer to obtain a high-frequency coefficient; perform intra-layer prediction on the detail layer to obtain a low-frequency coefficient; and encode the high-frequency coefficient and the low-frequency coefficient.

[0178] In some embodiments of the present application, the encoding part 13 is further configured to hierarchically divide the point cloud data according to the number of detail level division layers, encode the number of detail level division layers, and write the encoded bits into the bit stream.

[0179] In practical applications, such as Figure 7 As shown, the embodiment of the present application also provides a point cloud segmentation device, including:

[0180] The memory 14 is used to store executable point cloud hierarchical division instructions;

[0181] The processor 15 is used to implement the hierarchical division method of point cloud data provided in the embodiment of the present application when executing the executable hierarchical division instruction stored in the memory 14.

[0182] Among them, the processor 15 can be implemented by software, hardware, firmware or a combination thereof, and can use circuits, single or multiple application specific integrated circuits (ASICs), single or multiple general-purpose integrated circuits, single or multiple microprocessors, single or multiple programmable logic devices, or a combination of the aforementioned circuits or devices, or other suitable circuits or devices, so that the processor 15 can execute the corresponding steps of hierarchical division of point cloud data in the embodiment of the aforementioned measurement platform.

[0183] It is understandable that, since the encoder can adopt simulated division of point cloud data and perform fitting processing in the process of encoding the point cloud data to be processed, and determine the fitting parameters that can ensure the optimal encoding performance, the fitting parameters can be used to further specifically confirm the number of detail level division layers that the point cloud data can be divided into. The number of detail level division layers is not necessarily the largest, but it can already ensure the optimal encoding performance, thereby achieving the accuracy of the number of detail division layers while ensuring the encoding performance.

[0184] The components in the embodiments of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software function modules.

[0185] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage media include: ferromagnetic random access memory (FRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic surface storage, optical disk, or compact disc read-only memory (CD-ROM), etc., which can store program codes, and the embodiments of the present disclosure are not limited thereto.

[0186] An embodiment of the present application provides a computer-readable storage medium storing executable point cloud hierarchical division instructions, which are used to cause a processor to execute and implement the hierarchical division method of point cloud data provided by the embodiment of the present application.

[0187] Industrial Applicability

[0188] The embodiments of the present application provide a method for hierarchical division of point cloud data, an encoder and a storage medium. During the process of encoding the point cloud data to be processed, the encoder can adopt simulated division for the point cloud data and perform fitting processing to determine the fitting parameters that can ensure the optimal encoding performance. In this way, the fitting parameters can be used to further specifically determine the number of detail level division layers into which the point cloud data can be divided. The number of detail level division layers is not necessarily the largest, but it can ensure the optimal encoding performance, thereby achieving the goal of improving the accuracy of the number of detail division layers while ensuring the encoding performance.

Claims

1. A hierarchical division method of point cloud data, applied to an encoder, characterized in that: include: Obtain the point cloud data to be processed; Pre-analyzing and pre-fitting the point cloud data to determine fitting parameters; Based on the fitting parameters, determining the number of detail level division layers with optimal coding performance; According to the number of detail level division layers, the point cloud data is hierarchically divided, and the point cloud data is encoded; The pre-analysis and pre-fitting of the point cloud data to determine the fitting parameters includes: Pre-dividing the point cloud data to obtain enhancement layer division parameters, estimated enhancement layer, detail layer division parameters and estimated detail layer; the pre-analysis includes: the pre-dividing; Predicting the estimated enhancement layer and the estimated detail layer to determine an enhancement layer prediction error and a detail layer prediction error; Perform pre-fitting based on the enhancement layer prediction error, the enhancement layer division parameter, the detail layer prediction error and the detail layer division parameter to determine the fitting parameter; The step of determining the number of detail level division layers with optimal coding performance based on the fitting parameters includes: According to the fitting parameters, the optimal layer calculation is performed using a preset layer number model to obtain the number of detail level division layers; wherein the preset layer number model represents the corresponding relationship between the fitting parameters and the number of detail level division layers when the encoding performance is optimal.

2. The method according to claim 1, characterized in that The predicting the estimated enhancement layer and the estimated detail layer to determine an enhancement layer prediction error and a detail layer prediction error comprises: performing a first prediction process on the estimated enhancement layer to determine the enhancement layer prediction error of the estimated enhancement layer; A second prediction process is performed on the estimated detail layer to determine the detail layer prediction error of the estimated detail layer.

3. The method according to claim 1, characterized in that The fitting parameters include: a first fitting parameter and a second fitting parameter; the pre-fitting based on the enhancement layer prediction error, the enhancement layer division parameter, the detail layer prediction error and the detail layer division parameter to determine the fitting parameters includes: According to the enhancement layer prediction error and the enhancement layer division parameter, a first error model is used to perform pre-fitting processing to determine a first fitting parameter; According to the detail layer prediction error and the detail layer division parameter, a second error model is used to perform pre-fitting processing to determine second fitting parameters.

4. The method according to claim 1, characterized in that The optimal coding performance corresponds to a minimum prediction error.

5. The method according to claim 4, characterized in that The method further comprises: A preset prediction error model is determined, wherein the preset prediction error model is constructed by weighting the enhancement layer mean absolute error and the detail layer mean absolute error, and the preset prediction error model includes a weighted ratio parameter.

6. The method according to claim 5, characterized in that After determining the preset prediction error model, the method further includes: Determining a correspondence between a partition parameter and the weighted ratio parameter, wherein the partition parameter includes at least one of an enhancement layer partition parameter and a detail layer partition parameter; Based on the corresponding relationship and the preset prediction error model, determining a first prediction error model corresponding to the partition parameter; Constructing a first functional relationship between the partition parameter and the enhancement layer prediction error, and a second functional relationship between the partition parameter and the detail layer prediction error; Based on the first functional relationship and the second functional relationship, the first prediction error model is converted to determine a prediction error minimization model; According to the prediction error minimization model, the preset layer number model corresponding to the minimum prediction error is determined.

7. The method according to claim 6, wherein determining the preset number of layers model corresponding to the minimum prediction error according to the prediction error minimization model comprises: The prediction error minimization model is subjected to a derivative operation with respect to the number of division layers to determine the preset number of layers model.

8. The method according to claim 6, characterized in that The converting the first prediction error model based on the first functional relationship and the second functional relationship to determine a prediction error minimization model includes: Performing power function fitting on the first functional relationship and the second functional relationship respectively to obtain a first error model and a second error model with the division parameter as a variable; The first error model and the second error model are used to transform the first prediction error model to determine the prediction error minimization model.

9. The method according to claim 1, characterized in that: The pre-division includes: division based on sampling distance.

10. The method according to claim 9, characterized in that The enhancement layer division parameter is the sampling distance used by the current layer, and the detail layer division parameter is the distance between adjacent points of the current layer.

11. The method according to claim 1, characterized in that The pre-division includes: division based on Morton codes.

12. The method according to claim 11, characterized in that The enhancement layer division parameter is the bit right shift number of the Morton code of the current layer, and the detail layer division parameter is the bit right shift number of the Morton code of the previous layer based on the division order; wherein the Morton code is determined according to the position parameter of the midpoint of the point cloud data.

13. The method according to claim 1, wherein determining the number of detail level division layers with optimal coding performance based on the fitting parameters comprises: The number of detail level division levels is set to a value not greater than a division level threshold.

14. The method according to claim 13, characterized in that include: The division layer number threshold is a preset value.

15. The method according to claim 14, characterized in that include: The preset value is a value corresponding to a compliance parameter, wherein the compliance parameter includes at least one of the following parameters: grade, level, and level.

16. The method according to claim 1, characterized in that The step of dividing the point cloud data into layers according to the number of layers of the detail level, and encoding or decoding the point cloud data comprises: According to the number of detail level division layers, the point cloud data is divided into levels to obtain an enhancement layer and a detail layer; Performing inter-layer prediction and intra-layer prediction on the enhancement layer to obtain high-frequency coefficients; Performing intra-layer prediction on the detail layer to obtain low-frequency coefficients; The high frequency coefficients and the low frequency coefficients are encoded.

17. The method according to claim 1, characterized in that The step of dividing the point cloud data into layers according to the number of layers of the detail level division and encoding the point cloud data comprises: The point cloud data is hierarchically divided according to the number of detail level division layers, and the number of detail level division layers is encoded, and the encoded bits are written into a bit stream.

18. An encoder, characterized in that: include: An acquisition part, configured to acquire point cloud data to be processed; The determination part is configured to pre-analyze and pre-fit the point cloud data to determine fitting parameters; and determine the number of detail level division layers with optimal encoding performance based on the fitting parameters; The determining part is specifically configured to pre-divide the point cloud data to obtain enhancement layer division parameters, estimated enhancement layers, detail layer division parameters and estimated detail layers; The pre-analysis includes: the pre-division; predicting the estimated enhancement layer and the estimated detail layer to determine the enhancement layer prediction error and the detail layer prediction error; pre-fitting based on the enhancement layer prediction error, the enhancement layer division parameter, the detail layer prediction error and the detail layer division parameter to determine the fitting parameter; performing optimal layer calculation using a preset layer number model according to the fitting parameter to obtain the number of detail level division layers; wherein the preset layer number model represents the corresponding relationship between the fitting parameter and the number of detail level division layers when the coding performance is optimal; A division part is configured to divide the point cloud data into levels according to the number of levels of detail division; The encoding part is configured to encode the point cloud data.

19. An encoder, characterized in that include: Memory and processor; The memory is used to store a computer program that can be run on the processor; The processor is configured to execute the method according to any one of claims 1 to 17 when running the computer program.

20. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores point cloud hierarchy partitioning instructions, which are used to cause the processor to implement the method according to any one of claims 1 to 17 when executing.

Images