Point cloud processing method and device, computer equipment and storage medium
The method optimizes point cloud attribute encoding and decoding by using prediction reference points and weighted reconstruction to enhance processing efficiency.
Patent Information
- Application Number
- CN202310647826.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-31
- Publication Date
- 2024-12-03
AI Technical Summary
The encoding and decoding performance of point cloud attributes is hindered by the large amount of attribute information, leading to inefficiencies in the processing of high-precision point clouds.
A point cloud processing method and device that determines prediction reference points to reconstruct attribute information using distance-based attribute prediction, incorporating weighting coefficients and exchange of reconstruction information to optimize encoding and decoding processes.
Improves the encoding and decoding performance of point cloud attributes by enhancing attribute prediction and reconstruction, thereby optimizing the processing efficiency of point clouds.
Smart Images

Figure CN119071486A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a point cloud processing method, a point cloud processing device, a computer equipment, and a computer-readable storage medium. Background Art
[0002] With the continuous development of science and technology, it is now possible to obtain a large number of high-precision point clouds at a low cost and in a short period of time. The point cloud can include multiple points, and each point in the point cloud has geometric information and attribute information. In order to improve the transmission efficiency of the point cloud, it is usually necessary to encode the point cloud before transmitting it; specifically, after the encoder encodes the geometric information and attribute information of each point in the point cloud, it can transmit the encoded point cloud to the decoding end, and the decoding end can decode the encoded point cloud to reconstruct the geometric information and attribute information of each point in the point cloud. Usually, the amount of attribute information of each point in the point cloud is large, and a large amount of attribute information causes pressure in the encoding and decoding process, making the encoding and decoding performance of the point cloud attributes not high. Therefore, how to improve the encoding and decoding performance of point cloud attributes has become a current research hotspot. Summary of the invention
[0003] The embodiments of the present application provide a point cloud processing method, apparatus, computer equipment, and storage medium, which can improve the encoding and decoding performance of point cloud attributes.
[0004] On the one hand, an embodiment of the present application provides a point cloud processing method, the point cloud processing method comprising:
[0005] Determine the points to be decoded in the point cloud;
[0006] Determine a prediction reference point of the point to be decoded in a candidate point set of the point to be decoded;
[0007] According to the distance between the to-be-decoded point and the prediction reference point, the attribute prediction information of the to-be-decoded point is obtained;
[0008] According to the attribute prediction information of the to-be-decoded point, the attribute of the to-be-decoded point is reconstructed to obtain the attribute reconstruction information of the to-be-decoded point.
[0009] Accordingly, an embodiment of the present application provides a point cloud processing device, the point cloud processing device comprising:
[0010] A determination unit, used to determine the points to be decoded in the point cloud;
[0011] A processing unit, configured to determine a prediction reference point of a point to be decoded from a set of candidate points of the point to be decoded;
[0012] The processing unit is further used to perform attribute prediction on the point to be decoded according to the distance between the point to be decoded and the prediction reference point, so as to obtain attribute prediction information of the point to be decoded;
[0013] The processing unit is further used to reconstruct the attributes of the points to be decoded according to the attribute prediction information of the points to be decoded, so as to obtain the attribute reconstruction information of the points to be decoded.
[0014] In one implementation, the processing unit is used to perform attribute prediction on the to-be-decoded point according to the distance between the to-be-decoded point and the prediction reference point, and when obtaining the attribute prediction information of the to-be-decoded point, specifically performs the following steps:
[0015] According to the geometric distance between the point to be decoded and the prediction reference point, k first prediction reference points whose geometric distances to the point to be decoded are less than a first distance threshold are obtained from the prediction reference points, and r second prediction reference points whose geometric distances to the point to be decoded are equal to the first distance threshold are obtained from the prediction reference points; k and r are both non-negative integers;
[0016] According to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points, attribute prediction is performed on the point to be decoded to obtain the attribute prediction information of the point to be decoded.
[0017] In one implementation, the processing unit is further configured to perform the following steps:
[0018] Selecting a third prediction reference point from the r second prediction reference points;
[0019] The processing unit is used to perform attribute prediction on the to-be-decoded point according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points, and when the attribute prediction information of the to-be-decoded point is obtained, specifically to perform the following steps:
[0020] According to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the third prediction reference point, attribute prediction is performed on the point to be decoded to obtain the attribute prediction information of the point to be decoded.
[0021] In one implementation, the processing unit is configured to perform the following steps when selecting the third prediction reference point from the r second prediction reference points:
[0022] Obtain the difference information between the attribute reconstruction information of any two second prediction reference points among the r second prediction reference points, and determine the two second prediction reference points corresponding to the largest difference information as the third prediction reference point; or, determine the second prediction reference point among the r second prediction reference points that is closest to the point to be decoded in the geometric space component as the third prediction reference point.
[0023] In one implementation, the processing unit is used to perform attribute prediction on the to-be-decoded point according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the to-be-decoded point, specifically to perform the following steps:
[0024] When k=0 and r>0, attribute prediction is performed on the point to be decoded according to the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the point to be decoded;
[0025] When k>0 and r>0, attribute prediction is performed on the point to be decoded according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the point to be decoded;
[0026] When k>0 and r=0, attribute prediction is performed on the point to be decoded according to the attribute reconstruction information of the k first prediction reference points to obtain the attribute prediction information of the point to be decoded.
[0027] In one implementation, the processing unit is used to perform attribute prediction on the to-be-decoded point according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the to-be-decoded point, specifically to perform the following steps:
[0028] Obtaining a first weighting coefficient for each of the k first prediction reference points, and obtaining a second weighting coefficient for each of the r second prediction reference points;
[0029] Obtain weight optimization coefficients of r second prediction reference points;
[0030] According to the weight optimization coefficient, optimizing the second weighting coefficient of each of the r second prediction reference points to obtain the second optimized weighting coefficient of each of the r second prediction reference points;
[0031] According to the first weighting coefficients of the k first prediction reference points and the second optimized weighting coefficients of the r second prediction reference points, the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points are weighted and summed to obtain the attribute prediction information of the point to be decoded.
[0032] In one implementation, the processing unit, when used to obtain the weight optimization coefficients of the r second prediction reference points, is used to perform any of the following:
[0033] Numerical calculation is performed on the total number of second prediction reference points to obtain a weight optimization coefficient;
[0034] Numerical calculation is performed on the total number of second prediction reference points and the attribute quantization step length of the to-be-decoded point to obtain a weight optimization coefficient;
[0035] According to the attribute coding condition satisfied by the to-be-decoded point, a weight optimization coefficient corresponding to the satisfied attribute coding condition is determined.
[0036] In one implementation, the point to be decoded belongs to a target point cloud group in the point cloud; the processing unit is used to predict the attribute of the point to be decoded according to the distance between the point to be decoded and the prediction reference point, and when the attribute prediction information of the point to be decoded is obtained, it is specifically used to perform the following steps:
[0037] When the index value of the to-be-decoded point in the target point cloud group is greater than the upper limit of the length of the candidate point set, the default prediction value determination distance is determined according to the geometric distance between the to-be-decoded point and the target prediction reference point in the prediction reference point;
[0038] If the default prediction value determination distance is greater than the second distance threshold, obtaining attribute default prediction information;
[0039] According to the attribute default prediction information, the attribute prediction of the point to be decoded is performed to obtain the attribute prediction information of the point to be decoded;
[0040] Among them, the target prediction reference point is the p prediction reference points among the prediction reference points whose geometric distance to the point to be decoded is closest or farthest, and p is a positive integer; the default prediction value determination distance includes any one of the following: the maximum geometric distance, the minimum geometric distance or the mean geometric distance among the geometric distances between the point to be decoded and the p prediction reference points; the attribute default prediction information includes any one of the following: fixed attribute default prediction information, and attribute default prediction information determined according to the attribute bit depth of the point to be decoded.
[0041] In one implementation, the points to be decoded in the point cloud are arranged in sequence according to the decoding order; the processing unit, when used to obtain the attribute default prediction information, is specifically used to perform the following steps:
[0042] Obtaining attribute determination information of a preceding point of the point to be decoded; the preceding point refers to a point in the point cloud that is located before the point to be decoded in decoding order; the attribute determination information includes a mean value of attribute reconstruction information or a mean value of attribute prediction information of the preceding point;
[0043] According to the relationship between the attribute determination information and the attribute determination threshold, the attribute default prediction information is determined.
[0044] In one implementation, the point to be decoded belongs to a target point cloud group in the point cloud; the processing unit is further configured to perform the following steps:
[0045] When the index value of the to-be-decoded point in the target point cloud group is greater than the upper limit of the length of the candidate point set, the exchange determination distance is obtained; the exchange determination distance includes any of the following: the geometric distance between the to-be-decoded point and the farthest predicted reference point among the predicted reference points, the geometric distance between the to-be-decoded point and the nearest predicted reference point among the predicted reference points, and the average of the geometric distances between the to-be-decoded point and the predicted reference points;
[0046] If the exchange determination distance is greater than or equal to the third distance threshold, the candidate point set remains unchanged;
[0047] If the exchange determination distance is less than the third distance threshold, the farthest candidate point with the farthest geometric distance from the point to be decoded is determined in the candidate point set, and the reconstruction information of the farthest candidate point is exchanged with the reconstruction information of the nth candidate point in the candidate point set;
[0048] The attribute reconstruction information of the to-be-decoded point is updated to the candidate point set, and the updated candidate point set is applied to the attribute prediction process of the subsequent to-be-decoded points of the to-be-decoded point.
[0049] In one implementation, there are multiple farthest candidate points; the processing unit is further configured to perform the following steps:
[0050] Select the farthest candidate point of the target among the farthest candidate points;
[0051] The processing unit is used to exchange the reconstruction information of the farthest candidate point with the reconstruction information of the nth candidate point in the candidate point set, and is specifically used to perform the following steps:
[0052] The reconstruction information of the farthest candidate point of the target is exchanged with the reconstruction information of the nth candidate point in the candidate point set.
[0053] In one implementation, the processing unit is used to select the target farthest candidate point from the farthest candidate points, and is specifically used to perform the following steps:
[0054] According to the order of the farthest candidate points in the candidate point set, the farthest candidate point of the target is selected from the farthest candidate points;
[0055] Selecting the target farthest candidate point from among the farthest candidate points according to the search order of the farthest candidate points in the candidate point set;
[0056] According to the attribute distance between each farthest candidate point and the point to be decoded, the farthest candidate point of the target is selected from each farthest candidate point;
[0057] According to the attribute distance between each farthest candidate point and the preceding point of the point to be decoded, the target farthest candidate point is selected from each farthest candidate point; the preceding point refers to the point that is located before the point to be decoded in the decoding order
[0058] According to the geometric distance between each farthest candidate point and the to-be-decoded point in the geometric space component, a target farthest candidate point is selected from each farthest candidate point.
[0059] In one implementation, the attribute encoding mode of the point to be decoded is an attribute prediction mode; the processing unit is used to select the target farthest candidate point from each farthest candidate point according to the attribute distance between each farthest candidate point and the preceding point of the point to be decoded, and is specifically used to perform the following steps:
[0060] Determine the attribute distance between the preceding point and the farthest candidate point according to the difference between the attribute reconstruction information of the preceding point and the attribute reconstruction information of the farthest candidate point;
[0061] Among the farthest candidate points, the farthest candidate point with the largest or smallest attribute distance to the previous point is selected as the target farthest candidate point.
[0062] In one implementation, the attribute coding mode of the point to be decoded is an attribute prediction transform coding mode, and the point to be decoded belongs to a target point cloud group in the point cloud; the processing unit is used to select the target farthest candidate point from each farthest candidate point according to the attribute distance between each farthest candidate point and the preceding point of the point to be decoded, and is specifically used to perform the following steps:
[0063] Select a target preceding point from the preceding points of the point to be decoded;
[0064] Determine the attribute distance between the target predecessor point and the farthest candidate point according to the difference between the attribute reconstruction information of the target predecessor point and the attribute reconstruction information of the farthest candidate point;
[0065] Among the farthest candidate points, the farthest candidate point with the largest or smallest attribute distance to the target predecessor point is selected as the target farthest candidate point.
[0066] In one implementation, the processing unit is used to select a target preamble point from preamble points of a point to be decoded, and is specifically used to perform the following steps:
[0067] The first pre-sequence point or the last pre-sequence point in the preceding point cloud group of the target point cloud group is selected as the target pre-sequence point; all the points to be decoded in the target point cloud group select the first pre-sequence point or the last pre-sequence point in the preceding point cloud group as the target pre-sequence point;
[0068] According to the position of the to-be-decoded point in the target point cloud group, a target preceding point is selected from the preceding points of the to-be-decoded point;
[0069] The preceding point cloud group is a point cloud group whose decoding order is before the target point cloud group.
[0070] In one implementation, the processing unit is used to select the target preceding point from the preceding points of the point to be decoded according to the position of the point to be decoded in the target point cloud group, and is specifically used to perform the following steps:
[0071] If the point to be decoded is the first point to be decoded in the target point cloud group, the first pre-sequence point or the last pre-sequence point in the pre-sequence point cloud group of the target point cloud group is selected as the target pre-sequence point;
[0072] If the point to be decoded is not the first point to be decoded in the target point cloud group, the preceding point in the target point cloud group that is arranged first, or the preceding point in the target point cloud group that is arranged one position before the point to be decoded in the decoding order, is selected as the target preceding point.
[0073] In one implementation, there is attribute information of multiple attribute types at the point to be decoded; the processing unit is used to select the target farthest candidate point from the farthest candidate points, and is specifically used to perform any of the following:
[0074] Select the target farthest candidate point under each attribute type from the farthest candidate points;
[0075] Selecting a target farthest candidate point under any of the following attribute types from the farthest candidate points: a first predicted attribute type, a second predicted attribute type, and a target attribute type; the target attribute type is any one of the multiple attribute types;
[0076] The farthest candidate point of the target is selected from the farthest candidate points based on the attribute weighted distance of multiple attribute types or the comprehensive distance of geometric attributes.
[0077] In one implementation, the point to be decoded belongs to a target point cloud group in the point cloud, and the target point cloud group is grouped according to the target right shift bit number; the processing unit is further used to perform the following steps:
[0078] According to the subdivision of the preceding point cloud grouping of the target point cloud grouping, the target right shift bit number is updated; the updated target right shift bit number is used as the grouping basis for the subsequent point cloud grouping of the target point cloud grouping;
[0079] Among them, the preceding point cloud group is a point cloud group whose decoding order is before the target point cloud group, and the subsequent point cloud group is a point cloud group whose decoding order is after the target point cloud group.
[0080] In one implementation, the processing unit is used to update the target right shift bit number according to the subdivision of the preceding point cloud group of the target point cloud group, and is specifically used to perform any of the following:
[0081] If the previous point cloud group is subdivided, the target right shift number is reduced, or when the target right shift number meets the update condition, the target right shift number is reduced;
[0082] If the previous point cloud group is not subdivided, the target right shift bit number is kept unchanged or increased;
[0083] The updating method of the target right shift bit number is determined according to the number of previous point cloud groups subdivided in the previous point cloud grouping.
[0084] In one implementation, there is attribute information of multiple attribute types at the point to be decoded; the processing unit is used to determine the prediction reference point of the point to be decoded from the candidate point set of the point to be decoded, and is specifically used to perform the following steps:
[0085] Obtaining the geometric distance between the point to be decoded and the candidate points in the candidate point set;
[0086] Obtain the decoded attribute distance between the point to be decoded and the candidate points in the candidate point set;
[0087] Determine the comprehensive distance between the point to be decoded and the corresponding candidate point according to the geometric distance between the point to be decoded and the corresponding candidate point, and the decoded attribute distance between the point to be decoded and the corresponding candidate point;
[0088] According to the comprehensive distance between the point to be decoded and the candidate points in the candidate point set, a prediction reference point of the point to be decoded is determined in the candidate point set.
[0089] In one implementation, the decoded attribute distance between the point to be decoded and the candidate point is determined according to the difference between the attribute reconstruction information of the point to be decoded under the decoded attribute type and the attribute reconstruction information of the candidate point under the decoded attribute type; when the decoded attribute type is a color attribute type, the processing unit, when used to obtain the decoded attribute distance between the point to be decoded and the candidate point in the candidate point set, is specifically used to perform the following steps:
[0090] Determine the attribute reconstruction information selected for the to-be-decoded point and the candidate point according to the color space;
[0091] The decoded attribute distance is determined according to the difference between the attribute reconstruction information selected for the point to be decoded and the attribute reconstruction information selected for the candidate point.
[0092] In one implementation, the processing unit is used to determine the attribute reconstruction information selected by the to-be-decoded point and the candidate point according to the color space, and is specifically used to perform the following steps:
[0093] When the color space is the first color space, the attribute reconstruction component of the to-be-decoded point under the target color component of the first color space is used as the attribute reconstruction information selected for the to-be-decoded point, and the attribute reconstruction component of the candidate point under the target color component of the first color space is used as the attribute reconstruction information selected for the candidate point;
[0094] When the color space is the second color space, the attribute reconstruction components of the point to be decoded under each color component in the second color space are used as the attribute reconstruction information selected for the point to be decoded, and the attribute reconstruction components of the candidate point under each color component in the second color space are used as the attribute reconstruction information selected for the candidate point.
[0095] In one implementation, the comprehensive distance between the point to be decoded and the corresponding candidate point is obtained by summing the geometric distance between the point to be decoded and the corresponding candidate point and the weighted result of the decoded attribute distance between the point to be decoded and the corresponding candidate point; the process of determining the weighted factor of the attribute distance between the point to be decoded and the corresponding candidate point includes:
[0096] Calculating the weighting factor according to the attribute quantization parameter;
[0097] The attribute quantization parameter is an attribute quantization parameter corresponding to a decoded attribute type, or an attribute quantization parameter corresponding to an attribute type to be decoded.
[0098] In one implementation, the processing unit is used to reconstruct the attributes of the to-be-decoded point according to the attribute prediction information of the to-be-decoded point, and after obtaining the attribute reconstruction information of the to-be-decoded point, is further used to perform the following steps:
[0099] Get the cutoff range of the attribute reconstruction information of the point to be decoded;
[0100] The attribute reconstruction information of the decoding point is truncated according to the truncation range.
[0101] In one implementation, the processing unit is used to obtain the truncation range of the attribute reconstruction information of the to-be-decoded point, and is specifically used to perform any of the following:
[0102] Parse the truncation range from the point cloud bitstream. The truncation range is calculated by the encoding device based on the attribute output bit depth.
[0103] Obtaining the truncation range corresponding to the attribute type of the point to be decoded;
[0104] The truncation range is determined according to the bit depth condition satisfied by the attribute output bit depth of the to-be-decoded point and the attribute type possessed by the to-be-decoded point.
[0105] In an embodiment of the present application, in the point cloud attribute decoding stage, attribute prediction is performed on the point to be decoded based on the distance between the point to be decoded and the predicted reference point, and the attribute approximation between the predicted reference point used for attribute prediction of the point to be decoded and the point to be decoded is taken into consideration. This can improve the attribute prediction effect of the point to be decoded and improve the decoding performance of the point cloud attributes.
[0106] On the other hand, an embodiment of the present application provides a point cloud processing method, the point cloud processing method comprising:
[0107] Determine the points to be encoded in the point cloud;
[0108] Determine a prediction reference point of the point to be coded from a set of candidate points of the point to be coded;
[0109] According to the distance between the to-be-coded point and the prediction reference point, the to-be-coded point is subjected to attribute prediction to obtain attribute prediction information of the to-be-coded point;
[0110] According to the attribute prediction information of the point to be encoded, attribute encoding is performed on the point to be encoded.
[0111] Accordingly, an embodiment of the present application provides a point cloud processing device, the point cloud processing device comprising:
[0112] A determination unit, used to determine the points to be encoded in the point cloud;
[0113] A processing unit, used to determine a prediction reference point of the point to be encoded in a candidate point set of the point to be encoded;
[0114] The processing unit is further used to perform attribute prediction on the to-be-coded point according to the distance between the to-be-coded point and the prediction reference point, so as to obtain attribute prediction information of the to-be-coded point;
[0115] The processing unit is further used to perform attribute encoding on the to-be-encoded point according to the attribute prediction information of the to-be-encoded point.
[0116] In one implementation, the processing unit is used to perform attribute prediction on the to-be-coded point according to the distance between the to-be-coded point and the prediction reference point, and when obtaining the attribute prediction information of the to-be-coded point, specifically performs the following steps:
[0117] According to the geometric distance between the point to be encoded and the prediction reference point, k first prediction reference points whose geometric distances to the point to be encoded are less than a first distance threshold are obtained from the prediction reference points, and r second prediction reference points whose geometric distances to the point to be encoded are equal to the first distance threshold are obtained from the prediction reference points; k and r are both non-negative integers;
[0118] According to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points, attribute prediction is performed on the point to be encoded to obtain the attribute prediction information of the point to be encoded.
[0119] In one implementation, the processing unit is further configured to perform the following steps:
[0120] Selecting a third prediction reference point from the r second prediction reference points;
[0121] The processing unit is used to perform attribute prediction on the to-be-coded point according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points, and when the attribute prediction information of the to-be-coded point is obtained, specifically to perform the following steps:
[0122] According to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the third prediction reference point, attribute prediction is performed on the point to be encoded to obtain attribute prediction information of the point to be encoded.
[0123] In one implementation, the processing unit is configured to perform the following steps when selecting the third prediction reference point from the r second prediction reference points:
[0124] Obtain the difference information between the attribute reconstruction information of any two second prediction reference points among the r second prediction reference points, and determine the two second prediction reference points corresponding to the largest difference information as the third prediction reference point; or, determine the second prediction reference point among the r second prediction reference points that is closest to the point to be encoded in the geometric space component as the third prediction reference point.
[0125] In one implementation, the processing unit is used to perform attribute prediction on the to-be-coded point according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the to-be-coded point, specifically to perform the following steps:
[0126] When k=0 and r>0, attribute prediction is performed on the point to be coded according to the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the point to be coded;
[0127] When k>0 and r>0, attribute prediction is performed on the point to be coded according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain attribute prediction information of the point to be coded;
[0128] When k>0 and r=0, attribute prediction is performed on the point to be encoded according to the attribute reconstruction information of the k first prediction reference points to obtain the attribute prediction information of the point to be encoded.
[0129] In one implementation, the processing unit is used to perform attribute prediction on the to-be-coded point according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the to-be-coded point, specifically to perform the following steps:
[0130] Obtaining a first weighting coefficient for each of the k first prediction reference points, and obtaining a second weighting coefficient for each of the r second prediction reference points;
[0131] Obtain weight optimization coefficients of r second prediction reference points;
[0132] According to the weight optimization coefficient, optimizing the second weighting coefficient of each of the r second prediction reference points to obtain the second optimized weighting coefficient of each of the r second prediction reference points;
[0133] According to the first weighting coefficients of the k first prediction reference points and the second optimized weighting coefficients of the r second prediction reference points, the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points are weightedly summed to obtain the attribute prediction information of the point to be encoded.
[0134] In one implementation, the processing unit, when used to obtain the weight optimization coefficients of the r second prediction reference points, is specifically used to perform any of the following:
[0135] Numerical calculation is performed on the total number of second prediction reference points to obtain a weight optimization coefficient;
[0136] Numerical calculation is performed on the total number of second prediction reference points and the attribute quantization step length of the to-be-coded point to obtain a weight optimization coefficient;
[0137] According to the attribute coding condition satisfied by the to-be-coded point, a weight optimization coefficient corresponding to the satisfied attribute coding condition is determined.
[0138] In one implementation, the point to be coded belongs to a target point cloud group in the point cloud; the processing unit is used to predict the attribute of the point to be coded according to the distance between the point to be coded and the prediction reference point, and when the attribute prediction information of the point to be coded is obtained, it is specifically used to perform the following steps:
[0139] When the index value of the to-be-coded point in the target point cloud group is greater than the upper limit of the length of the candidate point set, the default prediction value determination distance is determined according to the geometric distance between the to-be-coded point and the target prediction reference point in the prediction reference point;
[0140] If the default prediction value determination distance is greater than the second distance threshold, obtaining attribute default prediction information;
[0141] According to the attribute default prediction information, the attribute prediction of the to-be-coded point is performed to obtain the attribute prediction information of the to-be-coded point;
[0142] Among them, the target prediction reference point is the p prediction reference points among the prediction reference points whose geometric distance to the point to be encoded is closest or farthest, and p is a positive integer; the default prediction value determination distance includes any of the following: the maximum geometric distance, the minimum geometric distance or the mean geometric distance among the geometric distances between the point to be encoded and the p prediction reference points; the attribute default prediction information includes any of the following: fixed attribute default prediction information, and attribute default prediction information determined according to the attribute bit depth of the point to be encoded.
[0143] In one implementation, the points to be encoded in the point cloud are arranged in sequence according to the decoding order; the processing unit, when used to obtain the attribute default prediction information, is specifically used to perform the following steps:
[0144] Obtaining attribute determination information of a preceding point of the point to be encoded; the preceding point refers to a point in the point cloud that is located before the point to be encoded in a decoding order; the attribute determination information includes a mean value of attribute reconstruction information or a mean value of attribute prediction information of the preceding point;
[0145] According to the relationship between the attribute determination information and the attribute determination threshold, the attribute default prediction information is determined.
[0146] In one implementation, the point to be encoded belongs to a target point cloud group in the point cloud; the processing unit is further configured to perform the following steps:
[0147] When the index value of the to-be-coded point in the target point cloud group is greater than the upper limit of the length of the candidate point set, the exchange determination distance is obtained; the exchange determination distance includes any of the following: the geometric distance between the to-be-coded point and the farthest predicted reference point among the predicted reference points, the geometric distance between the to-be-coded point and the nearest predicted reference point among the predicted reference points, and the average of the geometric distances between the to-be-coded point and the predicted reference points;
[0148] If the exchange determination distance is greater than or equal to the third distance threshold, the candidate point set remains unchanged;
[0149] If the exchange determination distance is less than the third distance threshold, the farthest candidate point with the farthest geometric distance from the point to be encoded is determined in the candidate point set, and the reconstruction information of the farthest candidate point is exchanged with the reconstruction information of the nth candidate point in the candidate point set;
[0150] The attribute reconstruction information of the point to be encoded is updated to the candidate point set, and the updated candidate point set is applied to the attribute prediction process of the subsequent points to be decoded of the point to be encoded.
[0151] In one implementation, there are multiple farthest candidate points; the processing unit is further configured to perform the following steps:
[0152] Select the farthest candidate point of the target among the farthest candidate points;
[0153] The processing unit is used to exchange the reconstruction information of the farthest candidate point with the reconstruction information of the nth candidate point in the candidate point set, and is specifically used to perform the following steps:
[0154] The reconstruction information of the farthest candidate point of the target is exchanged with the reconstruction information of the nth candidate point in the candidate point set.
[0155] In one implementation, the processing unit is used to select the target farthest candidate point from the farthest candidate points, and is specifically used to perform the following steps:
[0156] According to the order of the farthest candidate points in the candidate point set, the farthest candidate point of the target is selected from the farthest candidate points;
[0157] Selecting the target farthest candidate point from among the farthest candidate points according to the search order of the farthest candidate points in the candidate point set;
[0158] According to the attribute distance between each farthest candidate point and the point to be encoded, the target farthest candidate point is selected from each farthest candidate point;
[0159] According to the attribute distance between each farthest candidate point and the preceding point of the point to be encoded, the target farthest candidate point is selected from each farthest candidate point; the preceding point refers to the point that is located before the point to be encoded in the encoding order;
[0160] According to the geometric distance between each farthest candidate point and the point to be encoded in the geometric space component, a target farthest candidate point is selected from each farthest candidate point.
[0161] In one implementation, the attribute encoding mode of the point to be encoded is an attribute prediction mode; the processing unit is used to select the target farthest candidate point from each farthest candidate point according to the attribute distance between each farthest candidate point and the preceding point of the point to be encoded, and is specifically used to perform the following steps:
[0162] Determine the attribute distance between the preceding point and the farthest candidate point according to the difference between the attribute reconstruction information of the preceding point and the attribute reconstruction information of the farthest candidate point;
[0163] Among the farthest candidate points, the farthest candidate point with the largest or smallest attribute distance to the previous point is selected as the target farthest candidate point.
[0164] In one implementation, the attribute coding mode of the point to be coded is the attribute prediction transformation coding mode, and the point to be coded belongs to the target point cloud group in the point cloud; the processing unit is used to select the target farthest candidate point from each farthest candidate point according to the attribute distance between each farthest candidate point and the preceding point of the point to be coded, and is specifically used to perform the following steps:
[0165] Select a target preceding point from the preceding points of the point to be encoded;
[0166] Determine the attribute distance between the target predecessor point and the farthest candidate point according to the difference between the attribute reconstruction information of the target predecessor point and the attribute reconstruction information of the farthest candidate point;
[0167] Among the farthest candidate points, the farthest candidate point with the largest or smallest attribute distance to the target predecessor point is selected as the target farthest candidate point.
[0168] In one implementation, the processing unit is used to select a target preceding point from the preceding points of the point to be encoded, and is specifically used to perform the following steps:
[0169] The first pre-sequence point or the last pre-sequence point in the preceding point cloud group of the target point cloud group is selected as the target pre-sequence point; all the points to be encoded in the target point cloud group select the first pre-sequence point or the last pre-sequence point in the preceding point cloud group as the target pre-sequence point;
[0170] According to the position of the point to be encoded in the target point cloud group, the target preceding point is selected from the preceding points of the point to be encoded;
[0171] The preceding point cloud group is a point cloud group whose encoding order is located before the target point cloud group.
[0172] In one implementation, the processing unit is used to select the target preceding point from the preceding points of the point to be encoded according to the position of the point to be encoded in the target point cloud group, and is specifically used to perform the following steps:
[0173] If the point to be encoded is the first point to be encoded in the target point cloud group, the first pre-sequence point or the last pre-sequence point in the pre-sequence point cloud group of the target point cloud group is selected as the target pre-sequence point;
[0174] If the point to be encoded is not the first point to be encoded in the target point cloud group, the preceding point that is arranged first in the target point cloud group, or the preceding point that is arranged one position before the point to be encoded in the target point cloud group in the coding order, is selected as the target preceding point.
[0175] In one implementation, the point to be encoded has attribute information of multiple attribute types; the processing unit is used to select the target farthest candidate point from the farthest candidate points, including any of the following:
[0176] Select the target farthest candidate point under each attribute type from the farthest candidate points;
[0177] Selecting a target farthest candidate point under any of the following attribute types from the farthest candidate points: a first predicted attribute type, a second predicted attribute type, and a target attribute type; the target attribute type is any one of the multiple attribute types;
[0178] The farthest predicted point of the target is selected from the farthest candidate points based on the attribute weighted distance of multiple attribute types or the comprehensive distance of geometric attributes.
[0179] In one implementation, the point to be encoded belongs to a target point cloud group in the point cloud, and the target point cloud group is grouped according to the target right shift bit number; the processing unit is further used to perform the following steps:
[0180] According to the subdivision of the preceding point cloud grouping of the target point cloud grouping, the target right shift bit number is updated; the updated target right shift bit number is used as the grouping basis for the subsequent point cloud grouping of the target point cloud grouping;
[0181] Among them, the preceding point cloud group is a point cloud group whose decoding order is before the target point cloud group, and the subsequent point cloud group is a point cloud group whose decoding order is after the target point cloud group.
[0182] In one implementation, the processing unit is used to update the target right shift bit number according to the subdivision of the preceding point cloud group of the target point cloud group, and is specifically used to perform any of the following:
[0183] If the previous point cloud group is subdivided, the target right shift number is reduced, or when the target right shift number meets the update condition, the target right shift number is reduced;
[0184] If the previous point cloud group is not subdivided, the target right shift bit number is kept unchanged or increased;
[0185] The updating direction of the target right shift bit number is determined according to the number of previous point cloud groups subdivided in the previous point cloud grouping.
[0186] In one implementation, there is attribute information of multiple attribute types for the point to be encoded; the processing unit is used to determine the predicted reference point of the point to be encoded in the candidate point set of the point to be encoded, and is specifically used to perform the following steps:
[0187] Obtain the geometric distance between the point to be encoded and the candidate points in the candidate point set;
[0188] Obtain the encoded attribute distance between the point to be encoded and the candidate points in the candidate point set;
[0189] Determine the comprehensive distance between the point to be coded and the corresponding candidate point according to the geometric distance between the point to be coded and the corresponding candidate point, and the coded attribute distance between the point to be coded and the corresponding candidate point;
[0190] According to the comprehensive distance between the point to be encoded and the candidate points in the candidate point set, the prediction reference point of the point to be encoded is determined in the candidate point set.
[0191] In one implementation, the coded attribute distance between the point to be coded and the candidate point is determined according to the difference between the attribute reconstruction information of the point to be coded under the coded attribute type and the attribute reconstruction information of the candidate point under the coded attribute type; when the coded attribute type is a color attribute type, the processing unit, when used to obtain the coded attribute distance between the point to be coded and the candidate point in the candidate point set, is specifically used to perform the following steps:
[0192] Determine the attribute reconstruction information selected for the points to be coded and the candidate points according to the color space;
[0193] The coded attribute distance is determined according to the difference between the attribute reconstruction information selected for the point to be coded and the attribute reconstruction information selected for the candidate point.
[0194] In one implementation, the processing unit is used to determine the attribute reconstruction information selected by the to-be-encoded point and the candidate point according to the color space, and is specifically used to perform the following steps:
[0195] When the color space is the first color space, the attribute reconstruction component of the to-be-encoded point under the target color component of the first color space is used as the attribute reconstruction information selected by the to-be-encoded point, and the attribute reconstruction component of the candidate point under the target color component of the first color space is used as the attribute reconstruction information selected by the candidate point;
[0196] When the color space is the second color space, the attribute reconstruction components of the point to be encoded under each color component in the second color space are used as the attribute reconstruction information selected for the point to be encoded, and the attribute reconstruction components of the alternative point under each color component in the second color space are used as the attribute reconstruction information selected for the alternative point.
[0197] In one implementation, the comprehensive distance between the point to be coded and the corresponding candidate point is obtained by summing the weighted results of the geometric distance between the point to be coded and the corresponding candidate point and the encoded attribute distance between the point to be coded and the corresponding candidate point; the process of determining the weighted factor of the attribute distance between the point to be coded and the corresponding candidate point includes:
[0198] Calculating weighting factors based on attribute quantification parameters;
[0199] The attribute quantization parameter is an attribute quantization parameter corresponding to a decoded attribute type, or an attribute quantization parameter corresponding to an attribute type to be decoded.
[0200] In one implementation, the processing unit is further configured to perform the following steps:
[0201] Obtaining a truncation range of the point to be coded, the truncation range of the point to be coded is used to truncate the attribute reconstruction information of the point to be coded during the attribute decoding process of the point to be coded;
[0202] Encode the truncated range of the points to be encoded into the point cloud code stream.
[0203] In one implementation, the processing unit, when used to obtain the truncation range of the point to be encoded, is specifically used to perform any of the following:
[0204] Calculate the cutoff range based on the attribute output bit depth;
[0205] Get the truncation range corresponding to the attribute type of the point to be coded;
[0206] The truncation range is determined according to the bit depth condition satisfied by the attribute output bit depth of the to-be-encoded point and the attribute type possessed by the to-be-encoded point.
[0207] In an embodiment of the present application, in the point cloud attribute coding stage, attribute prediction is performed on the point to be coded based on the distance between the point to be coded and the prediction reference point, and the attribute approximation between the prediction reference point used for attribute prediction of the point to be coded and the point to be coded is taken into consideration. This can improve the attribute prediction effect of the point to be coded and improve the coding performance of the point cloud attributes.
[0208] Accordingly, an embodiment of the present application provides a computer device, the computer device comprising:
[0209] a processor suitable for implementing a computer program;
[0210] Computer-readable storage medium, the computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the above-mentioned point cloud processing method.
[0211] Accordingly, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is read and executed by a processor of a computer device, the computer device executes the above-mentioned point cloud processing method.
[0212] Accordingly, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned point cloud processing method.
[0213] In an embodiment of the present application, in the point cloud attribute decoding stage, attribute prediction is performed on the point to be decoded according to the distance between the point to be decoded and the prediction reference point, and the attribute approximation between the prediction reference point used for attribute prediction of the point to be decoded and the point to be decoded is taken into account, which can improve the attribute prediction effect of the point to be decoded and improve the decoding performance of the point cloud attributes. In the point cloud attribute encoding stage, attribute prediction is performed on the point to be encoded according to the distance between the point to be encoded and the prediction reference point, and the attribute approximation between the prediction reference point used for attribute prediction of the point to be encoded and the point to be encoded is taken into account, which can improve the attribute prediction effect of the point to be encoded and improve the encoding performance of the point cloud attributes. Therefore, the embodiment of the present application can improve the encoding and decoding performance of point cloud attributes. BRIEF DESCRIPTION OF THE DRAWINGS
[0214] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0215] Figure 1 It is a schematic diagram of a coding framework of a point cloud coding provided in an embodiment of the present application;
[0216] Figure 2 It is a schematic diagram of the architecture of a point cloud processing system provided in an embodiment of the present application;
[0217] Figure 3 It is a flowchart of a point cloud processing method provided in an embodiment of the present application;
[0218] Figure 4 It is a flowchart of another point cloud processing method provided in an embodiment of the present application;
[0219] Figure 5 is a structural schematic diagram of a point cloud processing device provided in an embodiment of the present application;
[0220] Figure 6 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0221] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0222] In order to more clearly understand the technical solution provided by the embodiments of the present application, the basic concepts involved in the embodiments of the present application are first introduced here:
[0223] (1) Point Cloud. A point cloud is a set of irregularly distributed discrete points in space that express the spatial structure and surface properties of a three-dimensional object or scene. Point clouds can be divided into different categories according to different classification standards. For example, according to the way point clouds are acquired, they can be divided into dense point clouds and sparse point clouds. For example, according to the time series type of point clouds, they can be divided into static point clouds and dynamic point clouds.
[0224] (2) Point Cloud Data. The geometric information and attribute information of each point in the point cloud together constitute the point cloud data. Geometric information can also be called three-dimensional position information. The geometric information of a point in the point cloud refers to the spatial coordinates (x, y, z) of the point, which can include the coordinate values of the point in each coordinate axis direction of the three-dimensional coordinate system, for example, the coordinate value x in the X-axis direction, the coordinate value y in the Y-axis direction, and the coordinate value z in the Z-axis direction. The attribute information of a point in the point cloud may include at least one of the following: color information, material information, laser reflection intensity information (also called reflectivity); usually, each point in the point cloud has the same amount of attribute information. For example, each point in the point cloud can have two kinds of attribute information: color information and laser reflection intensity. For another example, each point in the point cloud can have three kinds of attribute information: color information, material information, and laser reflection intensity information.
[0225] (3) Point Cloud Compression (PCC). Point cloud coding refers to the process of encoding the geometric information and attribute information of each point in the point cloud to obtain a compressed code stream (also called a point cloud code stream). Point cloud coding can include two main processes: geometric information encoding and attribute information encoding. The current mainstream point cloud coding technology can be divided into point cloud coding based on geometry and point cloud coding based on projection according to different types of point clouds. Here, G-PCC (Geometry-Based Point Cloud Compression) in MPEG (Moving Picture Expert Group, an international audio and video coding standard) and AVS-PCC, a point cloud coding standard in AVS (Audio Video Coding Standard, China's national video coding standard) are used as examples for introduction.
[0226] The coding frameworks of G-PCC and AVS-PCC are roughly the same. Taking G-PCC as an example, Figure 1 The coding framework of G-PCC is shown, which can be divided into the geometric information coding process and the attribute information coding process. In the geometric information coding process, the geometric information of each point in the point cloud is encoded to obtain a geometric bit stream; in the attribute information coding process, the attribute information of each point in the point cloud is encoded to obtain an attribute bit stream; the geometric bit stream and the attribute bit stream together constitute the compressed code stream of the point cloud.
[0227] For the geometric information encoding process, the main operations and processing can be described as follows:
[0228] ① Pre-Processing: It can include coordinate transformation and voxelization. Through scaling and translation operations, the point cloud data in the three-dimensional space is converted into integer form, and its minimum geometric position is moved to the coordinate origin.
[0229] ② Geometric coding: Geometric coding can include two modes, namely, octree-based geometric coding (Octree) and triangle-based geometric coding (Trisoup). These two coding modes can be used under different conditions. Among them:
[0230] Octree-based geometric coding: Octree is a tree data structure that evenly divides the pre-set bounding box in three-dimensional space. Each node has eight child nodes. By using "1" and "0" to indicate whether each child node of the octree is occupied or not, the occupancy code information (Occupancy Code) is obtained as the code stream of the point cloud geometry information.
[0231] Geometric coding based on triangular representation: Divide the point cloud into blocks of a certain size, locate the intersection points of the point cloud surface at the edge of the block and construct triangles. The geometric information is compressed by encoding the intersection points.
[0232] ③ Geometry Quantization: The degree of quantization is usually determined by the quantization parameter (QP). The larger the QP value, the larger the range of coefficients will be quantized to the same output, which usually results in greater distortion and a lower bit rate. On the contrary, the smaller the QP value, the smaller the range of coefficients will be quantized to the same output, which usually results in less distortion and a higher bit rate.
[0233] ④ Geometry Entropy Encoding: Statistical compression encoding is performed on the occupancy code information of the octree, and finally a binary (0 or 1) compressed code stream is output. Statistical coding is a lossless coding method that can effectively reduce the bit rate required to express the same signal. The commonly used statistical coding method is context-based binary arithmetic coding (Content Adaptive Binary Arithmetic Coding, CABAC).
[0234] For the attribute information encoding process, the main operations and processing can be seen in the following description:
[0235] ① Attribute recoloring: In the case of lossy coding, after the geometric coordinate information is encoded, the encoder needs to decode and reconstruct the geometric information, that is, restore the geometric information of each point in the point cloud. Find the attribute information corresponding to one or more neighboring points in the original point cloud as the attribute information of the reconstructed point.
[0236] ② Attribute information processing: Attribute information processing can include three attribute encoding modes, namely, attribute prediction (Prediction), attribute transformation (Transform), and attribute prediction transformation (Prediction & Transform). These three attribute encoding modes can be used under different conditions. Among them:
[0237] Attribute prediction: According to information such as distance or spatial relationship, the prediction reference point of the point to be encoded is determined in the encoded points, and the attribute prediction information of the point to be encoded is calculated based on the attribute information of the prediction reference point based on the set criteria. The difference between the actual attribute information of the point to be encoded and the predicted attribute information is calculated as the attribute residual information, and the attribute residual information is quantized and entropy encoded.
[0238] Attribute transformation: using DCT (Discrete Cosine Transform) and Haar (Haar Transform) and other transformation methods, the attribute information is grouped and transformed to obtain transformation coefficients, and the transformation coefficients are quantized and entropy encoded.
[0239] Attribute prediction transformation: The preceding steps are the same as attribute prediction. After obtaining the attribute residual information of the point to be encoded, the attribute residual information of the point to be encoded is transformed using a transformation algorithm to obtain transformation coefficients, and then the transformation coefficients are quantized and entropy encoded.
[0240] ③Attribute Quantization: The degree of quantization is usually determined by the quantization parameter. The transform coefficients and / or attribute residual information obtained by attribute information processing are quantized, and the quantized results are entropy coded. For example, in attribute prediction, the quantized attribute residual information is entropy coded; in attribute transformation and attribute prediction transformation, the quantized transform coefficients are entropy coded.
[0241] ④ Attribute Entropy Coding: Quantized attribute residual information and / or transform coefficients are generally compressed using entropy coding methods such as Run Length Coding and Arithmetic Coding. The corresponding attribute coding mode, quantization parameters and other information are also encoded using entropy encoders.
[0242] (4) Point cloud decoding. Point cloud decoding refers to the process of decoding the compressed bitstream obtained by point cloud encoding to reconstruct the point cloud; in detail, the geometric bitstream in the compressed bitstream is parsed to reconstruct the geometric information of each point in the point cloud, and the attribute bitstream in the compressed bitstream is parsed to reconstruct the attribute information of each point in the point cloud. Here is a detailed introduction to the point cloud decoding process in attribute prediction mode and attribute prediction transformation mode:
[0243] In the attribute prediction mode, the quantized attribute residual information of the point to be decoded can be obtained from the attribute bit stream through entropy decoding, and the quantized attribute residual information can be dequantized to obtain the attribute residual information of the point to be decoded; according to information such as distance or spatial relationship, the prediction reference point of the point to be decoded is determined in the decoded points, and based on the set criteria, the attribute prediction information of the point to be decoded is calculated according to the attribute reconstruction information of the prediction reference point; then, the attribute information of the point to be decoded (which can be called attribute reconstruction information) can be reconstructed according to the attribute residual information of the point to be decoded and the attribute prediction information of the point to be decoded.
[0244] In the attribute prediction transformation mode, the quantized transformation coefficients of the point to be decoded can be obtained from the attribute bit stream through entropy decoding, the quantized transformation coefficients can be dequantized to obtain the transformation coefficients, and the transformation coefficients can be inversely transformed to obtain the attribute residual information of the point to be decoded; according to information such as distance or spatial relationship, the prediction reference point of the point to be decoded is determined among the decoded points, and based on the set criteria, the attribute prediction information of the point to be decoded is calculated according to the attribute reconstruction information of the prediction reference point; then, the attribute information of the point to be decoded (which can be called attribute reconstruction information) can be reconstructed according to the attribute residual information of the point to be decoded and the attribute prediction information of the point to be decoded.
[0245] Based on the introduction of the above basic concepts, an embodiment of the present application proposes a point cloud processing method. In the point cloud attribute decoding stage, the point cloud processing method predicts the attributes of the point to be decoded according to the distance between the point to be decoded and the predicted reference point, and takes into account the attribute approximation between the predicted reference point used for attribute prediction of the point to be decoded and the point to be decoded, which can improve the attribute prediction effect of the point to be decoded and improve the decoding performance of the point cloud attributes; in the point cloud attribute encoding stage, the point cloud processing method predicts the attributes of the point to be encoded according to the distance between the point to be encoded and the predicted reference point, and takes into account the attribute approximation between the predicted reference point used for attribute prediction of the point to be encoded and the point to be encoded, which can improve the attribute prediction effect of the point to be encoded and improve the encoding performance of the point cloud attributes.
[0246] In the following, in conjunction with the accompanying drawings, a point cloud processing system suitable for implementing the point cloud processing method provided in an embodiment of the present application is introduced.
[0247] like Figure 2As shown, the point cloud processing system may include an encoding device 201 and a decoding device 202. The encoding device 201 may be a terminal or a server, and the decoding device 202 may be a terminal or a server. The encoding device 201 and the decoding device 202 may establish a direct communication connection through wired communication, or may establish an indirect communication connection through wireless communication. Among them, the terminal may be a smart phone, a tablet computer, a laptop computer, a desktop computer, an intelligent voice interaction device, a smart watch, a vehicle terminal, a smart home appliance, and an aircraft, etc., but is not limited thereto. The server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.
[0248] (1) For the encoding device 201:
[0249] The encoding device 201 can obtain point cloud data (i.e., geometric information and attribute information of each point in the point cloud), and the point cloud data can be obtained by scene capture or device generation. Scene capture point cloud data refers to point cloud data obtained by collecting real-world visual scenes through the capture device associated with the encoding device 201; wherein, the capture device is used to provide point cloud data acquisition services for the encoding device 201, and the capture device may include but is not limited to any of the following: camera equipment, sensor equipment, and scanning equipment; wherein, the camera equipment may include ordinary cameras, stereo cameras, and light field cameras, etc., the sensor equipment may include laser equipment, radar equipment, etc., and the scanning equipment may include three-dimensional laser scanning equipment, etc.; the capture device associated with the encoding device 201 may refer to a hardware component set in the encoding device 201, for example, the capture device is a camera, sensor, etc. of the terminal, and the capture device associated with the encoding device 201 may also refer to a hardware device connected to the encoding device 201, such as a camera connected to a server, etc. Device generation point cloud data refers to the point cloud data generated by the encoding device 201 according to virtual objects (for example, virtual three-dimensional objects and virtual three-dimensional scenes obtained through three-dimensional modeling, etc.).
[0250] The encoding device 201 can encode the geometric information of each point in the point cloud to obtain a geometric bit stream, and can encode the attribute information of each point in the point cloud to obtain an attribute bit stream. The encoding device 201 can transmit the encoded geometric bit stream and the attribute bit stream together to the decoding device 202. In particular, in the encoding process of the point cloud attribute, the encoding device 201 can perform attribute prediction on the point to be encoded according to the distance between the point to be encoded and the prediction reference point, and obtain the attribute prediction information of the point to be encoded. The attribute approximation between the prediction reference point used for attribute prediction of the point to be encoded and the point to be encoded is considered, which can improve the attribute prediction effect of the point to be encoded and improve the encoding performance of the point cloud attribute.
[0251] (2) For the decoding device 202:
[0252] After receiving the compressed code stream (i.e., the attribute bit stream and the geometry bit stream) transmitted by the encoding device 201, the decoding device 202 can decode the geometry bit stream to reconstruct the geometry information of each point in the point cloud (which can be called geometry reconstruction information), and can decode the attribute bit stream to reconstruct the attribute information of each point in the point cloud (which can be called attribute reconstruction information). In particular, in the decoding process of the point cloud attribute, the decoding device 202 can perform attribute prediction on the point to be decoded according to the distance between the point to be decoded and the prediction reference point, and obtain the attribute prediction information of the point to be decoded. Considering the attribute approximation between the prediction reference point used for attribute prediction of the point to be decoded and the point to be decoded, the attribute prediction effect of the code point to be decoded can be improved, and the decoding performance of the point cloud attribute can be improved.
[0253] It can be understood that the point cloud processing system described in the embodiment of the present application is to more clearly illustrate the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided in the embodiment of the present application. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.
[0254] The point cloud processing method provided in the embodiment of the present application is described in more detail below with reference to the accompanying drawings.
[0255] The present application embodiment provides a point cloud processing method, which mainly introduces the point cloud attribute decoding process. The point cloud processing method can be executed by a computer device, which can be the decoding device 202 in the above-mentioned point cloud processing system. Figure 3 As shown, the point cloud processing method may include but is not limited to the following steps S301 to S304:
[0256] S301, determining the points to be decoded in the point cloud.
[0257] As described above, a point cloud refers to a discrete point set. A point cloud may include multiple points. Each point in the point cloud is attribute-decoded in sequence according to the attribute decoding order. The points to be decoded are points in the point cloud that have not been attribute-decoded.
[0258] S302: Determine a prediction reference point of the point to be decoded in a candidate point set of the point to be decoded.
[0259] The candidate point set of the point to be decoded is a decoded point set in the point cloud used to determine the predicted reference point of the point to be decoded. The predicted reference point of the point to be decoded can be determined in the candidate point set of the point to be decoded. The candidate point set may include reconstruction information of each candidate point, and the reconstruction information may include geometric reconstruction information and attribute reconstruction information. The geometric reconstruction information refers to geometric information reconstructed by geometric decoding, and the attribute reconstruction information refers to attribute information reconstructed by attribute decoding.
[0260] When determining the predicted reference point of the point to be decoded from the candidate point set of the point to be decoded, it is necessary to consider whether the number of attribute types of the point to be decoded is single or multiple; if the number of attribute types of the point to be decoded is single, it can indicate that the point to be decoded has attribute information under one attribute type, for example, the point to be decoded has color information or reflectivity information; if the number of attribute types of the point to be decoded is multiple, it can indicate that the point to be decoded has attribute information under multiple attribute types, for example, the point to be decoded has color information and reflectivity information. The following introduces the method of determining the predicted reference point of the point to be decoded when the point to be decoded has attribute information under one attribute type and when the point to be decoded has attribute information under multiple attribute types:
[0261] 1. The point to be decoded has attribute information under an attribute type:
[0262] When the point to be decoded has attribute information under an attribute type, the method of determining the prediction reference point of the point to be decoded may include any one of the following: prediction reference point search based on point cloud sequence and prediction reference point search based on point cloud sequence and spatial offset. The following are two methods for determining the prediction reference point of the point to be decoded:
[0263] (1) Prediction reference point search based on point cloud sequence:
[0264] After geometrically decoding each point in the point cloud, the geometric reconstruction information of the point can be converted into a Hilbert code (or Morton code), and then the points in the point cloud are sorted in ascending order according to the Hilbert code (or Morton code), and the points in the reordered point cloud sequence (which can be called the Hilbert sequence (or Morton sequence)) are sequentially attribute decoded. The candidate point set can be a decoded point set consisting of decoded points whose decoding order is before the point to be decoded.
[0265] The point to be decoded belongs to the target point cloud group in the point cloud. Let the target point cloud group have L points, s represents the index value of the point to be decoded in the target point cloud group, and the point to be decoded P i is the nth point in the group, L>n≥0. Let the candidate point set be Sp, and the upper limit of the length of the candidate point set Sp is maxNumOfNeighbours, that is, the candidate point set Sp contains at most maxNumOfNeighbours candidate points. Sp contains the reconstruction information of the candidate points (including geometric reconstruction information and attribute reconstruction information). For the point to be decoded P i , the calculation method of the prediction reference point is as follows:
[0266] If s=0 (i.e., the index value of the point to be decoded in the target point cloud group is 0), there is no need to search, and the attribute default prediction information is directly used as the attribute prediction information of the point to be decoded. For example, taking the color attribute as an example, the color attribute default prediction information [128, 128, 128] is used as the attribute prediction information of the point to be decoded.
[0267] If s=1 (that is, the index value of the point to be decoded in the target point cloud group is 1, then the candidate point set can be made P0 in Sp as the prediction reference point.
[0268] If s=2 (ie, the index value of the point to be decoded in the target point cloud group is 2), then P0 and P1 in the candidate point set Sp can be used as prediction reference points.
[0269] If maxNumOfNeighbours≥s≥3 (i.e., the index value of the point to be decoded in the target point cloud group is greater than or equal to 3, and the upper limit of the length of the candidate point set Sp is greater than or equal to the index value of the point to be decoded in the target point cloud group), then the candidate point set can be P in Sp. s-1 , P s-2 and P s-3 is the reference point for prediction.
[0270] If s>maxNumOfNeighbours (i.e. the index value of the point to be decoded in the target point cloud group is greater than the upper limit of the length of the candidate point set Sp), the point to be decoded P can be traversed in the candidate point set Sp. i (x i ,y i , z i ) and up to 13 equidistant points as the prediction reference points P of the points to be decoded. j (x j ,y j , z j ), where the distance refers to the geometric distance. The geometric distance d can be calculated using Manhattan distance, which is defined as: d = |x i -x j|+|y i -y j |+|z i -z j |. If the distance between two points is the same, the point that is traversed first is preferred. The search method for the 3 closest points and up to 13 equidistant points as the prediction points of the current point is as follows:
[0271] ① The closest point set is initialized to an empty set. When the number of points in the predicted reference point set is less than 3, that is, when processing the first 3 points of the to-be-decoded point in the candidate point set Sp, insert these 3 points into the closest point set in order, and after each point is inserted, sort the points in the closest point set in ascending order according to the distance between the to-be-decoded points in the closest point set, so that the distances of the three points after sorting satisfy d1≤d2≤d3. Here d k Represents the geometric distance between the kth point in the set of closest points and the point to be decoded, k = 1, 2, 3.
[0272] ② Continue to search forward for the nth point in the candidate point set Sp. When the number of points in the closest point set is equal to 3, if the distance d between the nth point and the point to be decoded is n is equal to d3, then the nth point can be added to the set of equidistant points; if the distance d between the nth point and the point to be decoded n If it is less than d3, then the point corresponding to d3 can be moved out of the set of closest points and d temp = d3, and add n points to the closest point set. After adding the new point, sort the points in the closest point set in ascending order according to the geometric distance between the point in the closest point set and the point to be decoded. Then compare the distance d3 after reordering with the removed point. temp For comparison, if d temp = d3, then the removed points can be added to the equidistant point set, otherwise, the equidistant point set can be cleared;
[0273] ③ Until all points in the candidate point set Sp are searched, finally 3 closest points and up to 13 equidistant points are obtained as prediction reference points for the points to be decoded.
[0274] (2) Prediction reference point search based on point cloud sequence and spatial offset:
[0275] After geometrically decoding each point in the point cloud, the coordinate value of the Z axis in the geometric reconstruction information of the point can be multiplied by θ (spatial offset coefficient (axisBias)), and then the geometric reconstruction information of the point can be converted into a Hilbert code (or Morton code), and then the points in the point cloud are sorted in ascending order according to the Hilbert code (or Morton code), and the points in the reordered point cloud sequence (which can be called the Hilbert sequence (or Morton sequence)) are sequentially attribute decoded. The candidate point set can be a decoded point set consisting of decoded points whose decoding order is before the point to be decoded.
[0276] The point to be decoded belongs to the target point cloud group in the point cloud. Let the target point cloud group have L points, s represents the index value of the point to be decoded in the target point cloud group, and the point to be decoded P i is the nth point in the group, L>n≥0. Let the candidate point set be Sp, and the upper limit of the length of the candidate point set Sp is maxNumOfNeighbours, that is, the candidate point set Sp contains at most maxNumOfNeighbours candidate points. Sp contains the reconstruction information of the candidate points (including geometric reconstruction information and attribute reconstruction information). For the point to be decoded P i , the calculation method of the prediction reference point is as follows:
[0277] If s=0 (i.e., the index value of the point to be decoded in the target point cloud group is 0), there is no need to search, and the attribute default prediction information is directly used as the attribute prediction information of the point to be decoded. For example, taking the reflectivity attribute as an example, the reflectivity attribute default prediction information [0] is used as the attribute prediction information of the point to be decoded.
[0278] If s=1 (ie, the index value of the point to be decoded in the target point cloud group is 1), then P0 in the candidate point set Sp can be used as the prediction reference point.
[0279] If s=2 (ie, the index value of the point to be decoded in the target point cloud group is 2), then P0 and P1 in the candidate point set Sp can be used as prediction reference points.
[0280] If maxNumOfNeighbours≥s≥3 (i.e., the index value of the point to be decoded in the target point cloud group is greater than or equal to 3, and the upper limit of the length of the candidate point set Sp is greater than or equal to the index value of the point to be decoded in the target point cloud group), then the candidate point set can be P in Sp. s-1 , P s-2 and P s-3 is the reference point for prediction.
[0281] If s>maxNumOfNeighbours (i.e. the index value of the point to be decoded in the target point cloud group is greater than the upper limit of the length of the candidate point set Sp), the point to be decoded P can be traversed in the candidate point set Sp. i(x i ,y i , z i ) as the prediction reference point P of the point to be decoded j (x j ,y j , z j ), where the distance refers to the geometric distance. The geometric distance d can be calculated using Manhattan distance, which is defined as: d = |x i -x j |+|y i -y j |+|z i -z j |. If the distance between two points is the same, the point traversed first can be given priority.
[0282] 2. The point to be decoded has attribute information under an attribute type:
[0283] When the point to be decoded has attribute information under multiple attribute types (i.e., a point to be decoded with multiple attributes), the method of determining the prediction reference point of the point to be decoded may include:
[0284] Let the candidate point set be Sp, and the upper limit of the length of the candidate point set Sp is maxNumOfNeighbours, that is, the candidate point set Sp contains at most maxNumOfNeighbours candidate points. Sp contains the reconstruction information of the candidate points (including geometric reconstruction information and attribute reconstruction information).
[0285] If cross-type attribute prediction is not allowed, the prediction reference points of the to-be-decoded points under each attribute type may be determined respectively according to the above method of determining the prediction reference points of the to-be-decoded points when the to-be-decoded points have attribute information under one attribute type.
[0286] If cross-type attribute prediction is allowed, the prediction reference point of the point to be decoded can be determined in the candidate point set Sp according to the above method of determining the prediction reference point of the point to be decoded under any attribute type when the point to be decoded has attribute information under one attribute type, and attribute prediction is performed based on the prediction reference point under the corresponding attribute type, and then attribute reconstruction is performed to obtain the attribute reconstruction information of the point to be decoded under the corresponding attribute type, and the corresponding attribute type can be called the decoded attribute type. For the attribute type to be decoded, the geometric distance disGeom between the point to be decoded and the candidate points in the candidate point set Sp can be obtained; the decoded attribute distance disAttr between the point to be decoded and the candidate points in the candidate point set Sp can be obtained; the comprehensive distance dis between the point to be decoded and the corresponding candidate point can be determined based on the geometric distance between the point to be decoded and the corresponding candidate point, and the decoded attribute distance between the point to be decoded and the corresponding candidate point; then, the prediction reference point of the point to be decoded can be determined in the candidate point set based on the comprehensive distance between the point to be decoded and the candidate points in the candidate point set. Among them:
[0287] (1) For the geometric distance disGeom:
[0288] The geometric distance disGeom between the point to be decoded and the candidate point can be calculated using Manhattan distance.
[0289] (2) For the decoded attribute distance disAttr:
[0290] The decoded attribute distance disAttr between the point to be decoded and the candidate point is determined according to the difference between the attribute reconstruction information of the point to be decoded under the decoded attribute type and the attribute reconstruction information of the candidate point under the decoded attribute type.
[0291] When the decoded attribute type is the reflectivity attribute, the calculation formula of the decoded attribute distance disAttr can be referred to the following formula 1:
[0292] disAttr=|curRefl-neighRefl| Formula 1
[0293] In the above formula 1, curRefl represents the reflectivity attribute reconstruction information of the point to be decoded, and neighRefl represents the reflectivity attribute reconstruction information of the candidate point.
[0294] When the decoded attribute type is a color attribute, the attribute reconstruction information selected by the to-be-decoded point and the candidate point can be determined according to the color space; then, the decoded attribute distance can be determined according to the difference between the attribute reconstruction information selected by the to-be-decoded point and the attribute reconstruction information selected by the candidate point.
[0295] When the color space is the first color space, the attribute reconstruction component of the point to be decoded under the target color component of the first color space can be used as the attribute reconstruction information selected by the point to be decoded, and the attribute reconstruction component of the candidate point under the target color component of the first color space can be used as the attribute reconstruction information selected by the candidate point; that is, the decoded attribute distance disAttr can be determined according to the difference between the attribute reconstruction component of the point to be decoded under the target color component of the first color space and the attribute reconstruction component of the candidate point under the target color component of the first color space; this method can be applied to the case where the attribute reconstruction component under the target color component has most of the information in the attribute reconstruction information of the first color space. For example, if the first color space is YUV (a color space), the Y component (brightness component) can be used to calculate the decoded attribute distance, and the calculation formula can be seen in the following formula 2:
[0296] disAttr=|curColor[0]-neighColor[0]| Formula 2
[0297] In the above formula 2, curColor[0] represents the attribute reconstruction component of the to-be-decoded point under the Y component, and neighColor[0] represents the attribute reconstruction component of the candidate point under the Y component.
[0298] When the color space is the second color space, the attribute reconstruction components of the point to be decoded under each color component of the second color space can be used as the attribute reconstruction information selected by the point to be decoded, and the attribute reconstruction components of the candidate point under each color component of the second color space can be used as the attribute reconstruction information selected by the candidate point; that is, the decoded attribute distance disAttr can be determined according to the difference between the attribute reconstruction components of the point to be decoded and the candidate point under each color component; this method can be applied to the case where the attribute reconstruction components under each color component respectively have part of the information in the attribute reconstruction information of the second color space. For example, the second color space is RGB (another color space), and the decoded attribute distance can be calculated using the R component (red component), the G component (green component), and the B component (blue component). The calculation formula can be found in the following formula 3:
[0299] disAttr=|curColor[0]-neighColor[0]|+|curColor[1]-neighColor[1]|
[0300] +|curColor[2]-neighColor[2]|Formula 3
[0301] In the above formula 3, curColor[0] represents the attribute reconstruction component of the point to be decoded under the R component, and neighColor[0] represents the attribute reconstruction component of the candidate point under the R component; curColor[1] represents the attribute reconstruction component of the point to be decoded under the G component, and neighColor[1] represents the attribute reconstruction component of the candidate point under the G component; curColor[2] represents the attribute reconstruction component of the point to be decoded under the B component, and neighColor[2] represents the attribute reconstruction component of the candidate point under the B component.
[0302] (3) For the comprehensive distance dis:
[0303] The comprehensive distance dis between the point to be decoded and the corresponding candidate point is obtained by summing the weighted results of the geometric distance disGeom between the point to be decoded and the corresponding candidate point and the decoded attribute distance disAttr between the point to be decoded and the corresponding candidate point. The weighting factor can be expressed as λ, which can be calculated according to the attribute quantization parameter. The calculation formula of the weighting factor λ can be referred to the following formula 4:
[0304] λ=-crossAttrTypePredParam1·attrQuantParam+
[0305] crossAttrTypePredParam2 formula 4
[0306] In the above formula 4, crossAttrTypePredParam1 and crossAttrTypePredParam2 represent parameters of the weighting factor in the weight calculation process; attrQuantParam represents an attribute quantization parameter, which may be an attribute quantization parameter corresponding to a decoded attribute type, or an attribute quantization parameter corresponding to an attribute type to be decoded.
[0307] S303: Perform attribute prediction on the point to be decoded according to the distance between the point to be decoded and the prediction reference point to obtain attribute prediction information of the point to be decoded.
[0308] In one implementation of step S303, the distance between the point to be decoded and the predicted reference point may specifically refer to a geometric distance, and according to the geometric distance between the point to be decoded and the predicted reference point, k first predicted reference points whose geometric distances to the point to be decoded are less than a first distance threshold are obtained from the predicted reference points, and r second predicted reference points whose geometric distances to the point to be decoded are equal to the first distance threshold are obtained from the predicted reference points; k and r are both non-negative integers; and then, according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points, attribute prediction of the point to be decoded can be performed to obtain attribute prediction information of the point to be decoded.
[0309] In this implementation, based on the attribute reconstruction information of k first prediction reference points and the attribute reconstruction information of r second prediction reference points, attribute prediction is performed on the point to be decoded to obtain the attribute prediction information of the point to be decoded, which may specifically include:
[0310] (1) When k>0 and r>0 (i.e., when the prediction reference points include both a first prediction reference point whose geometric distance to the point to be decoded is less than the first distance threshold and a second prediction reference point whose geometric distance to the point to be decoded is equal to the first distance threshold), the attribute prediction of the point to be decoded can be performed based on the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the point to be decoded. Specifically:
[0311] First, the point to be decoded can be expressed as (x i ,y i ,z i ), the k first prediction reference points can be expressed as (x jn ,y jn ,z jn ) n=1,2,...k , the attribute reconstruction values of the k first prediction reference points can be expressed as The first weighting coefficients of the k first prediction reference points can be obtained. The first weighting coefficients of the k first prediction reference points can be expressed as the following formula 5:
[0312]
[0313] The r second prediction reference points can be expressed as (x jm ,y jm ,z jm ) m=1,2,...r , the attribute reconstruction value of the r second prediction reference points can be expressed as The second weighting coefficients of the r second prediction reference points can be obtained. The second weighting coefficients of the r second prediction reference points can be expressed as the following formula 6:
[0314]
[0315] Secondly, the weight optimization coefficients d of the r second prediction reference points can be obtained wm The weight optimization coefficient d of the r second prediction reference points wm The method of obtaining , which can include any of the following:
[0316] ① The total number of second prediction reference points r can be numerically calculated to obtain the weight optimization coefficient d wm , d wm =min(r,n) / r.
[0317] ② The total number of second prediction reference points r and the attribute quantization step size Qstep of the points to be decoded can be numerically calculated to obtain the weight optimization coefficient d wm , d wm =min(r,Qstep) / r.
[0318] ③ According to the attribute coding condition satisfied by the point to be decoded, the weight optimization coefficient corresponding to the satisfied attribute coding condition can be determined. The attribute coding condition can include an attribute loss condition or an attribute coding mode; when the attribute coding condition includes an attribute loss condition, the weight optimization coefficient corresponding to the satisfied attribute coding condition can be determined according to the attribute loss condition satisfied by the attribute quantization parameter of the point to be decoded; when the attribute coding condition includes an attribute coding mode, the weight optimization coefficient corresponding to the attribute coding mode of the point to be decoded can be determined according to the attribute coding mode of the point to be decoded. Taking the attribute loss condition as an example, if the attribute quantization parameter of the point to be decoded is equal to the first value (attrQuantParam=0), the attribute quantization parameter of the point to be decoded satisfies the attribute lossless condition, and the weight optimization coefficient corresponding to the attribute lossless condition is d wm =1 / r; if the attribute quantization parameter of the point to be decoded is greater than the first value (attrQuantParam>0), the attribute quantization parameter of the point to be decoded satisfies the attribute lossy condition, and the weight optimization coefficient corresponding to the attribute lossy condition is d wm =1.
[0319] Then, the second weighting coefficients of the r second prediction reference points can be optimized according to the weight optimization coefficients to obtain the second optimized weighting coefficients of the r second prediction reference points. jm It can be expressed as the following formula 7:
[0320]
[0321] In particular, when the attribute quantization parameter of the point to be decoded satisfies the attribute lossless condition (ie, when attrQuantParam=0), the weight optimization coefficient corresponding to the attribute lossless condition is d wm =1 / r, the second optimization weighting coefficient W of each of the r second prediction reference points jm It can be expressed as the following formula 8:
[0322]
[0323] It can be understood that when the attribute quantization parameter of the point to be decoded satisfies the attribute lossless condition (that is, when attrQuantParam=0), the second weighting coefficient of any second prediction reference point can be simplified to the derivative of the geometric distance between the corresponding second reference prediction point and the point to be decoded divided by the total number of second prediction reference points (r).
[0324] When the attribute quantization parameter of the point to be decoded satisfies the attribute lossy condition (i.e., when attrQuantParam=1), the weight optimization coefficient corresponding to the attribute lossy condition is d wm =1, the second optimization weighting coefficient W of each of the r second prediction reference points jm It can be expressed as the following formula 9:
[0325]
[0326] It can be understood that when the attribute quantization parameter of the point to be decoded satisfies the attribute lossy condition (ie when attrQuantParam=1), the second weighting coefficient of any second prediction reference point can be simplified to the derivative of the geometric distance between the corresponding second reference prediction point and the point to be decoded.
[0327] Then, the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points can be weighted and summed according to the first weight coefficients of the k first prediction reference points and the second optimized weight coefficients of the r second prediction reference points to obtain the attribute prediction information of the point to be decoded. The calculation formula of the attribute prediction information of the point to be decoded can be seen in the following formula 10:
[0328]
[0329] (2) When k=0 and r>0 (i.e., when all prediction reference points are second prediction reference points whose geometric distance to the point to be decoded is equal to the first distance threshold), the attribute prediction information of the point to be decoded can be obtained by performing attribute reconstruction on the r second prediction reference points. Specifically:
[0330] First, the r second prediction reference points can be expressed as (xjm ,y jm ,z jm ) m=1,2,...r , the attribute reconstruction value of the r second prediction reference points can be expressed as The second weighting coefficient W of each of the r second prediction reference points can be obtained: jm .
[0331] Secondly, the weight optimization coefficients d of the r second prediction reference points can be obtained wm The coefficient d can be optimized according to the weight wm , optimize the second weighting coefficients of the r second prediction reference points to obtain the second optimized weighting coefficients W of the r second prediction reference points. jm It should be noted that the weight optimization coefficient d wm The acquisition method is the same as that in the case where k>0 and r>0, and will not be repeated here; the second optimization weighting coefficient W jm The calculation formula of can refer to the above formula 7, which will not be repeated here.
[0332] Then, the attribute reconstruction information of the r second prediction reference points may be weighted and summed according to the second optimized weight coefficients of the r second prediction reference points to obtain the attribute prediction information of the point to be decoded. The calculation formula of the attribute prediction information of the point to be decoded can be seen in the following formula 11:
[0333]
[0334] (3) When k>0, r=0 (i.e., when all prediction reference points are first prediction reference points whose geometric distance to the point to be decoded is less than the first distance threshold), the attribute prediction information of the point to be decoded can be obtained by performing attribute reconstruction on the point to be decoded based on the attribute reconstruction information of the k first prediction reference points. Specifically:
[0335] First, the point to be decoded can be expressed as (x i ,y i ,z i ), the k first prediction reference points can be expressed as (x jn ,y jn ,z jn ) n=1,2,...k , the attribute reconstruction values of the k first prediction reference points can be expressed as The first weighting coefficient W of each of the k first prediction reference points can be obtained: jn It should be noted that the first weighting coefficient W jn The calculation formula of can refer to the above formula 5, which will not be repeated here.
[0336] Then, the attribute reconstruction information of the k first prediction reference points can be weighted and summed according to the first weight coefficients of the k first prediction reference points to obtain the attribute prediction information of the point to be decoded. The calculation formula of the attribute prediction information of the point to be decoded can be seen in the following formula 12:
[0337]
[0338] In this implementation, in order to speed up the attribute prediction efficiency, a part of the second prediction reference points can be selected from the r second prediction reference points to participate in the attribute prediction. Specifically, the third prediction reference point can be selected from the r second prediction reference points, and then, the attribute prediction of the point to be decoded can be performed based on the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the third prediction reference point to obtain the attribute prediction information of the point to be decoded. The method of performing attribute prediction based on the first prediction reference point and the third prediction reference point is the same as the method of performing attribute prediction based on the first prediction reference point and the second prediction reference point. For details, please refer to the method of performing attribute prediction based on the first prediction reference point and the second prediction reference point, which will not be repeated here.
[0339] Among them, the method of selecting the third prediction reference point from the r second prediction reference points may specifically include: obtaining the difference information between the attribute reconstruction information of any two second prediction reference points from the r second prediction reference points, and determining the two second prediction reference points corresponding to the largest difference information as the third prediction reference point. Alternatively, the second prediction reference point closest to the point to be decoded in the geometric space component from the r second prediction reference points may be determined as the third prediction reference point; for example, the second prediction reference point closest to the point to be decoded in the X direction from the r second prediction reference points may be determined as the third prediction reference point, the second prediction reference point closest to the point to be decoded in the Y direction may be determined as the third prediction reference point, or the second prediction reference point closest to the point to be decoded in the Z direction may be determined as the third prediction reference point.
[0340] In another implementation of step S303, the distance between the point to be decoded and the predicted reference point may specifically refer to the geometric distance. When the index value of the point to be decoded in the target point cloud group is greater than the upper limit of the length of the candidate point set (i.e., s>maxNumOfNeighbours), it may be determined whether it is necessary to perform attribute prediction on the point to be decoded according to the attribute default prediction information to obtain the attribute prediction information of the point to be decoded based on the geometric distance between the point to be decoded and the predicted reference point; performing attribute prediction on the point to be decoded according to the attribute default prediction information may specifically refer to using the attribute default prediction information as the attribute prediction information of the point to be decoded. Specifically:
[0341] First, when the index value of the point to be decoded in the target point cloud group is greater than the upper limit of the length of the alternative point set, the default prediction value determination distance can be determined according to the geometric distance between the point to be decoded and the target prediction reference point in the prediction reference points; the target prediction reference point can be the p prediction reference points in the prediction reference points that are the closest or farthest from the point to be decoded in terms of geometric distance, where p is a positive integer; the default prediction value determination distance can include any one of the following: the maximum geometric distance, the minimum geometric distance, or the average geometric distance among the geometric distances between the point to be decoded and the p prediction reference points. Then, if the default prediction value determination distance is greater than the second distance threshold (d2), the attribute default prediction information can be obtained, and based on the attribute default prediction information, the attribute prediction of the point to be decoded can be performed to obtain the attribute prediction information of the point to be decoded, that is, the attribute default prediction information is used as the attribute prediction information of the point to be decoded. Among them, the attribute default prediction information includes any one of the following: ① Fixed attribute default prediction information. For example, for the color attribute, the fixed attribute default prediction information can be [0, 0, 0], [128, 128, 128], [256, 256, 256], etc.; ② Attribute default prediction information determined according to the attribute bit depth of the point to be decoded. For example, the attribute default prediction information can be 1<<colorOutputDepth-1, 1<<colorOutputDepth, 1<<(colorOutputDepth-1), where colorOutputDepth represents the attribute bit depth.
[0342] Furthermore, it is also possible to determine the selected fixed attribute default prediction information. Specifically, the attribute determination information of the previous points (k1 previous points, where k1 is a positive integer) of the point to be decoded can be obtained; the previous points refer to the points in the point cloud whose decoding order is before the point to be decoded; the attribute determination information can include the average value of the attribute reconstruction information or the average value of the attribute prediction information of the previous points. Then, according to the relationship between the attribute determination information and the attribute determination threshold, the attribute default prediction information can be determined; for example, the attribute determination information can be represented as A1, and the attribute determination threshold can be represented as A_th. If A1 - A_th > 0, the fixed first attribute default prediction information (A_default1) can be selected, otherwise, the fixed second attribute default prediction information (A_default2) can be selected.
[0343] In addition, in addition to determining whether to use the attribute default prediction information as the attribute prediction information of the point to be decoded based on the geometric distance between the point to be decoded and the prediction reference point, the determination can also be canceled, and the attribute default prediction information can be directly used as the attribute prediction information of the point to be decoded; it should be noted that, in this case, the attribute default prediction information, and the specific circumstances of determining the selected fixed attribute default prediction information are the same as the above-mentioned determination of whether to use the attribute default prediction information as the attribute prediction information of the point to be decoded. For details, please refer to the above-mentioned description of determining whether to use the attribute default prediction information as the attribute prediction information of the point to be decoded, which will not be repeated here.
[0344] S304, reconstructing the attributes of the points to be decoded according to the attribute prediction information of the points to be decoded, and obtaining the attribute reconstruction information of the points to be decoded.
[0345] After obtaining the attribute prediction information of the point to be decoded, the attribute of the point to be decoded can be reconstructed according to the attribute prediction information of the point to be decoded to obtain the attribute reconstruction information of the point to be decoded. Specifically, the attribute residual information of the point to be decoded can be parsed from the point cloud code stream, and then, the attribute of the point to be decoded can be reconstructed according to the attribute residual information of the point to be decoded and the attribute prediction information of the point to be decoded to obtain the attribute reconstruction information of the point to be decoded, and the attribute reconstruction information of the point to be decoded is equal to the sum of the attribute residual information of the point to be decoded and the attribute prediction information of the point to be decoded. After that, the truncation range of the attribute reconstruction information of the point to be decoded can be obtained, and the attribute reconstruction information of the point to be decoded can be truncated according to the truncation range, that is, the final attribute reconstruction information of the point to be decoded is the attribute reconstruction information after truncation. The process of attribute reconstruction and truncation can be seen in the following formula 13:
[0346]
[0347] As shown in the above formula 11, Indicates the attribute reconstruction information of the point to be decoded. Represents the attribute residual information of the point to be decoded, Represents the attribute prediction information of the point to be decoded. MinRange and MaxRange represent the truncation range. Among them, the method of obtaining the truncation range of the attribute reconstruction information of the point to be decoded may include any of the following: First, the truncation range may be directly parsed from the point cloud code stream, and the truncation range may be calculated by the encoding device according to the attribute output bit depth, MinRange = -(1<<(outputBitDepth-1)), MaxRange = (1<<(outputBitDepth-1))-1, outputBitDepth represents the attribute output bit depth. The second method is to obtain the stage range corresponding to the attribute type of the point to be decoded; for example, when the attribute type of the point to be decoded is a color attribute, MinRange = 0, MaxRange = (1 << (outputBitDepth-1))-1, and outputBitDepth represents the attribute output bit depth; for another example, when the attribute type of the point to be decoded is a reflectivity attribute, MinRange = -(1 << (outputBitDepth-1)), MaxRange = (1 << (outputBitDepth-1))-1, and outputBitDepth represents the attribute output bit depth. The third method is to determine the truncation range according to the bit depth condition satisfied by the attribute output bit depth of the point to be decoded and the attribute type of the point to be decoded; for example, when outputDepth>D (D is the bit depth threshold), MinRange = -(1 << (outputBitDepth-1)), MaxRange = (1 << (outputBitDepth-1))-1.
[0348] After step S304, when the index value of the point to be decoded in the target point cloud group is greater than the upper limit of the length of the alternative point set (i.e., s>maxNumOfNeighbours), in order to improve the accuracy of attribute prediction of subsequent points to be decoded of the point to be decoded, the farthest alternative point with the farthest geometric distance from the point to be decoded can be determined in the alternative point set (specifically, the alternative point set in the range of [n, maxNumOfNeighbours-1]), and the reconstruction information of the farthest alternative point (which may include geometric reconstruction information and attribute reconstruction information) is exchanged with the reconstruction information of the nth alternative point in the alternative point set, and then, the attribute reconstruction information of the point to be decoded can be updated to the alternative point set; its essence is to replace the farthest alternative point, which is not conducive to attribute prediction of subsequent points to be decoded, with the point to be decoded, and the subsequent points to be decoded refer to points whose decoding order is after the point to be decoded.
[0349] In addition, the exchange situation of the farthest candidate point and the nth point can also be determined. Specifically, when the index value of the to-be-decoded point in the target point cloud group is greater than the upper limit of the length of the candidate point set (i.e., s>maxNumOfNeighbours), the exchange determination distance can be obtained; the exchange determination distance can include any of the following: the geometric distance between the to-be-decoded point and the farthest predicted reference point in the predicted reference points (the predicted reference point with the farthest distance from the set between the to-be-decoded points), the geometric distance between the to-be-decoded point and the nearest predicted reference point in the predicted reference points (the predicted reference point with the closest distance from the set between the to-be-decoded points), and the average of the geometric distances between the to-be-decoded point and the predicted reference points. Then, if the exchange determination distance is greater than or equal to the third distance threshold (d1), the candidate point set can be kept unchanged, that is, no exchange; if the exchange determination distance is less than the third distance threshold (d1), the farthest candidate point with the farthest geometric distance from the point to be decoded can be determined in the candidate point set (specifically, the candidate point set in the range of [n, maxNum OfNeighbours-1]), and the reconstruction information of the farthest candidate point is exchanged with the reconstruction information of the nth candidate point in the candidate point set. Then, the attribute reconstruction information of the point to be decoded can be updated to the candidate point set, and the updated candidate point set can be applied to the attribute prediction process of the subsequent points to be decoded.
[0350] Furthermore, if there are multiple farthest candidate points with the farthest geometric distance from the point to be decoded in the candidate point set, that is, there are multiple farthest candidate points with equal distances, the target farthest candidate point can be selected from the farthest candidate points, and then the reconstruction information of the target farthest candidate point can be exchanged with the reconstruction information of the nth candidate point in the candidate point set. The following introduces different ways of selecting the target farthest candidate point from the farthest candidate points when the point to be decoded has attribute information under one attribute type and has attribute information under multiple attribute types:
[0351] (1) When the point to be decoded has attribute information under an attribute type, the method of selecting the target farthest candidate point from the farthest candidate points may include any of the following: First, according to the ranking of each farthest candidate point in the candidate point set, the target farthest candidate point is selected from each farthest candidate point; the ranking is the arrangement order of each farthest candidate point in the candidate point set, and the farthest candidate point with the first arrangement order can be selected as the target farthest candidate point, or the farthest candidate point with the last arrangement order can be selected as the target farthest candidate point. Second, according to the search order of each farthest candidate point in the candidate point set, the target farthest candidate point is selected from each farthest candidate point; the search order is the traversal order of each farthest candidate point in the candidate point set, and the farthest candidate point with the first search order can be selected as the target farthest candidate point, or the farthest candidate point with the last search order can be selected as the target farthest candidate point. The third method is to select the target farthest candidate point from among the farthest candidate points according to the attribute distance between each farthest candidate point and the point to be decoded, and the farthest candidate point with the smallest or largest attribute distance to the point to be decoded can be selected as the target farthest candidate point. Taking the reflectivity attribute as an example, the attribute distance between the farthest candidate point and the point to be decoded can be the difference between the reflectivity attribute prediction information of the point to be decoded and the reflectivity attribute reconstruction information of the farthest candidate point. Taking color attribute as an example, assume that the color attribute prediction information of the point to be decoded is predColor, and the color attribute reconstruction information of the farthest candidate point is farthestColor; the attribute distance between the farthest candidate point and the point to be decoded can be the sum of the differences between the attribute reconstruction component of the farthest candidate point and the attribute prediction component of the point to be decoded under each color component, and the attribute distance between the farthest candidate point and the point to be decoded is distA = |predColor[0]–farestColor[0]|+|predColor[1]–farestColor[1]|+|predColor[2]–farestColor[2]|; or, the attribute distance between the farthest candidate point and the point to be decoded can be the difference between the attribute reconstruction component of the farthest candidate point and the attribute prediction component of the point to be decoded under any color component, and the attribute distance between the farthest candidate point and the point to be decoded is distA = |predColor[0]–farestColor[0]|. The fourth method is to select a target farthest candidate point from each farthest candidate point according to the geometric distance between each farthest candidate point and the point to be decoded in the geometric space component; for example, the farthest candidate point with the farthest geometric distance from the point to be decoded in the X direction is selected as the target farthest candidate point, the farthest candidate point with the farthest geometric distance from the point to be decoded in the Y direction is selected as the target farthest candidate point, or the farthest candidate point with the farthest geometric distance from the point to be decoded in the Z direction is selected as the target farthest candidate point.Fifthly, a target farthest prediction point may be selected from among the farthest prediction points according to the attribute distance between each farthest prediction point and a preceding point of the point to be decoded, wherein the preceding point refers to a point that precedes the point to be decoded in the decoding order.
[0352] For the fifth method of selecting the target farthest candidate point among the farthest candidate points (i.e., selecting the target farthest prediction point among each farthest prediction point based on the attribute distance between each farthest prediction point and the predecessor point of the point to be decoded): when the attribute coding mode of the point to be decoded is the attribute prediction mode, the attribute distance between the predecessor point and the farthest candidate point can be determined based on the difference between the attribute reconstruction information of the predecessor point and the attribute reconstruction information of the farthest candidate point (similar to the method of calculating the attribute distance between the point to be decoded and the farthest candidate point in the third method of selecting the target farthest candidate point among the farthest candidate points mentioned above, which will not be repeated here); then, among each farthest candidate point, the farthest candidate point with the largest or smallest attribute distance to the predecessor point can be selected as the target farthest candidate point. Alternatively, when the attribute coding mode of the point to be decoded is the attribute prediction transform coding mode, the point to be decoded belongs to the target point cloud group in the point cloud; the target preceding point can be selected from the preceding points of the point to be decoded; based on the difference between the attribute reconstruction information of the target preceding point and the attribute reconstruction information of the farthest alternative point, the attribute distance between the target preceding point and the farthest alternative point is determined (similar to the method of calculating the attribute distance between the point to be decoded and the farthest alternative point in the third method of selecting the target farthest alternative point from the farthest alternative points mentioned above, which will not be repeated here); then, among the farthest alternative points, the farthest alternative point with the largest or smallest attribute distance to the target preceding point can be selected as the target farthest alternative point.
[0353] Among them, the method of selecting the target pre-sequence point in the pre-sequence points of the points to be decoded may include any of the following: ① Select the first pre-sequence point or the last pre-sequence point in the pre-sequence point cloud group of the target point cloud group as the target pre-sequence point, the pre-sequence point cloud group is a point cloud group whose decoding order is before the target point cloud group; and all the points to be decoded in the target point cloud group select the first pre-sequence point or the first pre-sequence point in the pre-sequence point cloud group as the target pre-sequence point. That is to say, there are M points [p1, p2, …, pM] in the target point cloud group, and all the points to be decoded in the target point cloud group select the first pre-sequence point (i.e., the first pre-sequence point) or the last pre-sequence point (i.e., the last pre-sequence point) in the pre-sequence point cloud group as the target pre-sequence point. ② The target pre-sequence point can be selected from the pre-sequence points of the points to be decoded according to the position of the points to be decoded in the target point cloud group. Specifically: if the point to be decoded is the first point to be decoded in the target point cloud group, select the first pre-order point or the last pre-order point in the previous point cloud group of the target point cloud group as the target pre-order point; that is, there are M points [p1, p2, ..., pM] in the target point cloud group, for the point p1 to be decoded, the first pre-order point (i.e., the first pre-order point) or the last pre-order point (i.e., the last pre-order point) in the previous point cloud group can be selected as the target pre-order point. If the point to be decoded is not the first point to be decoded in the target point cloud group, select the first pre-order point in the target point cloud group, or the pre-order point in the target point cloud group whose decoding order is arranged one position before the point to be decoded, as the target pre-order point; that is, there are M points [p1, p2, ..., pM] in the target point cloud group, for the point p2, ..., pM to be decoded, the first pre-order point in the target point cloud group, or the pre-order point in the target point cloud group whose decoding order is arranged one position before the point to be decoded, can be selected as the target pre-order point.
[0354] (2) When the point to be decoded has attribute information under multiple attribute types, the method of selecting the target farthest candidate point from the farthest candidate points may include any of the following: ① Select the target farthest candidate point under each attribute type respectively from the farthest candidate points; the selection method under any attribute type can refer to the above-mentioned method of selecting the target farthest candidate point from the farthest candidate points when the point to be decoded has attribute information under one attribute type, which will not be repeated here. ② Select the target farthest candidate point under any of the following attribute types from the farthest candidate points: first prediction attribute type, post prediction attribute type, and target attribute type; the target attribute type is any of the multiple attribute types; the selection method under any attribute type can refer to the above-mentioned method of selecting the target farthest candidate point from the farthest candidate points when the point to be decoded has attribute information under one attribute type, which will not be repeated here. ③ Select the farthest predicted point of the target based on the attribute weighted distance of multiple attribute types or the geometric attribute comprehensive distance among the farthest candidate points; the attribute weighted distance may refer to the weighted result of the attribute distance between the farthest candidate point and the point to be decoded under each attribute type, or may refer to the weighted result of the attribute distance between the farthest candidate point and the preceding point of the point to be decoded under each attribute type; the geometric attribute comprehensive distance may refer to the geometric attribute comprehensive distance between the farthest candidate point and the point to be decoded, or may refer to the geometric attribute comprehensive distance between the farthest candidate point and the preceding point of the point to be decoded. The calculation of the geometric attribute comprehensive distance may refer to the relevant contents of the above formulas 1 to 4.
[0355] It should be added that the point to be decoded belongs to the target point cloud group in the point cloud, and the target point cloud group is grouped based on the target right shift number L. The specific grouping rule is that the Hilbert code (or Morton code) of each point in the point cloud is right-shifted by the target right shift number L, and the same points are divided into a point cloud group (or macroblock). If the number of points in the point cloud group is greater than the maximum transformation order MaxTransNum, the point cloud group needs to be further subdivided, and the grouping needs to be dynamically adjusted based on the target right shift number L. The dynamic adjustment can be based on the subdivision of the preceding point cloud group of the target point cloud group. The preceding point cloud group refers to the point cloud group whose decoding order is before the target point cloud group. Specifically, the target right shift number can be updated according to the subdivision of the preceding point cloud group of the target point cloud group; the updated target right shift number can be used as the grouping basis for the subsequent point cloud group of the target point cloud group, and the subsequent point cloud group refers to the point cloud group whose decoding order is after the target point cloud group.
[0356] More specifically, according to the subdivision of the previous point cloud grouping for the target point cloud grouping, the target right shift number can be updated, which can include any of the following: First, if the previous point cloud grouping is subdivided, the target right shift number can be decreased. For example, L = L - 1; or if the previous point cloud grouping is subdivided, the target right shift number can be decreased when the target right shift number meets the update condition. For example, when L < th_L (shift determination threshold), L = L - 1. Second, if the previous point cloud grouping is not subdivided, the target right shift number can remain unchanged, or if the previous point cloud grouping is not subdivided, the target right shift number can be increased. For example, L = L + 1. Third, the update direction of the target right shift number can be determined according to the number of previous point cloud groupings that are subdivided in the previous point cloud grouping.
[0357] In the embodiment of the present application, in the point cloud attribute decoding stage, according to the distance between the point to be decoded and the predicted reference point, the attribute of the point to be decoded is predicted, considering the attribute approximation between the predicted reference point used for predicting the attribute of the point to be decoded and the point to be decoded, which can improve the attribute prediction effect of the point to be decoded and improve the decoding performance of the point cloud attribute.
[0358] The embodiment of the present application provides a point cloud processing method, which mainly introduces the point cloud attribute encoding process. The point cloud processing method can be executed by a computer device, and the computer device can be the encoding device 201 in the above point cloud processing system. As Figure 4 shown, the point cloud processing method can include but is not limited to the following steps S401 - step S404:
[0359] S401, determine the point to be encoded in the point cloud.
[0360] As described above, a point cloud refers to a discrete point set. A point cloud can include multiple points, and each point in the point cloud is sequentially encoded according to the attribute encoding order. The point to be encoded is the point in the point cloud that has not been encoded for attributes.
[0361] S402, determine the predicted reference point of the point to be encoded in the alternative point set of the point to be encoded.
[0362] The alternative point set of the point to be encoded is the set of encoded points in the point cloud used to determine the predicted reference point of the point to be encoded, and the predicted reference point of the point to be encoded can be determined in the alternative point set of the point to be encoded. The alternative point set can include the reconstruction information of each alternative point. The reconstruction information can include geometric reconstruction information and attribute reconstruction information. The geometric reconstruction information refers to the geometric information reconstructed by geometric decoding, and the attribute reconstruction information refers to the attribute information reconstructed by attribute decoding.
[0363] When determining the predicted reference point of the point to be coded from the candidate point set of the point to be coded, it is necessary to consider whether the number of attribute types of the point to be coded is single or multiple; if the number of attribute types of the point to be coded is single, it can indicate that the point to be coded has attribute information under one attribute type, for example, the point to be coded has color information or reflectivity information; if the number of attribute types of the point to be coded is multiple, it can indicate that the point to be coded has attribute information under multiple attribute types, for example, the point to be coded has color information and reflectivity information. The following describes how to determine the predicted reference point of the point to be coded when the point to be coded has attribute information under one attribute type and when the point to be coded has attribute information under multiple attribute types:
[0364] 1. The point to be coded has attribute information under an attribute type:
[0365] When the point to be encoded has attribute information under an attribute type, the method for determining the prediction reference point of the point to be encoded may include any one of the following: prediction reference point search based on point cloud sequence and prediction reference point search based on point cloud sequence and spatial offset. When the point to be encoded has attribute information under an attribute type, the method for determining the prediction reference point of the point to be encoded is the same as the method for determining the prediction reference point of the point to be decoded when the point to be decoded has attribute information under an attribute type. For details, please refer to the above Figure 3 In the illustrated embodiment, when the to-be-decoded point has attribute information under one attribute type, the manner of determining the prediction reference point of the to-be-decoded point is not described in detail herein.
[0366] 2. The point to be coded has attribute information under an attribute type:
[0367] When the point to be coded has attribute information under multiple attribute types (i.e., a multi-attribute point to be coded), the method of determining the prediction reference point of the point to be coded may include:
[0368] Let the candidate point set be Sp, and the upper limit of the length of the candidate point set Sp is maxNumOfNeighbours, that is, the candidate point set Sp contains at most maxNumOfNeighbours candidate points. Sp contains the reconstruction information of the candidate points (including geometric reconstruction information and attribute reconstruction information).
[0369] If cross-type attribute prediction is not allowed, the prediction reference points of the to-be-coded point under each attribute type can be determined respectively according to the above method of determining the prediction reference point of the to-be-coded point when the to-be-coded point has attribute information under one attribute type.
[0370] If cross-type attribute prediction is allowed, the prediction reference point of the point to be encoded can be determined in the candidate point set Sp according to the above method of determining the prediction reference point of the point to be encoded under any attribute type when the point to be encoded has attribute information under one attribute type, and attribute prediction is performed based on the prediction reference point under the corresponding attribute type, and then attribute reconstruction is performed to obtain the attribute reconstruction information of the point to be encoded under the corresponding attribute type, and the corresponding attribute type can be called the encoded attribute type. For the attribute type to be encoded, the geometric distance disGeom between the point to be encoded and the candidate points in the candidate point set Sp can be obtained; the encoded attribute distance disAttr between the point to be encoded and the candidate points in the candidate point set Sp can be obtained; the comprehensive distance dis between the point to be encoded and the corresponding candidate point can be determined based on the geometric distance between the point to be encoded and the corresponding candidate point, and the encoded attribute distance between the point to be encoded and the corresponding candidate point; then, the prediction reference point of the point to be encoded can be determined in the candidate point set based on the comprehensive distance between the point to be encoded and the candidate points in the candidate point set. Among them:
[0371] (1) For the geometric distance disGeom:
[0372] The geometric distance disGeom between the point to be encoded and the candidate point can be calculated using Manhattan distance.
[0373] (2) For the encoded attribute distance disAttr:
[0374] The coded attribute distance disAttr between the point to be coded and the candidate point is determined according to the difference between the attribute reconstruction information of the point to be coded under the coded attribute type and the attribute reconstruction information of the candidate point under the coded attribute type.
[0375] When the encoded attribute type is the reflectivity attribute, the calculation formula of the encoded attribute distance disAttr can refer to the above formula 1, in which curRefl represents the reflectivity attribute reconstruction information of the point to be encoded, and neighRefl represents the reflectivity attribute reconstruction information of the candidate point.
[0376] When the encoded attribute type is a color attribute, the attribute reconstruction information selected by the point to be encoded and the alternative point can be determined according to the color space; then, the encoded attribute distance can be determined according to the difference between the attribute reconstruction information selected by the point to be encoded and the attribute reconstruction information selected by the alternative point.
[0377] When the color space is the first color space, the attribute reconstruction component of the to-be-encoded point under the target color component of the first color space can be used as the attribute reconstruction information selected by the to-be-encoded point, and the attribute reconstruction component of the candidate point under the target color component of the first color space can be used as the attribute reconstruction information selected by the candidate point; that is, the encoded attribute distance disAttr can be determined according to the difference between the attribute reconstruction component of the to-be-encoded point under the target color component of the first color space and the attribute reconstruction component of the candidate point under the target color component of the first color space; this method can be applied to the case where the attribute reconstruction component under the target color component has most of the information in the attribute reconstruction information of the first color space. For example, when the first color space is YUV (a color space), the Y component (brightness component) can be used to calculate the encoded attribute distance. The calculation formula can be found in the above formula 2. In the above formula 2, curColor[0] represents the attribute reconstruction component of the to-be-encoded point under the Y component, and neighColor[0] represents the attribute reconstruction component of the candidate point under the Y component.
[0378] When the color space is a second color space, the attribute reconstruction components of the point to be encoded under each color component of the second color space can be used as the attribute reconstruction information selected for the point to be encoded, and the attribute reconstruction components of the alternative point under each color component of the second color space can be used as the attribute reconstruction information selected for the alternative point; that is, the encoded attribute distance disAttr can be determined according to the difference between the attribute reconstruction components of the point to be encoded and the alternative point under each color component; this method can be applicable to the situation where the attribute reconstruction components under each color component respectively have part of the information in the attribute reconstruction information of the second color space. For example, the second color space is RGB (another color space), and the R component (red component), G component (green component), and B component (blue component) can be used to calculate the encoded attribute distance. The calculation formula can be found in the above formula 3. In the above formula 3, curColor[0] represents the attribute reconstruction component of the point to be encoded under the R component, and neighColor[0] represents the attribute reconstruction component of the candidate point under the R component; curColor[1] represents the attribute reconstruction component of the point to be encoded under the G component, and neighColor[1] represents the attribute reconstruction component of the candidate point under the G component; curColor[2] represents the attribute reconstruction component of the point to be encoded under the B component, and neighColor[2] represents the attribute reconstruction component of the candidate point under the B component.
[0379] (3) For the comprehensive distance dis:
[0380] The comprehensive distance dis between the point to be encoded and the corresponding candidate point is obtained by summing the weighted results of the geometric distance disGeom between the point to be encoded and the corresponding candidate point and the encoded attribute distance disAttr between the point to be encoded and the corresponding candidate point. The weighting factor can be expressed as λ, and the weighting factor can be calculated according to the attribute quantization parameter. The calculation formula of the weighting factor λ can be seen in the above formula 4. In the above formula 4, crossAttrTypePredParam1 and crossAttrTypePredParam2 represent parameters of the weighting factor in the weight calculation process; attrQuantParam represents the attribute quantization parameter, and the attribute quantization parameter can be the attribute quantization parameter corresponding to the encoded attribute type, or the attribute quantization parameter corresponding to the attribute type to be encoded.
[0381] S403: performing attribute prediction on the point to be encoded according to the distance between the point to be encoded and the prediction reference point to obtain attribute prediction information of the point to be encoded.
[0382] In one implementation of step S403, the distance between the point to be encoded and the predicted reference point may specifically refer to a geometric distance, and according to the geometric distance between the point to be encoded and the predicted reference point, k first predicted reference points whose geometric distances to the point to be encoded are less than a first distance threshold are obtained from the predicted reference points, and r second predicted reference points whose geometric distances to the point to be encoded are equal to the first distance threshold are obtained from the predicted reference points; k and r are both non-negative integers; then, according to the attribute reconstruction information of the k first predicted reference points and the attribute reconstruction information of the r second predicted reference points, attribute prediction can be performed on the point to be encoded to obtain attribute prediction information of the point to be encoded.
[0383] In this implementation, based on the attribute reconstruction information of k first prediction reference points and the attribute reconstruction information of r second prediction reference points, attribute prediction is performed on the to-be-coded point to obtain the attribute prediction information of the to-be-coded point, which may specifically include:
[0384] (1) When k>0 and r>0 (i.e., when there is a first prediction reference point whose geometric distance to the point to be encoded is less than the first distance threshold and a second prediction reference point whose geometric distance to the point to be encoded is equal to the first distance threshold in the prediction reference points), the attribute of the point to be encoded can be predicted based on the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the point to be encoded. Specifically: obtain the first weighting coefficient of each of the k first prediction reference points and obtain the second weighting coefficient of each of the r second prediction reference points; obtain the weight optimization coefficient of the r second prediction reference points; optimize the second weighting coefficient of each of the r second prediction reference points according to the weight optimization coefficient to obtain the second optimized weighting coefficient of each of the r second prediction reference points; perform weighted summation on the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points according to the first weighting coefficient of each of the k first prediction reference points and the second optimized weighting coefficient of each of the r second prediction reference points to obtain the attribute prediction information of the point to be encoded.
[0385] The weight optimization coefficient d of the r second prediction reference points wm The acquisition method may include any of the following: ① The total number r of the second prediction reference points may be numerically calculated to obtain the weight optimization coefficient d wm , d wm =min(r,n) / r. ② The total number of second prediction reference points r and the attribute quantization step length Qstep of the points to be encoded can be numerically calculated to obtain the weight optimization coefficient d wm , d wm =min(r,Qstep) / r. ③ According to the attribute loss condition satisfied by the point to be encoded, the weight optimization coefficient corresponding to the satisfied attribute coding condition can be determined. The attribute coding condition can include an attribute loss condition or an attribute coding mode; when the attribute coding condition includes an attribute loss condition, the weight optimization coefficient corresponding to the satisfied attribute coding condition can be determined according to the attribute loss condition satisfied by the attribute quantization parameter of the point to be encoded; when the attribute coding condition includes an attribute coding mode, the weight optimization coefficient corresponding to the attribute coding mode of the point to be encoded can be determined according to the attribute coding mode of the point to be encoded. Take the attribute loss condition as an example. Taking the attribute loss condition as an example, if the attribute quantization parameter of the point to be encoded is equal to the first value (attrQuantParam=0), the attribute quantization parameter of the point to be encoded satisfies the attribute lossless condition, and the weight optimization coefficient corresponding to the attribute lossless condition is d wm =1 / r; if the attribute quantization parameter of the point to be encoded is greater than the first value (attrQuantParam>0), the attribute quantization parameter of the point to be encoded satisfies the attribute lossy condition, and the weight optimization coefficient corresponding to the attribute lossy condition is d wm =1.
[0386] In particular, when the attribute quantization parameter of the point to be encoded satisfies the attribute lossless condition (i.e., when attrQuantParam=0), the second weighting coefficient of any second prediction reference point can be simplified to the derivative of the geometric distance between the corresponding second reference prediction point and the point to be encoded divided by the total number of second prediction reference points (r), as specifically referred to in the above formula 8. When the attribute quantization parameter of the point to be encoded satisfies the attribute lossy condition (i.e., when attrQuantParam=1), the second weighting coefficient of any second prediction reference point can be simplified to the derivative of the geometric distance between the corresponding second reference prediction point and the point to be encoded, as specifically referred to in the above formula 9.
[0387] (2) When k=0 and r>0 (i.e., when all prediction reference points are second prediction reference points whose geometric distance to the point to be encoded is equal to the first distance threshold), the attribute prediction of the point to be encoded can be performed based on the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the point to be encoded. Specifically, the second weighting coefficient of each of the r second prediction reference points can be obtained; the weight optimization coefficient of the r second prediction reference points can be obtained; according to the weight optimization coefficient, the second weighting coefficient of each of the r second prediction reference points is optimized to obtain the second optimized weighting coefficient of each of the r second prediction reference points; then, the attribute reconstruction information of the r second prediction reference points can be weighted summed according to the second optimized weighting coefficient of each of the r second prediction reference points to obtain the attribute prediction information of the point to be encoded. It should be noted that in this case, the weight optimization coefficient d wm The acquisition method is the same as that in the case where k>0 and r>0, and will not be repeated here.
[0388] (3) When k>0, r=0 (i.e., when all prediction reference points are first prediction reference points whose geometric distance to the point to be encoded is less than the first distance threshold), the attribute prediction of the point to be encoded can be performed based on the attribute reconstruction information of the k first prediction reference points to obtain the attribute prediction information of the point to be encoded. Specifically, the attribute reconstruction information of the k first prediction reference points can be weighted and summed based on the first weighting coefficients of the k first prediction reference points to obtain the attribute prediction information of the point to be encoded.
[0389] In this implementation, in order to speed up the attribute prediction efficiency, a part of the second prediction reference points can be selected from the r second prediction reference points to participate in the attribute prediction. Specifically, the third prediction reference point can be selected from the r second prediction reference points, and then, the attribute prediction of the to-be-encoded point can be performed based on the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the third prediction reference point to obtain the attribute prediction information of the to-be-encoded point. The method of performing attribute prediction based on the first prediction reference point and the third prediction reference point is the same as the method of performing attribute prediction based on the first prediction reference point and the second prediction reference point. For details, please refer to the method of performing attribute prediction based on the first prediction reference point and the second prediction reference point, which will not be repeated here.
[0390] Among them, the method of selecting the third prediction reference point from the r second prediction reference points may specifically include: obtaining the difference information between the attribute reconstruction information of any two second prediction reference points from the r second prediction reference points, and determining the two second prediction reference points corresponding to the largest difference information as the third prediction reference point. Alternatively, the second prediction reference point closest to the point to be encoded in the geometric space component among the r second prediction reference points may be determined as the third prediction reference point; for example, the second prediction reference point closest to the point to be encoded in the X direction among the r second prediction reference points may be determined as the third prediction reference point, the second prediction reference point closest to the point to be encoded in the Y direction may be determined as the third prediction reference point, or the second prediction reference point closest to the point to be encoded in the Z direction may be determined as the third prediction reference point.
[0391] In another implementation of step S403, the distance between the point to be coded and the predicted reference point may specifically refer to the geometric distance. When the index value of the point to be coded in the target point cloud group is greater than the upper limit of the length of the candidate point set (i.e., s>maxNumOfNeighbours), it may be determined whether it is necessary to perform attribute prediction on the point to be coded based on the attribute default prediction information to obtain the attribute prediction information of the point to be coded according to the geometric distance between the point to be coded and the predicted reference point; performing attribute prediction on the point to be coded based on the attribute default prediction information may specifically refer to using the attribute default prediction information as the attribute prediction information of the point to be coded. Specifically:
[0392] First, when the index value of the point to be encoded in the target point cloud group is greater than the upper limit of the length of the alternative point set, the default prediction value determination distance can be determined according to the geometric distance between the point to be encoded and the target prediction reference point in the prediction reference points; the target prediction reference point can be the p prediction reference points in the prediction reference points that are the closest or farthest from the point to be encoded in terms of geometric distance, where p is a positive integer; the default prediction value determination distance can include any one of the following: the maximum geometric distance, the minimum geometric distance, or the average geometric distance among the geometric distances between the point to be encoded and the p prediction reference points. Then, if the default prediction value determination distance is greater than the second distance threshold (d2), the attribute default prediction information can be obtained, and according to the attribute default prediction information, the attribute prediction of the point to be encoded can be performed to obtain the attribute prediction information of the point to be encoded, that is, the attribute default prediction information is used as the attribute prediction information of the point to be encoded. Among them, the attribute default prediction information includes any one of the following: ① fixed attribute default prediction information, for example, for the color attribute, the fixed attribute default prediction information can be [0, 0, 0], [128, 128, 128], and [256, 256, 256], etc.; ② attribute default prediction information determined according to the attribute bit depth of the point to be encoded, for example, the attribute default prediction information can be 1<<colorOutputDepth-1, 1<<colorOutputDepth, 1<<(colorOutputDepth-1), where colorOutputDepth represents the attribute bit depth.
[0393] Furthermore, the selected fixed attribute default prediction information can also be determined. Specifically, the attribute determination information of the previous points (k1 previous points, where k1 is a positive integer) of the point to be encoded can be obtained; the previous points refer to the points in the point cloud whose encoding order is before the point to be encoded; the attribute determination information can include the average of the attribute reconstruction information or the average of the attribute prediction information of the previous points. Then, according to the relationship between the attribute determination information and the attribute determination threshold, the attribute default prediction information can be determined; for example, the attribute determination information can be represented as A1, and the attribute determination threshold can be represented as A_th. If A1 - A_th > 0, the fixed first attribute default prediction information (A_default1) can be selected, otherwise, the fixed second attribute default prediction information (A_default2) can be selected.
[0394] In addition, in addition to determining whether to use the attribute default prediction information as the attribute prediction information of the point to be encoded based on the geometric distance between the point to be encoded and the prediction reference point, the determination can also be canceled, and the attribute default prediction information can be directly used as the attribute prediction information of the point to be encoded; it should be noted that, in this case, the attribute default prediction information, and the specific circumstances of determining the selected fixed attribute default prediction information are the same as the above-mentioned determination of whether to use the attribute default prediction information as the attribute prediction information of the point to be encoded. For details, please refer to the above-mentioned determination of whether to use the attribute default prediction information as the attribute prediction information of the point to be encoded. No further details will be given here.
[0395] S404: Perform attribute encoding on the point to be encoded according to the attribute prediction information of the point to be encoded.
[0396] After obtaining the attribute prediction information of the point to be encoded, the attribute encoding of the point to be encoded can be performed according to the attribute prediction information of the point to be encoded; attribute encoding can specifically refer to determining the attribute residual information of the point to be encoded according to the real attribute information of the point to be encoded and the attribute prediction information of the point to be encoded, and then encoding the attribute residual information of the point to be encoded into the point cloud code stream; the attribute residual information of the point to be encoded is equal to the difference between the real attribute information of the point to be encoded and the attribute prediction information of the point to be encoded.
[0397] In addition, the encoding device can also obtain the truncated range of the point to be encoded, and encode the truncated range of the point to be encoded into the point cloud code stream. The truncated range of the point to be encoded can be used to truncate the attribute reconstruction information of the point to be encoded during the attribute decoding process of the point to be encoded. The method of obtaining the truncated range of the point to be encoded may include any of the following: First, the truncated range can be calculated according to the attribute output bit depth, MinRange = -(1<<(outputBitDepth-1)), MaxRange = (1<<(outputBitDepth-1))-1, outputBitDepth represents the attribute output bit depth. The second method is to obtain the stage range corresponding to the attribute type of the point to be encoded; for example, when the attribute type of the point to be encoded is a color attribute, MinRange = 0, MaxRange = (1 << (outputBitDepth-1))-1, and outputBitDepth represents the attribute output bit depth; for another example, when the attribute type of the point to be encoded is a reflectivity attribute, MinRange = -(1 << (outputBitDepth-1)), MaxRange = (1 << (outputBitDepth-1))-1, and outputBitDepth represents the attribute output bit depth. The third method is to determine the truncation range according to the bit depth condition satisfied by the attribute output bit depth of the point to be encoded and the attribute type of the point to be encoded; for example, when outputDep th> D (D is the bit depth threshold), MinRange = -(1 << (outputBitDepth-1)), MaxRang ge = (1 << (outputBitDepth-1))-1.
[0398] After step S404, when the index value of the point to be encoded in the target point cloud group is greater than the upper limit of the length of the alternative point set (i.e., s>maxNumOfNeighbours), in order to improve the accuracy of attribute prediction of subsequent points to be encoded of the point to be encoded, the farthest alternative point with the farthest geometric distance from the point to be encoded can be determined in the alternative point set (specifically, the alternative point set in the range of [n, maxNumOfNeighbours-1]), and the reconstruction information of the farthest alternative point (which may include geometric reconstruction information and attribute reconstruction information) is exchanged with the reconstruction information of the nth alternative point in the alternative point set, and then, the attribute reconstruction information of the point to be encoded can be updated to the alternative point set; its essence is to replace the farthest alternative point, which is not conducive to attribute prediction of subsequent points to be encoded, with the point to be encoded, and the subsequent points to be encoded refer to points whose encoding order is after the point to be encoded.
[0399] In addition, the exchange situation between the farthest candidate point and the nth point can also be determined. Specifically, when the index value of the to-be-coded point in the target point cloud group is greater than the upper limit of the length of the candidate point set (i.e., s>maxNumOfNeighbours), the exchange determination distance can be obtained; the exchange determination distance can include any of the following: the geometric distance between the to-be-coded point and the farthest predicted reference point among the predicted reference points (the predicted reference point with the farthest distance from the set between the to-be-coded points), the geometric distance between the to-be-coded point and the nearest predicted reference point among the predicted reference points (the predicted reference point with the closest distance from the set between the to-be-coded points), and the average of the geometric distances between the to-be-coded point and the predicted reference points. Then, if the exchange determination distance is greater than or equal to the third distance threshold (d1), the candidate point set can be kept unchanged, that is, no exchange; if the exchange determination distance is less than the third distance threshold (d1), the farthest candidate point with the farthest geometric distance from the point to be encoded can be determined in the candidate point set (specifically, the candidate point set in the range of [n, maxNum OfNeighbours-1]), and the reconstruction information of the farthest candidate point is exchanged with the reconstruction information of the nth candidate point in the candidate point set. Then, the attribute reconstruction information of the point to be encoded can be updated to the candidate point set, and the updated candidate point set can be applied to the attribute prediction process of the subsequent points to be encoded.
[0400] Furthermore, if there are multiple farthest candidate points with the farthest geometric distance from the point to be encoded in the candidate point set, that is, there are multiple farthest candidate points with equal distance, the target farthest candidate point can be selected from the farthest candidate points, and then the reconstruction information of the target farthest candidate point can be exchanged with the reconstruction information of the nth candidate point in the candidate point set. The following introduces different ways of selecting the target farthest candidate point from the farthest candidate points when the point to be encoded has attribute information under one attribute type and has attribute information under multiple attribute types:
[0401] (1) When the point to be encoded has attribute information under an attribute type, the method of selecting the target farthest candidate point from the farthest candidate points may include any of the following: First, according to the order of each farthest candidate point in the candidate point set, the target farthest candidate point is selected from each farthest candidate point; the order is the arrangement order of each farthest candidate point in the candidate point set, and the farthest candidate point with the first arrangement order can be selected as the target farthest candidate point, or the farthest candidate point with the last arrangement order can be selected as the target farthest candidate point. Second, according to the search order of each farthest candidate point in the candidate point set, the target farthest candidate point is selected from each farthest candidate point; the search order is the traversal order of each farthest candidate point in the candidate point set, and the farthest candidate point with the first search order can be selected as the target farthest candidate point, or the farthest candidate point with the last search order can be selected as the target farthest candidate point. The third method is to select the target farthest candidate point from among the farthest candidate points according to the attribute distance between each farthest candidate point and the point to be encoded, and the farthest candidate point with the smallest or largest attribute distance to the point to be encoded can be selected as the target farthest candidate point. Taking the reflectivity attribute as an example, the attribute distance between the farthest candidate point and the point to be encoded can be the difference between the reflectivity attribute prediction information of the point to be encoded and the reflectivity attribute reconstruction information of the farthest candidate point. Taking color attribute as an example, assume that the color attribute prediction information of the point to be encoded is predColor, and the color attribute reconstruction information of the farthest candidate point is farestColor; the attribute distance between the farthest candidate point and the point to be encoded can be the sum of the differences between the attribute reconstruction component of the farthest candidate point and the attribute prediction component of the point to be encoded under each color component, and the attribute distance between the farthest candidate point and the point to be encoded distA = |predColor[0]–farestColor[0]|+|predColor[1]–farestColor[1]|+|predColor[2]–farestColor[2]|; or, the attribute distance between the farthest candidate point and the point to be encoded can be the difference between the attribute reconstruction component of the farthest candidate point and the attribute prediction component of the point to be encoded under any color component, and the attribute distance between the farthest candidate point and the point to be encoded distA = |predColor[0]–farestColor[0]|. The fourth method is to select the target farthest candidate point from each farthest candidate point according to the geometric distance between each farthest candidate point and the point to be encoded in the geometric space component; for example, the farthest candidate point with the farthest geometric distance from the point to be encoded in the X direction is selected as the target farthest candidate point, the farthest candidate point with the farthest geometric distance from the point to be encoded in the Y direction is selected as the target farthest candidate point, or the farthest candidate point with the farthest geometric distance from the point to be encoded in the Z direction is selected as the target farthest candidate point.Fifthly, the target farthest prediction point may be selected from among the farthest prediction points according to the attribute distance between each farthest prediction point and the preceding point of the point to be encoded, wherein the preceding point refers to a point that precedes the point to be encoded in the encoding order.
[0402] For the fifth method of selecting the target farthest candidate point among the farthest candidate points (i.e., selecting the target farthest prediction point among each farthest prediction point based on the attribute distance between each farthest prediction point and the predecessor point of the point to be encoded): when the attribute encoding mode of the point to be encoded is the attribute prediction mode, the attribute distance between the predecessor point and the farthest candidate point can be determined based on the difference between the attribute reconstruction information of the predecessor point and the attribute reconstruction information of the farthest candidate point (similar to the method of calculating the attribute distance between the point to be encoded and the farthest candidate point in the third method of selecting the target farthest candidate point among the farthest candidate points mentioned above, which will not be repeated here); then, among each farthest candidate point, the farthest candidate point with the largest or smallest attribute distance to the predecessor point can be selected as the target farthest candidate point. Alternatively, when the attribute coding mode of the point to be encoded is the attribute prediction transform coding mode, the point to be encoded belongs to the target point cloud group in the point cloud; the target preceding point can be selected from the preceding points of the point to be encoded; based on the difference between the attribute reconstruction information of the target preceding point and the attribute reconstruction information of the farthest alternative point, the attribute distance between the target preceding point and the farthest alternative point is determined (similar to the method of calculating the attribute distance between the point to be encoded and the farthest alternative point in the third method of selecting the target farthest alternative point from the farthest alternative points mentioned above, which will not be repeated here); then, among each farthest alternative point, the farthest alternative point with the largest or smallest attribute distance to the target preceding point can be selected as the target farthest alternative point.
[0403] Among them, the method of selecting the target pre-sequence point in the pre-sequence points of the points to be encoded may include any of the following: ① Select the first pre-sequence point or the last pre-sequence point in the pre-sequence point cloud group of the target point cloud group as the target pre-sequence point, the pre-sequence point cloud group is a point cloud group whose coding order is before the target point cloud group; and all the points to be encoded in the target point cloud group select the first pre-sequence point or the last pre-sequence point in the pre-sequence point cloud group as the target pre-sequence point. That is to say, there are M points [p1, p2, …, pM] in the target point cloud group, and all the points to be encoded in the target point cloud group select the first pre-sequence point (i.e., the first pre-sequence point) or the last pre-sequence point (i.e., the last pre-sequence point) in the pre-sequence point cloud group as the target pre-sequence point. ② The target pre-sequence point can be selected from the pre-sequence points of the points to be encoded according to the position of the points to be encoded in the target point cloud group. Specifically: if the point to be encoded is the first point to be encoded in the target point cloud group, select the first pre-order point or the last pre-order point in the previous point cloud group of the target point cloud group as the target pre-order point; that is, there are M points [p1, p2, ..., pM] in the target point cloud group, for the point to be encoded p1, the first pre-order point (i.e., the first pre-order point) or the last pre-order point (i.e., the last pre-order point) in the previous point cloud group can be selected as the target pre-order point. If the point to be encoded is not the first point to be encoded in the target point cloud group, select the first pre-order point in the target point cloud group, or the pre-order point in the target point cloud group whose coding order is arranged before the point to be encoded, as the target pre-order point; that is, there are M points [p1, p2, ..., pM] in the target point cloud group, for the point to be encoded p2, ..., pM, the first pre-order point in the target point cloud group, or the pre-order point in the target point cloud group whose coding order is arranged before the point to be encoded, can be selected as the target pre-order point.
[0404] (2) When the point to be encoded has attribute information under multiple attribute types, the method of selecting the target farthest alternative point from the farthest alternative points may include any of the following: ① Select the target farthest alternative point under each attribute type respectively from the farthest alternative points; the selection method under any attribute type can refer to the above-mentioned method of selecting the target farthest alternative point from the farthest alternative points when the point to be encoded has attribute information under one attribute type, which will not be repeated here. ② Select the target farthest alternative point under any of the following attribute types from the farthest alternative points: first predicted attribute type, post-predicted attribute type, and target attribute type; the target attribute type is any of the multiple attribute types; the selection method under any attribute type can refer to the above-mentioned method of selecting the target farthest alternative point from the farthest alternative points when the point to be encoded has attribute information under one attribute type, which will not be repeated here. ③ Select the farthest predicted point of the target from the farthest candidate point based on the attribute weighted distance of multiple attribute types or the geometric attribute comprehensive distance; the attribute weighted distance may refer to the weighted result of the attribute distance between the farthest candidate point and the point to be encoded under each attribute type, or may refer to the weighted result of the attribute distance between the farthest candidate point and the preceding point of the point to be encoded under each attribute type; the geometric attribute comprehensive distance may refer to the geometric attribute comprehensive distance between the farthest candidate point and the point to be encoded, or may refer to the geometric attribute comprehensive distance between the farthest candidate point and the preceding point of the point to be encoded. For the calculation of the geometric attribute comprehensive distance, please refer to the relevant contents of the above formulas 1 to 4.
[0405] It should be added that the point to be encoded belongs to the target point cloud group in the point cloud, and the target point cloud group is grouped based on the target right shift number L. The specific grouping rule is that the Hilbert code (or Morton code) of each point in the point cloud is right-shifted by the target right shift number L, and the same points are divided into a point cloud group (or macroblock). If the number of points in the point cloud group is greater than the maximum transformation order MaxTransNum, the point cloud group needs to be further subdivided, and the grouping needs to be dynamically adjusted based on the target right shift number L. The dynamic adjustment can be based on the subdivision of the preceding point cloud group of the target point cloud group. The preceding point cloud group refers to the point cloud group whose coding order is before the target point cloud group. Specifically, the target right shift number can be updated according to the subdivision of the preceding point cloud group of the target point cloud group; the updated target right shift number can be used as the grouping basis for the subsequent point cloud group of the target point cloud group, and the subsequent point cloud group refers to the point cloud group whose coding order is after the target point cloud group.
[0406] More specifically, according to the subdivision situation of the previous point cloud grouping for the target point cloud grouping, updating the target right shift number may include any of the following: First, if the previous point cloud grouping is subdivided, the target right shift number can be reduced. For example, L = L - 1; or if the previous point cloud grouping is subdivided, the target right shift number can be reduced when the target right shift number meets the update condition. For example, when L < th_L (shift determination threshold), L = L - 1. Second, if the previous point cloud grouping is not subdivided, the target right shift number can remain unchanged, or if the previous point cloud grouping is not subdivided, the target right shift number can be increased. For example, L = L + 1. Third, the update direction of the target right shift number can be determined according to the number of previous point cloud groupings that are subdivided in the previous point cloud grouping.
[0407] In the embodiment of the present application, in the point cloud attribute encoding stage, according to the distance between the point to be encoded and the prediction reference point, attribute prediction is performed on the point to be encoded, and the attribute approximation situation between the prediction reference point used for attribute prediction of the point to be encoded and the point to be encoded is considered, which can improve the attribute prediction effect of the point to be encoded and improve the encoding performance of the point cloud attribute.
[0408] The method of the embodiment of the present application is elaborated in detail above. To facilitate better implementation of the above solution of the embodiment of the present application, correspondingly, the device of the embodiment of the present application is provided below.
[0409] Please refer to Figure 5 , Figure 5 which is a schematic structural diagram of a point cloud processing device provided by the embodiment of the present application. The point cloud processing device can be set in the computer device provided by the embodiment of the present application, and the computer device can be the decoding device mentioned throughout the text. Figure 5 The point cloud processing device shown can be a computer program (including program code) running in a computer device, and the point cloud processing device can be used to execute Figure 3 part or all of the steps in the method embodiment shown. Please refer to Figure 5 , and the point cloud processing device can include the following units:
[0410] The determination unit 501 is used to determine the point to be decoded in the point cloud;
[0411] The processing unit 502 is used to determine the prediction reference point of the point to be decoded in the alternative point set of the point to be decoded;
[0412] The processing unit 502 is further used to perform attribute prediction on the point to be decoded according to the distance between the point to be decoded and the prediction reference point, and obtain the attribute prediction information of the point to be decoded;
[0413] The processing unit 502 is further configured to reconstruct the attributes of the points to be decoded according to the attribute prediction information of the points to be decoded, so as to obtain the attribute reconstruction information of the points to be decoded.
[0414] In one implementation, the processing unit 502 is configured to perform attribute prediction on the to-be-decoded point according to the distance between the to-be-decoded point and the prediction reference point, and to obtain the attribute prediction information of the to-be-decoded point, specifically to perform the following steps:
[0415] According to the geometric distance between the point to be decoded and the prediction reference point, k first prediction reference points whose geometric distances to the point to be decoded are less than a first distance threshold are obtained from the prediction reference points, and r second prediction reference points whose geometric distances to the point to be decoded are equal to the first distance threshold are obtained from the prediction reference points; k and r are both non-negative integers;
[0416] According to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points, attribute prediction is performed on the point to be decoded to obtain the attribute prediction information of the point to be decoded.
[0417] In one implementation, the processing unit 502 is further configured to perform the following steps:
[0418] Selecting a third prediction reference point from the r second prediction reference points;
[0419] The processing unit 502 is used to perform attribute prediction on the to-be-decoded point according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points, and to obtain the attribute prediction information of the to-be-decoded point, specifically to perform the following steps:
[0420] According to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the third prediction reference point, attribute prediction is performed on the point to be decoded to obtain the attribute prediction information of the point to be decoded.
[0421] In one implementation, the processing unit 502 is configured to perform the following steps when selecting the third prediction reference point from the r second prediction reference points:
[0422] Obtain the difference information between the attribute reconstruction information of any two second prediction reference points among the r second prediction reference points, and determine the two second prediction reference points corresponding to the largest difference information as the third prediction reference point; or, determine the second prediction reference point among the r second prediction reference points that is closest to the point to be decoded in the geometric space component as the third prediction reference point.
[0423] In one implementation, the processing unit 502 is configured to perform attribute prediction on the to-be-decoded point according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the to-be-decoded point, specifically to perform the following steps:
[0424] When k=0 and r>0, attribute prediction is performed on the point to be decoded according to the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the point to be decoded;
[0425] When k>0 and r>0, attribute prediction is performed on the point to be decoded according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the point to be decoded;
[0426] When k>0 and r=0, attribute prediction is performed on the point to be decoded according to the attribute reconstruction information of the k first prediction reference points to obtain the attribute prediction information of the point to be decoded.
[0427] In one implementation, the processing unit 502 is configured to perform attribute prediction on the to-be-decoded point according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the to-be-decoded point, specifically to perform the following steps:
[0428] Obtaining a first weighting coefficient for each of the k first prediction reference points, and obtaining a second weighting coefficient for each of the r second prediction reference points;
[0429] Obtain weight optimization coefficients of r second prediction reference points;
[0430] According to the weight optimization coefficient, optimizing the second weighting coefficient of each of the r second prediction reference points to obtain the second optimized weighting coefficient of each of the r second prediction reference points;
[0431] According to the first weighting coefficients of the k first prediction reference points and the second optimized weighting coefficients of the r second prediction reference points, the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points are weighted and summed to obtain the attribute prediction information of the point to be decoded.
[0432] In one implementation, the processing unit 502, when used to obtain the weight optimization coefficients of the r second prediction reference points, is used to perform any of the following:
[0433] Numerical calculation is performed on the total number of second prediction reference points to obtain a weight optimization coefficient;
[0434] Numerical calculation is performed on the total number of second prediction reference points and the attribute quantization step length of the to-be-decoded point to obtain a weight optimization coefficient;
[0435] According to the attribute coding condition satisfied by the to-be-decoded point, a weight optimization coefficient corresponding to the satisfied attribute coding condition is determined.
[0436] In one implementation, the point to be decoded belongs to a target point cloud group in the point cloud; the processing unit 502 is used to perform attribute prediction on the point to be decoded according to the distance between the point to be decoded and the prediction reference point, and when the attribute prediction information of the point to be decoded is obtained, it is specifically used to perform the following steps:
[0437] When the index value of the to-be-decoded point in the target point cloud group is greater than the upper limit of the length of the candidate point set, the default prediction value determination distance is determined according to the geometric distance between the to-be-decoded point and the target prediction reference point in the prediction reference point;
[0438] If the default prediction value determination distance is greater than the second distance threshold, obtaining attribute default prediction information;
[0439] According to the attribute default prediction information, the attribute prediction of the point to be decoded is performed to obtain the attribute prediction information of the point to be decoded;
[0440] Among them, the target prediction reference point is the p prediction reference points among the prediction reference points whose geometric distance to the point to be decoded is closest or farthest, and p is a positive integer; the default prediction value determination distance includes any one of the following: the maximum geometric distance, the minimum geometric distance or the mean geometric distance among the geometric distances between the point to be decoded and the p prediction reference points; the attribute default prediction information includes any one of the following: fixed attribute default prediction information, and attribute default prediction information determined according to the attribute bit depth of the point to be decoded.
[0441] In one implementation, the points to be decoded in the point cloud are arranged in sequence according to the decoding order; the processing unit 502 is used to obtain the attribute default prediction information, specifically to perform the following steps:
[0442] Obtaining attribute determination information of a preceding point of the point to be decoded; the preceding point refers to a point in the point cloud that is located before the point to be decoded in decoding order; the attribute determination information includes a mean value of attribute reconstruction information or a mean value of attribute prediction information of the preceding point;
[0443] According to the relationship between the attribute determination information and the attribute determination threshold, the attribute default prediction information is determined.
[0444] In one implementation, the point to be decoded belongs to a target point cloud group in the point cloud; the processing unit 502 is further configured to perform the following steps:
[0445] When the index value of the to-be-decoded point in the target point cloud group is greater than the upper limit of the length of the candidate point set, the exchange determination distance is obtained; the exchange determination distance includes any of the following: the geometric distance between the to-be-decoded point and the farthest predicted reference point among the predicted reference points, the geometric distance between the to-be-decoded point and the nearest predicted reference point among the predicted reference points, and the average of the geometric distances between the to-be-decoded point and the predicted reference points;
[0446] If the exchange determination distance is greater than or equal to the third distance threshold, the candidate point set remains unchanged;
[0447] If the exchange determination distance is less than the third distance threshold, the farthest candidate point with the farthest geometric distance from the point to be decoded is determined in the candidate point set, and the reconstruction information of the farthest candidate point is exchanged with the reconstruction information of the nth candidate point in the candidate point set;
[0448] The attribute reconstruction information of the to-be-decoded point is updated to the candidate point set, and the updated candidate point set is applied to the attribute prediction process of the subsequent to-be-decoded points of the to-be-decoded point.
[0449] In one implementation, there are multiple farthest candidate points; the processing unit 502 is further configured to perform the following steps:
[0450] Select the farthest candidate point of the target among the farthest candidate points;
[0451] The processing unit 502 is used to exchange the reconstruction information of the farthest candidate point with the reconstruction information of the nth candidate point in the candidate point set, and is specifically used to perform the following steps:
[0452] The reconstruction information of the farthest candidate point of the target is exchanged with the reconstruction information of the nth candidate point in the candidate point set.
[0453] In one implementation, the processing unit 502 is configured to perform the following steps when selecting the target farthest candidate point from the farthest candidate points:
[0454] According to the order of the farthest candidate points in the candidate point set, the farthest candidate point of the target is selected from the farthest candidate points;
[0455] Selecting the target farthest candidate point from among the farthest candidate points according to the search order of the farthest candidate points in the candidate point set;
[0456] According to the attribute distance between each farthest candidate point and the point to be decoded, the farthest candidate point of the target is selected from each farthest candidate point;
[0457] A target farthest candidate point is selected from among the farthest candidate points according to the attribute distance between each farthest candidate point and the preceding point of the point to be decoded; the preceding point refers to a point that precedes the point to be decoded in decoding order.
[0458] According to the geometric distance between each farthest candidate point and the to-be-decoded point in the geometric space component, a target farthest candidate point is selected from each farthest candidate point.
[0459] In one implementation, the attribute coding mode of the point to be decoded is an attribute prediction mode; the processing unit 502 is used to select the target farthest candidate point from each farthest candidate point according to the attribute distance between each farthest candidate point and the preceding point of the point to be decoded, and is specifically used to perform the following steps:
[0460] Determine the attribute distance between the preceding point and the farthest candidate point according to the difference between the attribute reconstruction information of the preceding point and the attribute reconstruction information of the farthest candidate point;
[0461] Among the farthest candidate points, the farthest candidate point with the largest or smallest attribute distance to the previous point is selected as the target farthest candidate point.
[0462] In one implementation, the attribute coding mode of the point to be decoded is the attribute prediction transform coding mode, and the point to be decoded belongs to the target point cloud group in the point cloud; the processing unit 502 is used to select the target farthest candidate point from each farthest candidate point according to the attribute distance between each farthest candidate point and the preceding point of the point to be decoded, and is specifically used to perform the following steps:
[0463] Select a target preceding point from the preceding points of the point to be decoded;
[0464] Determine the attribute distance between the target predecessor point and the farthest candidate point according to the difference between the attribute reconstruction information of the target predecessor point and the attribute reconstruction information of the farthest candidate point;
[0465] Among the farthest candidate points, the farthest candidate point with the largest or smallest attribute distance to the target predecessor point is selected as the target farthest candidate point.
[0466] In one implementation, the processing unit 502 is used to select a target preamble point from the preamble points of the point to be decoded, and is specifically used to perform the following steps:
[0467] The first pre-sequence point or the last pre-sequence point in the preceding point cloud group of the target point cloud group is selected as the target pre-sequence point; all the points to be decoded in the target point cloud group select the first pre-sequence point or the last pre-sequence point in the preceding point cloud group as the target pre-sequence point;
[0468] According to the position of the to-be-decoded point in the target point cloud group, a target preceding point is selected from the preceding points of the to-be-decoded point;
[0469] The preceding point cloud group is a point cloud group whose decoding order is before the target point cloud group.
[0470] In one implementation, the processing unit 502 is used to select a target preceding point from the preceding points of the point to be decoded according to the position of the point to be decoded in the target point cloud group, and is specifically used to perform the following steps:
[0471] If the point to be decoded is the first point to be decoded in the target point cloud group, the first pre-sequence point or the last pre-sequence point in the pre-sequence point cloud group of the target point cloud group is selected as the target pre-sequence point;
[0472] If the point to be decoded is not the first point to be decoded in the target point cloud group, the preceding point in the target point cloud group that is arranged first, or the preceding point in the target point cloud group that is arranged one position before the point to be decoded in the decoding order, is selected as the target preceding point.
[0473] In one implementation, there is attribute information of multiple attribute types at the point to be decoded; the processing unit 502 is used to select the target farthest candidate point from the farthest candidate points, and is specifically used to perform any of the following:
[0474] Select the target farthest candidate point under each attribute type from the farthest candidate points;
[0475] Selecting a target farthest candidate point under any of the following attribute types from the farthest candidate points: a first predicted attribute type, a second predicted attribute type, and a target attribute type; the target attribute type is any one of the multiple attribute types;
[0476] The farthest candidate point of the target is selected from the farthest candidate points based on the attribute weighted distance of multiple attribute types or the comprehensive distance of geometric attributes.
[0477] In one implementation, the point to be decoded belongs to a target point cloud group in the point cloud, and the target point cloud group is grouped according to the target right shift bit number; the processing unit 502 is further used to perform the following steps:
[0478] According to the subdivision of the preceding point cloud grouping of the target point cloud grouping, the target right shift bit number is updated; the updated target right shift bit number is used as the grouping basis for the subsequent point cloud grouping of the target point cloud grouping;
[0479] Among them, the preceding point cloud group is a point cloud group whose decoding order is before the target point cloud group, and the subsequent point cloud group is a point cloud group whose decoding order is after the target point cloud group.
[0480] In one implementation, the processing unit 502 is used to update the target right shift bit number according to the subdivision of the preceding point cloud group of the target point cloud group, and is specifically used to perform any of the following:
[0481] If the previous point cloud group is subdivided, the target right shift number is reduced, or when the target right shift number meets the update condition, the target right shift number is reduced;
[0482] If the previous point cloud group is not subdivided, the target right shift bit number is kept unchanged or increased;
[0483] The updating method of the target right shift bit number is determined according to the number of previous point cloud groups subdivided in the previous point cloud grouping.
[0484] In one implementation, there is attribute information of multiple attribute types at the point to be decoded; the processing unit 502 is used to determine the prediction reference point of the point to be decoded from the candidate point set of the point to be decoded, and is specifically used to perform the following steps:
[0485] Obtaining the geometric distance between the point to be decoded and the candidate points in the candidate point set;
[0486] Obtain the decoded attribute distance between the point to be decoded and the candidate points in the candidate point set;
[0487] Determine the comprehensive distance between the point to be decoded and the corresponding candidate point according to the geometric distance between the point to be decoded and the corresponding candidate point, and the decoded attribute distance between the point to be decoded and the corresponding candidate point;
[0488] According to the comprehensive distance between the point to be decoded and the candidate points in the candidate point set, a prediction reference point of the point to be decoded is determined in the candidate point set.
[0489] In one implementation, the decoded attribute distance between the point to be decoded and the candidate point is determined according to the difference between the attribute reconstruction information of the point to be decoded under the decoded attribute type and the attribute reconstruction information of the candidate point under the decoded attribute type; when the decoded attribute type is a color attribute type, the processing unit 502 is used to obtain the decoded attribute distance between the point to be decoded and the candidate point in the candidate point set, specifically to perform the following steps:
[0490] Determine the attribute reconstruction information selected for the to-be-decoded point and the candidate point according to the color space;
[0491] The decoded attribute distance is determined according to the difference between the attribute reconstruction information selected for the point to be decoded and the attribute reconstruction information selected for the candidate point.
[0492] In one implementation, the processing unit 502 is used to determine the attribute reconstruction information selected by the to-be-decoded point and the candidate point according to the color space, and is specifically used to perform the following steps:
[0493] When the color space is the first color space, the attribute reconstruction component of the to-be-decoded point under the target color component of the first color space is used as the attribute reconstruction information selected for the to-be-decoded point, and the attribute reconstruction component of the candidate point under the target color component of the first color space is used as the attribute reconstruction information selected for the candidate point;
[0494] When the color space is the second color space, the attribute reconstruction components of the point to be decoded under each color component in the second color space are used as the attribute reconstruction information selected for the point to be decoded, and the attribute reconstruction components of the candidate point under each color component in the second color space are used as the attribute reconstruction information selected for the candidate point.
[0495] In one implementation, the comprehensive distance between the point to be decoded and the corresponding candidate point is obtained by summing the geometric distance between the point to be decoded and the corresponding candidate point and the weighted result of the decoded attribute distance between the point to be decoded and the corresponding candidate point; the process of determining the weighted factor of the attribute distance between the point to be decoded and the corresponding candidate point includes:
[0496] Calculating the weighting factor according to the attribute quantization parameter;
[0497] The attribute quantization parameter is an attribute quantization parameter corresponding to a decoded attribute type, or an attribute quantization parameter corresponding to an attribute type to be decoded.
[0498] In one implementation, the processing unit 502 is used to reconstruct the attributes of the to-be-decoded point according to the attribute prediction information of the to-be-decoded point, and after obtaining the attribute reconstruction information of the to-be-decoded point, is further used to perform the following steps:
[0499] Get the cutoff range of the attribute reconstruction information of the point to be decoded;
[0500] The attribute reconstruction information of the decoding point is truncated according to the truncation range.
[0501] In one implementation, the processing unit 502 is used to obtain the truncation range of the attribute reconstruction information of the to-be-decoded point, and is specifically used to perform any of the following:
[0502] Parse the truncation range from the point cloud bitstream. The truncation range is calculated by the encoding device based on the attribute output bit depth.
[0503] Obtaining the truncation range corresponding to the attribute type of the point to be decoded;
[0504] The truncation range is determined according to the bit depth condition satisfied by the attribute output bit depth of the to-be-decoded point and the attribute type possessed by the to-be-decoded point.
[0505] According to another embodiment of the present application, Figure 5The various units in the point cloud processing device shown can be individually or completely combined into one or several other units, or one (some) of the units can be further divided into multiple functionally smaller units, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the point cloud processing device may also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.
[0506] According to another embodiment of the present application, the program can be executed by running on a general computing device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM), and other processing elements and storage elements. Figure 3 A computer program (including program code) for each step involved in part or all of the method shown in the figure is used to construct a computer program (including program code) for each step involved in part or all of the method shown in the figure. Figure 5 The point cloud processing device shown in the embodiment of the present application is used to implement the point cloud processing method of the present application. The computer program can be recorded on a computer-readable storage medium, for example, and loaded into the above-mentioned computing device through the computer-readable storage medium and run therein.
[0507] In an embodiment of the present application, in the point cloud attribute decoding stage, attribute prediction is performed on the point to be decoded based on the distance between the point to be decoded and the predicted reference point, and the attribute approximation between the predicted reference point used for attribute prediction of the point to be decoded and the point to be decoded is taken into consideration. This can improve the attribute prediction effect of the point to be decoded and improve the decoding performance of the point cloud attributes.
[0508] See also Figure 5 , Figure 5 It is a structural schematic diagram of a point cloud processing device provided in an embodiment of the present application. The point cloud processing device can be set in the computer device provided in the embodiment of the present application, and the computer device can be the encoding device mentioned in the full text. Figure 5 The point cloud processing device shown may be a computer program (including program code) running in a computer device, and the point cloud processing device may be used to execute Figure 4 Some or all of the steps in the method embodiment shown. Figure 5 , the point cloud processing device may include the following units:
[0509] A determination unit 501, used to determine the points to be encoded in the point cloud;
[0510] A processing unit 502 is used to determine a prediction reference point of the point to be encoded from a candidate point set of the point to be encoded;
[0511] The processing unit 502 is further configured to perform attribute prediction on the to-be-coded point according to the distance between the to-be-coded point and the prediction reference point, and obtain attribute prediction information of the to-be-coded point;
[0512] The processing unit 502 is further configured to perform attribute encoding on the point to be encoded according to the attribute prediction information of the point to be encoded.
[0513] According to another embodiment of the present application, Figure 5 The various units in the point cloud processing device shown can be individually or completely combined into one or several other units, or one (some) of the units can be further divided into multiple functionally smaller units, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the point cloud processing device may also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.
[0514] According to another embodiment of the present application, the program can be executed by running on a general computing device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM), and other processing elements and storage elements. Figure 4 A computer program (including program code) for each step involved in part or all of the method shown in the figure is used to construct a computer program (including program code) for each step involved in part or all of the method shown in the figure. Figure 5 The point cloud processing device shown in the embodiment of the present application is used to implement the point cloud processing method of the present application. The computer program can be recorded on a computer-readable storage medium, for example, and loaded into the above-mentioned computing device through the computer-readable storage medium and run therein.
[0515] In an embodiment of the present application, in the point cloud attribute coding stage, attribute prediction is performed on the point to be coded based on the distance between the point to be coded and the prediction reference point, and the attribute approximation between the prediction reference point used for attribute prediction of the point to be coded and the point to be coded is taken into consideration. This can improve the attribute prediction effect of the point to be coded and improve the coding performance of the point cloud attributes.
[0516] Based on the above method and device embodiments, the present application embodiment provides a computer device. Figure 6 , Figure 6 It is a structural diagram of a computer device provided in an embodiment of the present application. Figure 6The computer device shown includes at least a processor 601, an input interface 602, an output interface 603, and a computer-readable storage medium 604. The processor 601, the input interface 602, the output interface 603, and the computer-readable storage medium 604 may be connected via a bus or other means.
[0517] The computer-readable storage medium 604 may be stored in a memory of a computer device. The computer-readable storage medium 604 is used to store a computer program. The computer program includes computer instructions. The processor 601 is used to execute the program instructions stored in the computer-readable storage medium 604. The processor 601 (or CPU (Central Processing Unit)) is the computing core and control core of the computer device, which is suitable for implementing one or more computer instructions, and is specifically suitable for loading and executing one or more computer instructions to implement a corresponding method flow or a corresponding function.
[0518] The embodiment of the present application also provides a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space that stores the operating system of the computer device. In addition, one or more computer instructions suitable for being loaded and executed by the processor are also stored in the storage space. These computer instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory (Non-VolatileMemory), such as at least one disk storage; optionally, it can also be at least one computer-readable storage medium located away from the aforementioned processor.
[0519] In some embodiments, the computer device may be a decoding device, and the computer device may load and execute one or more computer instructions stored in the computer-readable storage medium 604 by the processor 601 to implement the above-mentioned Figure 3 In a specific implementation, the computer instructions in the computer-readable storage medium 604 are loaded by the processor 601 and execute the following steps:
[0520] Determine the points to be decoded in the point cloud;
[0521] Determine a prediction reference point of the point to be decoded in a candidate point set of the point to be decoded;
[0522] According to the distance between the to-be-decoded point and the prediction reference point, the attribute prediction information of the to-be-decoded point is obtained;
[0523] According to the attribute prediction information of the to-be-decoded point, the attribute of the to-be-decoded point is reconstructed to obtain the attribute reconstruction information of the to-be-decoded point.
[0524] In one implementation, the computer instructions in the computer-readable storage medium 604 are loaded and executed by the processor 601 to perform attribute prediction on the point to be decoded according to the distance between the point to be decoded and the prediction reference point, and to obtain the attribute prediction information of the point to be decoded, specifically for performing the following steps:
[0525] According to the geometric distance between the point to be decoded and the prediction reference point, k first prediction reference points whose geometric distances to the point to be decoded are less than a first distance threshold are obtained from the prediction reference points, and r second prediction reference points whose geometric distances to the point to be decoded are equal to the first distance threshold are obtained from the prediction reference points; k and r are both non-negative integers;
[0526] According to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points, attribute prediction is performed on the point to be decoded to obtain the attribute prediction information of the point to be decoded.
[0527] In one implementation, the computer instructions in the computer readable storage medium 604 are loaded by the processor 601 and are further used to perform the following steps:
[0528] Selecting a third prediction reference point from the r second prediction reference points;
[0529] The computer instructions in the computer readable storage medium 604 are loaded and executed by the processor 601 to perform attribute prediction on the to-be-decoded point according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the to-be-decoded point, and are specifically used to perform the following steps:
[0530] According to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the third prediction reference point, attribute prediction is performed on the point to be decoded to obtain the attribute prediction information of the point to be decoded.
[0531] In one implementation, when the computer instructions in the computer-readable storage medium 604 are loaded and executed by the processor 601 to select the third prediction reference point from the r second prediction reference points, the computer instructions are specifically used to perform the following steps:
[0532] Obtain the difference information between the attribute reconstruction information of any two second prediction reference points among the r second prediction reference points, and determine the two second prediction reference points corresponding to the largest difference information as the third prediction reference point; or, determine the second prediction reference point among the r second prediction reference points that is closest to the point to be decoded in the geometric space component as the third prediction reference point.
[0533] In one implementation, the computer instructions in the computer-readable storage medium 604 are loaded and executed by the processor 601 to perform attribute prediction on the to-be-decoded point according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the to-be-decoded point, specifically for performing the following steps:
[0534] When k=0 and r>0, attribute prediction is performed on the point to be decoded according to the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the point to be decoded;
[0535] When k>0 and r>0, attribute prediction is performed on the point to be decoded according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the point to be decoded;
[0536] When k>0 and r=0, attribute prediction is performed on the point to be decoded according to the attribute reconstruction information of the k first prediction reference points to obtain the attribute prediction information of the point to be decoded.
[0537] In one implementation, the computer instructions in the computer-readable storage medium 604 are loaded and executed by the processor 601 to perform attribute prediction on the to-be-decoded point according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the to-be-decoded point, specifically for performing the following steps:
[0538] Obtaining a first weighting coefficient for each of the k first prediction reference points, and obtaining a second weighting coefficient for each of the r second prediction reference points;
[0539] Obtain weight optimization coefficients of r second prediction reference points;
[0540] According to the weight optimization coefficient, optimizing the second weighting coefficient of each of the r second prediction reference points to obtain the second optimized weighting coefficient of each of the r second prediction reference points;
[0541] According to the first weighting coefficients of the k first prediction reference points and the second optimized weighting coefficients of the r second prediction reference points, the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points are weighted and summed to obtain the attribute prediction information of the point to be decoded.
[0542] In one implementation, when the computer instructions in the computer-readable storage medium 604 are loaded and executed by the processor 601 to obtain the weight optimization coefficients of the r second prediction reference points, the computer instructions are used to perform any of the following:
[0543] Numerical calculation is performed on the total number of second prediction reference points to obtain a weight optimization coefficient;
[0544] Numerical calculation is performed on the total number of second prediction reference points and the attribute quantization step length of the to-be-decoded point to obtain a weight optimization coefficient;
[0545] According to the attribute coding condition satisfied by the to-be-decoded point, a weight optimization coefficient corresponding to the satisfied attribute coding condition is determined.
[0546] In one implementation, the point to be decoded belongs to a target point cloud group in the point cloud; the computer instructions in the computer readable storage medium 604 are loaded and executed by the processor 601 to perform attribute prediction on the point to be decoded according to the distance between the point to be decoded and the prediction reference point, and to obtain the attribute prediction information of the point to be decoded, specifically for executing the following steps:
[0547] When the index value of the to-be-decoded point in the target point cloud group is greater than the upper limit of the length of the candidate point set, the default prediction value determination distance is determined according to the geometric distance between the to-be-decoded point and the target prediction reference point in the prediction reference point;
[0548] If the default prediction value determination distance is greater than the second distance threshold, obtaining attribute default prediction information;
[0549] According to the attribute default prediction information, the attribute prediction of the point to be decoded is performed to obtain the attribute prediction information of the point to be decoded;
[0550] Among them, the target prediction reference point is the p prediction reference points among the prediction reference points whose geometric distance to the point to be decoded is closest or farthest, and p is a positive integer; the default prediction value determination distance includes any one of the following: the maximum geometric distance, the minimum geometric distance or the mean geometric distance among the geometric distances between the point to be decoded and the p prediction reference points; the attribute default prediction information includes any one of the following: fixed attribute default prediction information, and attribute default prediction information determined according to the attribute bit depth of the point to be decoded.
[0551] In one implementation, the points to be decoded in the point cloud are arranged in sequence according to the decoding order; when the computer instructions in the computer readable storage medium 604 are loaded and executed by the processor 601 to obtain the attribute default prediction information, they are specifically used to perform the following steps:
[0552] Obtaining attribute determination information of a preceding point of the point to be decoded; the preceding point refers to a point in the point cloud that is located before the point to be decoded in decoding order; the attribute determination information includes a mean value of attribute reconstruction information or a mean value of attribute prediction information of the preceding point;
[0553] According to the relationship between the attribute determination information and the attribute determination threshold, the attribute default prediction information is determined.
[0554] In one implementation, the point to be decoded belongs to a target point cloud group in the point cloud; the computer instructions in the computer readable storage medium 604 are loaded by the processor 601 and are also used to perform the following steps:
[0555] When the index value of the to-be-decoded point in the target point cloud group is greater than the upper limit of the length of the candidate point set, the exchange determination distance is obtained; the exchange determination distance includes any of the following: the geometric distance between the to-be-decoded point and the farthest predicted reference point among the predicted reference points, the geometric distance between the to-be-decoded point and the nearest predicted reference point among the predicted reference points, and the average of the geometric distances between the to-be-decoded point and the predicted reference points;
[0556] If the exchange determination distance is greater than or equal to the third distance threshold, the candidate point set remains unchanged;
[0557] If the exchange determination distance is less than the third distance threshold, the farthest candidate point with the farthest geometric distance from the point to be decoded is determined in the candidate point set, and the reconstruction information of the farthest candidate point is exchanged with the reconstruction information of the nth candidate point in the candidate point set;
[0558] The attribute reconstruction information of the to-be-decoded point is updated to the candidate point set, and the updated candidate point set is applied to the attribute prediction process of the subsequent to-be-decoded points of the to-be-decoded point.
[0559] In one implementation, the number of the farthest candidate points is multiple; the computer instructions in the computer-readable storage medium 604 are loaded by the processor 601 and are also used to execute the following steps:
[0560] Select the farthest candidate point of the target among the farthest candidate points;
[0561] The processing unit is used to exchange the reconstruction information of the farthest candidate point with the reconstruction information of the nth candidate point in the candidate point set, and is specifically used to perform the following steps:
[0562] The reconstruction information of the farthest candidate point of the target is exchanged with the reconstruction information of the nth candidate point in the candidate point set.
[0563] In one implementation, when the computer instructions in the computer-readable storage medium 604 are loaded and executed by the processor 601 to select the target farthest candidate point from the farthest candidate points, the computer instructions are specifically used to perform the following steps:
[0564] According to the order of the farthest candidate points in the candidate point set, the farthest candidate point of the target is selected from the farthest candidate points;
[0565] Selecting the target farthest candidate point from among the farthest candidate points according to the search order of the farthest candidate points in the candidate point set;
[0566] According to the attribute distance between each farthest candidate point and the point to be decoded, the farthest candidate point of the target is selected from each farthest candidate point;
[0567] A target farthest candidate point is selected from among the farthest candidate points according to the attribute distance between each farthest candidate point and the preceding point of the point to be decoded; the preceding point refers to a point that precedes the point to be decoded in decoding order.
[0568] According to the geometric distance between each farthest candidate point and the to-be-decoded point in the geometric space component, a target farthest candidate point is selected from each farthest candidate point.
[0569] In one implementation, the attribute encoding mode of the point to be decoded is an attribute prediction mode; the computer instructions in the computer readable storage medium 604 are loaded and executed by the processor 601 to select the target farthest candidate point from each farthest candidate point according to the attribute distance between each farthest candidate point and the preceding point of the point to be decoded, and are specifically used to perform the following steps:
[0570] Determine the attribute distance between the preceding point and the farthest candidate point according to the difference between the attribute reconstruction information of the preceding point and the attribute reconstruction information of the farthest candidate point;
[0571] Among the farthest candidate points, the farthest candidate point with the largest or smallest attribute distance to the previous point is selected as the target farthest candidate point.
[0572] In one implementation, the attribute coding mode of the point to be decoded is an attribute prediction transform coding mode, and the point to be decoded belongs to a target point cloud group in the point cloud; the computer instructions in the computer readable storage medium 604 are loaded and executed by the processor 601. When selecting a target farthest candidate point from each farthest candidate point according to the attribute distance between each farthest candidate point and the preceding point of the point to be decoded, the computer instructions are specifically used to perform the following steps:
[0573] Select a target preceding point from the preceding points of the point to be decoded;
[0574] Determine the attribute distance between the target predecessor point and the farthest candidate point according to the difference between the attribute reconstruction information of the target predecessor point and the attribute reconstruction information of the farthest candidate point;
[0575] Among the farthest candidate points, the farthest candidate point with the largest or smallest attribute distance to the target predecessor point is selected as the target farthest candidate point.
[0576] In one implementation, when the computer instructions in the computer-readable storage medium 604 are loaded and executed by the processor 601 to select a target preamble point from the preamble points of the point to be decoded, the computer instructions are specifically used to perform the following steps:
[0577] The first pre-sequence point or the last pre-sequence point in the preceding point cloud group of the target point cloud group is selected as the target pre-sequence point; all the points to be decoded in the target point cloud group select the first pre-sequence point or the last pre-sequence point in the preceding point cloud group as the target pre-sequence point;
[0578] According to the position of the to-be-decoded point in the target point cloud group, a target preceding point is selected from the preceding points of the to-be-decoded point;
[0579] The preceding point cloud group is a point cloud group whose decoding order is before the target point cloud group.
[0580] In one implementation, the computer instructions in the computer-readable storage medium 604 are loaded and executed by the processor 601. When selecting a target preceding point from the preceding points of the point to be decoded according to the position of the point to be decoded in the target point cloud group, the computer instructions are specifically used to perform the following steps:
[0581] If the point to be decoded is the first point to be decoded in the target point cloud group, the first pre-sequence point or the last pre-sequence point in the pre-sequence point cloud group of the target point cloud group is selected as the target pre-sequence point;
[0582] If the point to be decoded is not the first point to be decoded in the target point cloud group, the preceding point in the target point cloud group that is arranged first, or the preceding point in the target point cloud group that is arranged one position before the point to be decoded in the decoding order, is selected as the target preceding point.
[0583] In one implementation, there is attribute information of multiple attribute types at the point to be decoded; when the computer instructions in the computer-readable storage medium 604 are loaded and executed by the processor 601 to select the target farthest candidate point from the farthest candidate points, they are specifically used to perform any of the following:
[0584] Select the target farthest candidate point under each attribute type from the farthest candidate points;
[0585] Selecting a target farthest candidate point under any of the following attribute types from the farthest candidate points: a first predicted attribute type, a second predicted attribute type, and a target attribute type; the target attribute type is any one of the multiple attribute types;
[0586] The farthest candidate point of the target is selected from the farthest candidate points based on the attribute weighted distance of multiple attribute types or the comprehensive distance of geometric attributes.
[0587] In one implementation, the point to be decoded belongs to a target point cloud group in the point cloud, and the target point cloud group is grouped according to the target right shift bit number; the computer instructions in the computer readable storage medium 604 are loaded by the processor 601 and are also used to perform the following steps:
[0588] According to the subdivision of the preceding point cloud grouping of the target point cloud grouping, the target right shift bit number is updated; the updated target right shift bit number is used as the grouping basis for the subsequent point cloud grouping of the target point cloud grouping;
[0589] Among them, the preceding point cloud group is a point cloud group whose decoding order is before the target point cloud group, and the subsequent point cloud group is a point cloud group whose decoding order is after the target point cloud group.
[0590] In one implementation, the computer instructions in the computer-readable storage medium 604 are loaded and executed by the processor 601 to update the target right shift bit according to the subdivision of the preceding point cloud group of the target point cloud group, and are specifically used to perform any of the following:
[0591] If the previous point cloud group is subdivided, the target right shift number is reduced, or when the target right shift number meets the update condition, the target right shift number is reduced;
[0592] If the previous point cloud group is not subdivided, the target right shift bit number is kept unchanged or increased;
[0593] The updating method of the target right shift bit number is determined according to the number of previous point cloud groups subdivided in the previous point cloud grouping.
[0594] In one implementation, there is attribute information of multiple attribute types at the point to be decoded; when the computer instructions in the computer-readable storage medium 604 are loaded and executed by the processor 601 to determine the prediction reference point of the point to be decoded in the candidate point set of the point to be decoded, the computer instructions are specifically used to perform the following steps:
[0595] Obtaining the geometric distance between the point to be decoded and the candidate points in the candidate point set;
[0596] Obtain the decoded attribute distance between the point to be decoded and the candidate points in the candidate point set;
[0597] Determine the comprehensive distance between the point to be decoded and the corresponding candidate point according to the geometric distance between the point to be decoded and the corresponding candidate point, and the decoded attribute distance between the point to be decoded and the corresponding candidate point;
[0598] According to the comprehensive distance between the point to be decoded and the candidate points in the candidate point set, a prediction reference point of the point to be decoded is determined in the candidate point set.
[0599] In one implementation, the decoded attribute distance between the point to be decoded and the candidate point is determined according to the difference between the attribute reconstruction information of the point to be decoded under the decoded attribute type and the attribute reconstruction information of the candidate point under the decoded attribute type; when the decoded attribute type is a color attribute type, the computer instructions in the computer-readable storage medium 604 are loaded and executed by the processor 601 to obtain the decoded attribute distance between the point to be decoded and the candidate point in the candidate point set, specifically for performing the following steps:
[0600] Determine the attribute reconstruction information selected for the to-be-decoded point and the candidate point according to the color space;
[0601] The decoded attribute distance is determined according to the difference between the attribute reconstruction information selected for the point to be decoded and the attribute reconstruction information selected for the candidate point.
[0602] In one implementation, when the computer instructions in the computer-readable storage medium 604 are loaded and executed by the processor 601 to determine the attribute reconstruction information selected for the to-be-decoded point and the candidate point according to the color space, the computer instructions are specifically used to perform the following steps:
[0603] When the color space is the first color space, the attribute reconstruction component of the to-be-decoded point under the target color component of the first color space is used as the attribute reconstruction information selected for the to-be-decoded point, and the attribute reconstruction component of the candidate point under the target color component of the first color space is used as the attribute reconstruction information selected for the candidate point;
[0604] When the color space is the second color space, the attribute reconstruction components of the point to be decoded under each color component in the second color space are used as the attribute reconstruction information selected for the point to be decoded, and the attribute reconstruction components of the candidate point under each color component in the second color space are used as the attribute reconstruction information selected for the candidate point.
[0605] In one implementation, the comprehensive distance between the point to be decoded and the corresponding candidate point is obtained by summing the geometric distance between the point to be decoded and the corresponding candidate point and the weighted result of the decoded attribute distance between the point to be decoded and the corresponding candidate point; the process of determining the weighted factor of the attribute distance between the point to be decoded and the corresponding candidate point includes:
[0606] Calculating the weighting factor according to the attribute quantization parameter;
[0607] The attribute quantization parameter is an attribute quantization parameter corresponding to a decoded attribute type, or an attribute quantization parameter corresponding to an attribute type to be decoded.
[0608] In one implementation, the computer instructions in the computer-readable storage medium 604 are loaded and executed by the processor 601 to reconstruct the attributes of the to-be-decoded point according to the attribute prediction information of the to-be-decoded point, and after obtaining the attribute reconstruction information of the to-be-decoded point, are further used to execute the following steps:
[0609] Get the cutoff range of the attribute reconstruction information of the point to be decoded;
[0610] The attribute reconstruction information of the decoding point is truncated according to the truncation range.
[0611] In one implementation, when the computer instructions in the computer-readable storage medium 604 are loaded and executed by the processor 601 to obtain the truncation range of the attribute reconstruction information of the to-be-decoded point, they are specifically used to perform any of the following:
[0612] Parse the truncation range from the point cloud bitstream. The truncation range is calculated by the encoding device based on the attribute output bit depth.
[0613] Obtaining the truncation range corresponding to the attribute type of the point to be decoded;
[0614] The truncation range is determined according to the bit depth condition satisfied by the attribute output bit depth of the to-be-decoded point and the attribute type possessed by the to-be-decoded point.
[0615] In other embodiments, the computer device may be a coding device, and the computer device may load and execute one or more computer instructions stored in the computer-readable storage medium 604 by the processor 601 to implement the above-mentioned Figure 4 In a specific implementation, the computer instructions in the computer-readable storage medium 604 are loaded by the processor 601 and execute the following steps:
[0616] Determine the points to be encoded in the point cloud;
[0617] Determine a prediction reference point of the point to be coded from a set of candidate points of the point to be coded;
[0618] According to the distance between the to-be-coded point and the prediction reference point, the to-be-coded point is subjected to attribute prediction to obtain attribute prediction information of the to-be-coded point;
[0619] According to the attribute prediction information of the point to be encoded, attribute encoding is performed on the point to be encoded.
[0620] In an embodiment of the present application, in the point cloud attribute decoding stage, attribute prediction is performed on the point to be decoded according to the distance between the point to be decoded and the prediction reference point, and the attribute approximation between the prediction reference point used for attribute prediction of the point to be decoded and the point to be decoded is taken into account, which can improve the attribute prediction effect of the point to be decoded and improve the decoding performance of the point cloud attributes. In the point cloud attribute encoding stage, attribute prediction is performed on the point to be encoded according to the distance between the point to be encoded and the prediction reference point, and the attribute approximation between the prediction reference point used for attribute prediction of the point to be encoded and the point to be encoded is taken into account, which can improve the attribute prediction effect of the point to be encoded and improve the encoding performance of the point cloud attributes. Therefore, the embodiment of the present application can improve the encoding and decoding performance of point cloud attributes.
[0621] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the point cloud processing method provided in the above-mentioned various optional manners.
[0622] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A point cloud processing method, It is characterized in that include: Determine the points to be decoded in the point cloud; Determining a prediction reference point of the point to be decoded in a candidate point set of the point to be decoded; According to the distance between the to-be-decoded point and the prediction reference point, attribute prediction is performed on the to-be-decoded point to obtain attribute prediction information of the to-be-decoded point; According to the attribute prediction information of the point to be decoded, the attribute of the point to be decoded is reconstructed to obtain the attribute reconstruction information of the point to be decoded.
2. The method according to claim 1, It is characterized in that The step of performing attribute prediction on the point to be decoded according to the distance between the point to be decoded and the prediction reference point to obtain attribute prediction information of the point to be decoded includes: According to the geometric distance between the point to be decoded and the prediction reference point, k first prediction reference points whose geometric distances to the point to be decoded are less than a first distance threshold are obtained from the prediction reference points, and r second prediction reference points whose geometric distances to the point to be decoded are equal to the first distance threshold are obtained from the prediction reference points; k and r are both non-negative integers; According to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points, attribute prediction is performed on the point to be decoded to obtain the attribute prediction information of the point to be decoded.
3. The method according to claim 2, It is characterized in that The method further comprises: Selecting a third prediction reference point from the r second prediction reference points; The step of performing attribute prediction on the point to be decoded according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the point to be decoded includes: According to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the third prediction reference point, attribute prediction is performed on the point to be decoded to obtain the attribute prediction information of the point to be decoded.
4. The method according to claim 3, It is characterized in that The selecting a third prediction reference point from the r second prediction reference points comprises: obtaining difference information between the attribute reconstruction information of any two second prediction reference points among the r second prediction reference points, and determining the two second prediction reference points corresponding to the largest difference information as the third prediction reference points; or, A second prediction reference point among the r second prediction reference points, which is closest to the point to be decoded in terms of the geometric space component, is determined as the third prediction reference point.
5. The method according to claim 2, It is characterized in that The step of performing attribute prediction on the point to be decoded according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the point to be decoded includes: When k=0 and r>0, performing attribute prediction on the point to be decoded according to the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the point to be decoded; When k>0 and r>0, performing attribute prediction on the point to be decoded according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the point to be decoded; When k>0 and r=0, attribute prediction is performed on the point to be decoded according to the attribute reconstruction information of the k first prediction reference points to obtain the attribute prediction information of the point to be decoded.
6. The method according to claim 5, It is characterized in that The step of performing attribute prediction on the point to be decoded according to the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points to obtain the attribute prediction information of the point to be decoded includes: Obtaining a first weighting coefficient for each of the k first prediction reference points, and obtaining a second weighting coefficient for each of the r second prediction reference points; Obtaining weight optimization coefficients of the r second prediction reference points; According to the weight optimization coefficient, optimizing the second weighting coefficient of each of the r second prediction reference points to obtain the second optimized weighting coefficient of each of the r second prediction reference points; According to the first weighting coefficients of the k first prediction reference points and the second optimized weighting coefficients of the r second prediction reference points, the attribute reconstruction information of the k first prediction reference points and the attribute reconstruction information of the r second prediction reference points are weightedly summed to obtain the attribute prediction information of the point to be decoded.
7. The method according to claim 6, It is characterized in that The obtaining of the weight optimization coefficients of the r second prediction reference points includes any one of the following: Performing numerical calculation on the total number of the second prediction reference points to obtain the weight optimization coefficient; Numerically calculating the total number of the second prediction reference points and the attribute quantization step length of the to-be-decoded point to obtain the weight optimization coefficient; According to the attribute coding condition satisfied by the to-be-decoded point, a weight optimization coefficient corresponding to the satisfied attribute coding condition is determined.
8. The method according to claim 1, It is characterized in that The point to be decoded belongs to a target point cloud group in the point cloud; and the attribute prediction of the point to be decoded is performed according to the distance between the point to be decoded and the prediction reference point to obtain the attribute prediction information of the point to be decoded, including: When the index value of the to-be-decoded point in the target point cloud group is greater than the upper limit of the length of the candidate point set, determining a default prediction value determination distance according to a geometric distance between the to-be-decoded point and a target prediction reference point in the prediction reference points; If the default prediction value determination distance is greater than a second distance threshold, obtaining attribute default prediction information; According to the attribute default prediction information, the attribute prediction of the to-be-decoded point is performed to obtain the attribute prediction information of the to-be-decoded point; Among them, the target prediction reference point is the p prediction reference points among the prediction reference points whose geometric distance to the point to be decoded is closest or farthest, and p is a positive integer; the default prediction value determination distance includes any one of the following: the maximum geometric distance, the minimum geometric distance or the mean geometric distance among the geometric distances between the point to be decoded and the p prediction reference points; the attribute default prediction information includes any one of the following: the fixed attribute default prediction information, the attribute default prediction information determined according to the attribute bit depth of the point to be decoded.
9. The method according to claim 8, It is characterized in that The points to be decoded in the point cloud are arranged in sequence according to the decoding order; the obtaining of attribute default prediction information includes: Acquire attribute determination information of a preceding point of the point to be decoded; the preceding point refers to a point in the point cloud that is located before the point to be decoded in decoding order; the attribute determination information includes a mean value of attribute reconstruction information or a mean value of attribute prediction information of the preceding point; The attribute default prediction information is determined according to the relationship between the attribute determination information and the attribute determination threshold.
10. The method according to claim 1, It is characterized in that The point to be decoded belongs to a target point cloud group in the point cloud; the method further comprises: When the index value of the to-be-decoded point in the target point cloud group is greater than the upper limit of the length of the candidate point set, the exchange determination distance is obtained; the exchange determination distance includes any one of the following: the geometric distance between the to-be-decoded point and the farthest predicted reference point among the predicted reference points, the geometric distance between the to-be-decoded point and the nearest predicted reference point among the predicted reference points, and the average of the geometric distances between the to-be-decoded point and the predicted reference points; If the exchange determination distance is greater than or equal to a third distance threshold, keeping the candidate point set unchanged; If the exchange determination distance is less than a third distance threshold, determining the farthest candidate point in the candidate point set that has the farthest geometric distance from the point to be decoded, and exchanging the reconstruction information of the farthest candidate point with the reconstruction information of the nth candidate point in the candidate point set; The attribute reconstruction information of the to-be-decoded point is updated to the candidate point set, and the updated candidate point set is applied to the attribute prediction process of subsequent to-be-decoded points of the to-be-decoded point.
11. The method according to claim 10, It is characterized in that The number of the farthest candidate points is multiple; the method further includes: Selecting a target farthest candidate point from among the farthest candidate points; The exchanging the reconstruction information of the farthest candidate point with the reconstruction information of the nth candidate point in the candidate point set includes: The reconstruction information of the farthest candidate point of the target is exchanged with the reconstruction information of the nth candidate point in the candidate point set.
12. The method according to claim 11, It is characterized in that The selecting the farthest candidate point of the target from the farthest candidate points comprises: Selecting the target farthest candidate point from among the farthest candidate points according to the order of the farthest candidate points in the candidate point set; Selecting the target farthest candidate point from among the farthest candidate points according to the search order of the farthest candidate points in the candidate point set; Selecting the target farthest candidate point from among the farthest candidate points according to the attribute distance between each of the farthest candidate points and the point to be decoded; Selecting the target farthest candidate point from among the farthest candidate points according to the attribute distance between each of the farthest candidate points and the preceding point of the point to be decoded; the preceding point refers to a point that precedes the point to be decoded in decoding order; The target farthest candidate point is selected from among the farthest candidate points according to the geometric distances between the farthest candidate points and the point to be decoded in the geometric space component.
13. The method according to claim 12, It is characterized in that The attribute coding mode of the point to be decoded is an attribute prediction mode; and selecting the target farthest candidate point from each of the farthest candidate points according to the attribute distance between each of the farthest candidate points and a preceding point of the point to be decoded comprises: Determining the attribute distance between the preceding point and the farthest candidate point according to the difference between the attribute reconstruction information of the preceding point and the attribute reconstruction information of the farthest candidate point; Among the farthest candidate points, the farthest candidate point with the largest or smallest attribute distance to the preceding point is selected as the target farthest candidate point.
14. The method according to claim 12, It is characterized in that The attribute coding mode of the point to be decoded is an attribute prediction transform coding mode, and the point to be decoded belongs to a target point cloud group in the point cloud; and selecting the target farthest candidate point from each of the farthest candidate points according to the attribute distance between each of the farthest candidate points and a preceding point of the point to be decoded comprises: Selecting a target preceding point among the preceding points of the point to be decoded; Determining the attribute distance between the target predecessor point and the farthest candidate point according to the difference between the attribute reconstruction information of the target predecessor point and the attribute reconstruction information of the farthest candidate point; Among the farthest candidate points, the farthest candidate point with the largest or smallest attribute distance to the target preceding point is selected as the target farthest candidate point.
15. The method of claim 14, It is characterized in that The selecting a target preamble point from the preamble point of the point to be decoded comprises: Select the first pre-sequence point or the last pre-sequence point in the preceding point cloud group of the target point cloud group as the target pre-sequence point; all the points to be decoded in the target point cloud group select the first pre-sequence point or the last pre-sequence point in the preceding point cloud group as the target pre-sequence point; According to the position of the to-be-decoded point in the target point cloud group, the target preceding point is selected from the preceding points of the to-be-decoded point; The preceding point cloud group is a point cloud group whose decoding order is before the target point cloud group.
16. The method of claim 15, It is characterized in that The selecting the target preceding point from the preceding points of the point to be decoded according to the position of the point to be decoded in the target point cloud group comprises: If the to-be-decoded point is the first to-be-decoded point in the target point cloud group, select the first pre-order point or the last pre-order point in the pre-order point cloud group of the target point cloud group as the target pre-order point; If the point to be decoded is not the first point to be decoded in the target point cloud group, select the preceding point that is arranged first in the target point cloud group, or the preceding point that is arranged one position before the point to be decoded in the decoding order in the target point cloud group, as the target preceding point.
17. The method of claim 11, It is characterized in that The point to be decoded has attribute information of multiple attribute types; the selecting of the target farthest candidate point from the farthest candidate points includes any one of the following: Selecting the target farthest candidate point under each attribute type from the farthest candidate points respectively; Selecting the target farthest candidate point under any of the following attribute types from among the farthest candidate points: a first-prediction attribute type, a post-prediction attribute type, and a target attribute type; the target attribute type is any one of the multiple attribute types; The target farthest candidate point is selected from the farthest candidate points based on attribute weighted distances of multiple attribute types or geometric attribute comprehensive distances.
18. The method of claim 1, It is characterized in that The to-be-decoded point belongs to a target point cloud group in the point cloud, and the target point cloud group is grouped according to the number of target right shift bits; the method further includes: According to the subdivision of the preceding point cloud grouping of the target point cloud grouping, the target right shift bit number is updated; the updated target right shift bit number is used as the grouping basis for the subsequent point cloud grouping of the target point cloud grouping; The preceding point cloud group is a point cloud group whose decoding order is before the target point cloud group, and the subsequent point cloud group is a point cloud group whose decoding order is after the target point cloud group.
19. The method of claim 18, It is characterized in that The updating of the target right shift bit number according to the subdivision of the preceding point cloud group of the target point cloud group includes any one of the following: If the preceding point cloud group is subdivided, the target right shift bit number is reduced, or when the target right shift bit number meets the update condition, the target right shift bit number is reduced; If the preceding point cloud group is not subdivided, the target right shift bit number is kept unchanged or increased; The updating method of the target number of right-shift bits is determined according to the number of the preceding point cloud groups subdivided in the preceding point cloud groups.
20. The method of claim 1, It is characterized in that The point to be decoded has attribute information of multiple attribute types; and determining the prediction reference point of the point to be decoded in the candidate point set of the point to be decoded includes: Obtaining a geometric distance between the to-be-decoded point and a candidate point in the candidate point set; Obtaining a decoded attribute distance between the point to be decoded and a candidate point in the candidate point set; Determine the comprehensive distance between the point to be decoded and the corresponding candidate point according to the geometric distance between the point to be decoded and the corresponding candidate point, and the decoded attribute distance between the point to be decoded and the corresponding candidate point; A prediction reference point of the point to be decoded is determined in the candidate point set according to a comprehensive distance between the point to be decoded and candidate points in the candidate point set.
21. The method of claim 20, It is characterized in that The decoded attribute distance between the point to be decoded and the candidate point is determined according to the difference between the attribute reconstruction information of the point to be decoded under the decoded attribute type and the attribute reconstruction information of the candidate point under the decoded attribute type; when the decoded attribute type is a color attribute type, the step of obtaining the decoded attribute distance between the point to be decoded and the candidate points in the candidate point set includes: Determining attribute reconstruction information selected by the to-be-decoded point and the candidate point respectively according to the color space; The decoded attribute distance is determined according to a difference between the attribute reconstruction information selected for the point to be decoded and the attribute reconstruction information selected for the candidate point.
22. The method of claim 21, It is characterized in that The step of determining attribute reconstruction information selected by the to-be-decoded point and the candidate point according to the color space includes: When the color space is a first color space, an attribute reconstruction component of the to-be-decoded point under a target color component in the first color space is used as the attribute reconstruction information selected for the to-be-decoded point, and an attribute reconstruction component of the candidate point under the target color component in the first color space is used as the attribute reconstruction information selected for the candidate point; When the color space is a second color space, the attribute reconstruction components of the point to be decoded under each color component of the second color space are used as the attribute reconstruction information selected for the point to be decoded, and the attribute reconstruction components of the alternative point under each color component of the second color space are used as the attribute reconstruction information selected for the alternative point.
23. The method of claim 21, It is characterized in that The comprehensive distance between the to-be-decoded point and the corresponding candidate point is obtained by summing the weighted result of the geometric distance between the to-be-decoded point and the corresponding candidate point and the decoded attribute distance between the to-be-decoded point and the corresponding candidate point; The process of determining the weighting factor of the attribute distance between the to-be-decoded point and the corresponding candidate point includes: Calculating the weighting factor according to the attribute quantization parameter; The attribute quantization parameter is the attribute quantization parameter corresponding to the decoded attribute type, or is the attribute quantization parameter corresponding to the attribute type to be decoded.
24. The method of claim 1, It is characterized in that After reconstructing the attributes of the to-be-decoded point according to the attribute prediction information of the to-be-decoded point to obtain the attribute reconstruction information of the to-be-decoded point, the method further includes: Obtaining a cutoff range of the attribute reconstruction information of the point to be decoded; The attribute reconstruction information of the to-be-decoded point is truncated according to the truncation range.
25. The method of claim 24, It is characterized in that The truncation range of obtaining the attribute reconstruction information of the to-be-decoded point includes any one of the following: Parsing the truncation range from the point cloud code stream, where the truncation range is calculated by the encoding device according to the attribute output bit depth; Obtaining the truncation range corresponding to the attribute type of the point to be decoded; The truncation range is determined according to a bit depth condition satisfied by the attribute output bit depth of the to-be-decoded point and an attribute type possessed by the to-be-decoded point.
26. A point cloud processing method, It is characterized in that include: Determine the points to be encoded in the point cloud; Determining a prediction reference point of the point to be encoded in a candidate point set of the point to be encoded; According to the distance between the to-be-encoded point and the prediction reference point, attribute prediction is performed on the to-be-encoded point to obtain attribute prediction information of the to-be-encoded point; Attribute encoding is performed on the point to be encoded according to the attribute prediction information of the point to be encoded.
27. A point cloud processing device, It is characterized in that include: A determination unit, used to determine the points to be decoded in the point cloud; A processing unit, configured to determine a prediction reference point of the point to be decoded from a candidate point set of the point to be decoded; The processing unit is further configured to perform attribute prediction on the point to be decoded according to the distance between the point to be decoded and the prediction reference point, so as to obtain attribute prediction information of the point to be decoded; The processing unit is further configured to reconstruct the attributes of the point to be decoded according to the attribute prediction information of the point to be decoded, so as to obtain the attribute reconstruction information of the point to be decoded.
28. A point cloud processing device, It is characterized in that include: A determination unit, used to determine the points to be encoded in the point cloud; A processing unit, configured to determine a prediction reference point of the point to be encoded from a candidate point set of the point to be encoded; The processing unit is further configured to perform attribute prediction on the point to be encoded according to the distance between the point to be encoded and the prediction reference point, so as to obtain attribute prediction information of the point to be encoded; The processing unit is further configured to perform attribute encoding on the point to be encoded according to the attribute prediction information of the point to be encoded.
29. A computer device, It is characterized in that The computer device comprises: a processor suitable for implementing a computer program; A computer-readable storage medium storing a computer program, wherein the computer program is suitable for being loaded by the processor and executing the point cloud processing method according to any one of claims 1 to 25, or executing the point cloud processing method according to claim 26.
30. A computer readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the point cloud processing method according to any one of claims 1 to 25, or executing the point cloud processing method according to any one of claim 26.
