A method for quality assessment of reduced reference point clouds based on support vector machines
By using a support vector machine-based approach, multiple features of point clouds are extracted and selected. By combining the Lasso predictor and the support vector machine model, the problem of inconsistency between point cloud quality assessment and human visual perception in existing technologies is solved, achieving more accurate point cloud quality prediction and cost savings.
Patent Information
- Application Number
- CN202210084819.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-25
- Publication Date
- 2025-11-14
- Estimated Expiration
- 2042-01-25
AI Technical Summary
Existing point cloud quality assessment methods ignore the influence of global and encoded information, resulting in a large difference between the predicted distorted point cloud quality and the quality perceived by the human visual system. Furthermore, existing technologies fail to effectively combine the objective geometric and color quality of point clouds, thus failing to accurately reflect the human visual experience.
A support vector machine-based approach is adopted to extract point cloud compressed features, geometric features, normal features, curvature features, and color features through point cloud feature extraction, feature selection, and training prediction processes. The most effective feature set is selected using the Lasso predictor and combined with the support vector machine model for quality evaluation.
It improves the accuracy of point cloud quality prediction, makes the point cloud communication system consistent with the human visual perception system, saves storage and time costs, and has strong generalization and model update capabilities.
Smart Images
Figure CN115294010B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing technology and relates to a method for evaluating the quality of reduced reference point clouds based on support vector machines. Background Technology
[0002] With the rapid development of 3D scanning technology and the increasing demand for 3D visual media, point cloud data is being used more and more widely in engineering and many aspects of daily life. However, the irregular and massive nature of point cloud data poses significant challenges to its transmission and storage. In practical point cloud communication systems, the acquired point cloud is first compressed into a bitstream by a point cloud encoder (e.g., a video-based point cloud compression encoder, V-PCC), then transmitted over the network to the decoding end for decoding. The reconstructed point cloud after encoding and decoding is then received by the human eye on the terminal display device. The subjective visual quality of the point cloud ultimately received by the human eye is affected by various factors. However, current point cloud distortion measurement techniques (pc_error) characterize the objective quality levels of geometry and color based on the geometric peak signal-to-noise ratio (PSNR) of point-to-point distortion (D1) or point-to-area distortion (D2) and the PSNR of the corresponding point color distortion, respectively. In existing quality assessment schemes, the objective geometric and color quality of point clouds are calculated independently. Furthermore, current quality assessment schemes only consider the information distortion factors of local point clouds, ignoring the influence of global and encoded information factors. This leads to a significant discrepancy between the distorted point cloud quality predicted by existing point cloud quality assessment algorithms and the perceived quality of the human visual system. Therefore, there is an urgent need to design a novel point cloud quality assessment method. Summary of the Invention
[0003] The purpose of this invention is to overcome the shortcomings of existing technologies and to design a reduced-reference point cloud quality assessment method based on support vector machines, thereby improving the accuracy of point cloud quality prediction, making the point cloud communication system consistent with the human visual perception system, and greatly saving storage and time costs.
[0004] To achieve the above objectives, this invention includes three processes: point cloud feature extraction, feature selection, and support vector machine training and prediction, specifically comprising the following steps:
[0005] (1) Point cloud feature extraction: Extract point cloud compression features, point cloud geometric features, point cloud normal features, curvature features and color features respectively;
[0006] (2) Feature selection: The Least Absolute Shrinkage and Selection Operator (Lasso) predictor is used to select the most effective features from the point cloud features extracted in step (1) to form a feature set;
[0007] (3) Support Vector Machine Training and Prediction: First, the point cloud data is divided into a training set and a test set, each accounting for 50%, and the point cloud content of the training set and the test set do not overlap; then, each point cloud in the training set is input into the feature vector corresponding to the feature set selected in step (2). The support vector machine model is obtained by combining the mean opinion score (MOS) of the labels; then, the feature vector of each point cloud in the test set is input. The prediction quality of point clouds in the test set is directly calculated using the trained support vector machine model.
[0008] When extracting point cloud compression features, this invention employs either video-based point cloud compression (V-PCC) or geometry-based point cloud compression (G-PCC) techniques to extract the geometric quantization parameters of the point cloud. ) and color quantization parameters ( Point cloud compression features are the most important feature.
[0009] The specific process of extracting point cloud geometric features in this invention is as follows: assuming Let N represent the geometric and color information of the i-th point in the point cloud P, and let N be the total number of points. Using the K-nearest neighbor method, select the set of the k nearest neighbors of the i-th point. For each point nearest neighbor set Midpoint, variance of relevant geometric and color information Median Mean absolute deviation absolute median and coefficient of variation The statistic is obtained using the following formula:
[0010] (1)
[0011] (2)
[0012] (3)
[0013] (4)
[0014] (5)
[0015] in This represents the value of the nth nearest neighbor of the i-th point. It represents all the nearest neighbors of the i-th point. The average value, It represents the nearest neighbor set of the i-th point. All points The value of the j-th point after sorting the values in ascending order. Represents the nearest neighbor set All K points Expected value; for geometric features, the value associated with all i-th points. This refers to based on geometric information The Euclidean distance of each point is calculated, and the statistical average of all points is used as the geometric feature value of the final point cloud P.
[0016] The point cloud normal feature described in this invention is obtained by taking the normal of each point from the original normal information acquired by the acquisition device or the calculated value based on the coordinate information of all the nearest K points according to an existing algorithm. The normal feature is obtained by calculating the angular similarity between the normal vector of the i-th point and each of the adjacent points to check the uniformity of the local surface shape. The angular similarity between the i-th point and its j-th nearest neighbor point is calculated as follows:
[0017] (6)
[0018] For the normal feature, the values associated with all i-th points refer to those based on geometric information. The similarity between each point and its nearest neighbors is calculated, and then the statistical value of each point is calculated according to formulas (1)-(5). , , , as well as Formulas (1)-(5) use The value is the angle similarity value calculated according to formula (6), and the average of the statistics of all points is used as the normal feature value of the final point cloud P.
[0019] The specific process for extracting curvature features in this invention is as follows: For each point nearest neighbor set All points in the region are represented by a local plane in the least-squares sense, i.e.
[0020] (7)
[0021] In the formula, For plane The normal vector, for The distance to the origin is calculated from this distance. The centroid passing through its K neighborhood points Then the covariance matrix M is,
[0022] (8)
[0023] Eigenvalue decomposition of the covariance matrix M yields E eigenvalues of the covariance matrix M. ), then point Surface curvature for:
[0024] (9)
[0025] Calculate each point Surface curvature Then, the statistical curvature-related features are used as the extracted curvature features for each point. nearest neighbor set Given all K nearest neighbors, calculate the standard deviation of the curvature of the current point and its nearest neighbors. ,Right now,
[0026] (10)
[0027] In the formula, Point nearest neighbor set The j-th nearest neighbor curvature, express nearest neighbor set The average curvature of all nearest neighbor points; then based on each point standard deviation of curvature Further calculate the maximum (Max), median (med), and standard deviation (TwiStd) of the standard deviation of curvature for all points in the point cloud.
[0028] (11)
[0029] In the formula, This indicates a standard deviation operation, ultimately using the standard deviation of curvature. , and Three statistical values are used as curvature features of the point cloud.
[0030] The process of extracting color features in this invention is as follows: First, according to the standard BT.709 formulated by the Radiocommunication Sector (ITU-R) of the International Telecommunication Union (ITU), each Color information Transform into Information, and then calculate color features according to formulas (10) and (11) for extracting curvature features, and use the formulas in the formulas. Use color brightness information Instead, we add the average value of the color measurement information to this, that is,
[0031] (12)
[0032] In the formula, This represents the operation of calculating the average. Point nearest neighbor set The j-th nearest neighbor Brightness information, express nearest neighbor set The average brightness of all nearest neighbor points.
[0033] The specific process of feature selection described in this invention is as follows: using the Least Absolute Shrinkage and Selection Operator (Lasso) predictor, the vector of linear regression coefficients is estimated by minimizing the sum of squared residuals. Minimizing the sum of squared residuals is constrained by the l1 norm of the coefficient vector.
[0034] (13)
[0035] In the formula, X represents the eigenvalue. This represents the coefficient corresponding to the feature.
[0036] Compared with existing technologies, this invention has the following advantages: First, it proposes for the first time a method for evaluating the quality of dereferenced point clouds in practical point cloud transmission systems, comprehensively considering various features affecting point cloud quality, such as compression features, geometric position features, normal vector features, curvature features, and color features. Second, it employs an effective Lasso predictor to further simplify numerous influencing features, greatly saving algorithm time and storage costs while ensuring accuracy, thus increasing its practicality. Third, this method has strong generalization ability and can automatically update the model continuously based on the expansion and optimization of the training set, more accurately predicting the quality of point clouds. Attached Figure Description
[0037] Figure 1 This is a schematic block diagram illustrating the process of reducing the quality of point clouds based on support vector machines in this invention.
[0038] Figure 2 shows the block diagram of an existing point cloud encoder, where (a) is the video-based point cloud compression (V-PCC) technology and (b) is the geometry-based point cloud compression (G-PCC) technology.
[0039] Figure 3 The diagram shows the result of feature selection in an embodiment of the present invention. Detailed Implementation
[0040] The present invention will be further described below with reference to the embodiments and accompanying drawings.
[0041] Example:
[0042] This embodiment includes three processes: point cloud feature extraction, feature selection, and support vector machine training and prediction. Figure 1 As shown, the specific steps include:
[0043] (1) Point cloud feature extraction:
[0044] (1-1) First, extract the point cloud compression features. Although a point cloud with sufficient density can represent objects in all directions, it also brings the challenge of storing a large amount of data. Currently, the Moving Pictures Expert Group (MPEG) has proposed two point cloud compression techniques: Video-based Point Cloud Compression (V-PCC), as shown in Figure 2(a), and Geometry-based Point Cloud Compression (G-PCC), as shown in Figure 2(b). This embodiment will describe V-PCC as an example. The main coding idea of V-PCC is to use existing video codecs to compress the geometric and color information of the point cloud. As shown in Figure 2(a), the three-dimensional point cloud (3DPC) is decomposed into multiple patches based on its geometric information. These patches then generate non-repeating projection patches on multiple orthogonal projection planes. All projection patches are arranged on two-dimensional video frames according to the principle of maximizing information redundancy. Based on this three-dimensional to two-dimensional projection relationship, corresponding geometric and color videos are generated. Then, existing video encoders, such as H.265 / High Efficiency Video Coding, are used. Video compression is performed using HEVC (High-Video Coding). During point cloud compression, the choice of quantization parameter (QP) has a significant impact on the reconstruction quality of the point cloud. Therefore, this embodiment extracts the geometric quantization parameters (QP) of the point cloud. ) and color quantization parameters ( () is the most important feature for point cloud compression;
[0045] (1-2) Extracting geometric features of point cloud: Assumption Let N represent the geometric and color information of the i-th point in the point cloud P, and let N be the total number of points. Using the K-nearest neighbor method, select the set of the k nearest neighbors of the i-th point. For each point nearest neighbor set Midpoint, variance of relevant geometric and color information Median Mean absolute deviation absolute median and coefficient of variation The statistic is obtained using the following formula:
[0046] (1)
[0047] (2)
[0048] (3)
[0049] (4)
[0050] (5)
[0051] in This represents the value of the nth nearest neighbor of the i-th point. It represents all the nearest neighbors of the i-th point. The average value, It represents the nearest neighbor set of the i-th point. All points The value of the j-th point after sorting the values in ascending order. Represents the nearest neighbor set All K points Expected value; for geometric features, the value associated with all i-th points. This refers to based on geometric information The Euclidean distance of each point is calculated, and the statistical average of all points is used as the point cloud geometric feature value of the final point cloud P.
[0052] (1-3) Extracting point cloud normal features: The normal of each point is obtained from the original normal information acquired by the acquisition device or calculated based on the coordinate information of all the nearest K points according to the existing algorithm. The normal feature is obtained by calculating the angular similarity between the normal vector of the i-th point and each of the adjacent points to check the uniformity of the local surface shape. The angular similarity between the i-th point and its j-th nearest neighbor is calculated as follows:
[0053] (6)
[0054] For the normal feature, the values associated with all i-th points refer to those based on geometric information. The similarity between each point and its nearest neighbors is calculated, and then the statistical value of each point is calculated according to formulas (1)-(5). , , , as well as Formulas (1)-(5) use The value is the angle similarity value calculated according to formula (6), and the average of the statistics of all points is used as the normal feature value of the final point cloud P.
[0055] (1-4) Curvature feature extraction: It is necessary to calculate the curvature information of each point. Refer to the NormalEstimation class for calculating point cloud normals in the Point Cloud Library (PCL) to calculate the curvature of each point. First, normal vector estimation is performed based on the local surface fitting method. When the point cloud is on a smooth sampling surface, the local neighborhood of any point can be fitted with a plane. Therefore, for each point nearest neighbor set All points in the region are represented by a local plane in the least-squares sense, i.e.
[0056] (7)
[0057] In the formula, For plane The normal vector, for The distance to the origin is calculated from this distance. The centroid passing through its K neighborhood points Then the covariance matrix M is,
[0058] (8)
[0059] Eigenvalue decomposition of the covariance matrix M yields E eigenvalues of the covariance matrix M. ), then point Surface curvature for:
[0060] (9)
[0061] Calculate each point Surface curvature Then, the statistical curvature-related features are used as the extracted curvature features for each point. nearest neighbor set Given all K nearest neighbors, calculate the standard deviation of the curvature of the current point and its nearest neighbors. ,Right now,
[0062] (10)
[0063] In the formula, Point nearest neighbor set The j-th nearest neighbor curvature, express nearest neighbor set The average curvature of all nearest neighbor points; then based on each point standard deviation of curvature Further calculate the maximum (Max), median (med), and standard deviation (TwiStd) of the standard deviation of curvature for all points in the point cloud.
[0064] (11)
[0065] In the formula, This indicates a standard deviation operation, ultimately using the standard deviation of curvature. , and Three statistical values are used as curvature features of the point cloud;
[0066] (1-5) Extracting color features: First, according to the standard BT.709 formulated by the Radiocommunication Sector (ITU-R) of the International Telecommunication Union (ITU), each Color information Transform into Information, and then calculate color features according to formulas (10) and (11) for extracting curvature features, and use the formulas in the formulas. Use color brightness information Instead, we add the average value of the color measurement information to this, that is,
[0067] (12)
[0068] In the formula, This represents the operation of calculating the average. Point nearest neighbor set The j-th nearest neighbor Brightness information, express nearest neighbor set The average brightness of all nearest neighbor points.
[0069] (2) Feature selection: In this embodiment, the feature selection problem is transformed into a sparse representation problem. The main purpose is to select the SN most effective features from the 19 extracted feature vectors. This embodiment uses the Least Absolute Shrinkage and Selection Operator (Lasso) predictor to represent the quality of the distorted point cloud with as few features as possible from a given overcomplete dictionary composed of all extracted features. The Lasso estimates the vector of linear regression coefficients by minimizing the sum of squared residuals, which is constrained by the l1 norm of the coefficient vector.
[0070] (13)
[0071] In the formula, X represents the eigenvalue. This represents the coefficients corresponding to the features; Lasso is a compression estimator that constructs a penalty function to obtain a more refined model, which compresses some coefficients and sets some coefficients to zero, thus retaining the advantage of subset shrinkage. It is a biased estimator for handling data with multicollinearity. Figure 3 This demonstrates that the Lasso predictor selected geometric and color quantization coefficient features. , ) and the average standard deviation of colors ( ) and the maximum standard deviation of color ( The most effective feature set SF was selected. , , );
[0072] (3) Training and prediction of Support Vector Machine (SVM):
[0073] (3-1) From 16×25=400 point cloud data, select 8×25=200 point clouds as the training set and the remaining 8×25=200 point clouds as the test set, and the point cloud content of the training set and the test set do not overlap.
[0074] (3-2) Input the feature vector for each point cloud in the training and test sets. The Mean Opinion Score (MOS) is calculated by first analyzing the feature vectors of 200 point clouds in the training set. With training labels by Input training set method:
[0075]
[0076] The SVM model is obtained through training; then the feature vectors of the remaining 200 point clouds are... Enter the test set in sequence:
[0077]
[0078] The trained SVM model is used to directly calculate the prediction quality of the point cloud in the test set. As described above, this embodiment repeated the experiment 1000 times. The mean, median, maximum, minimum, mode, and variance of the Pearson Linear Correlation Coefficient (PLCC), Spearman Rank-order Correlation Coefficient (SRCC), Kendall Rank-order Correlation Coefficient (KRCC), Mean Absolute Error (MAE), and Root Mean Square Error (RMSE) between the predicted MOS values and the actual MOS values are presented. The specific verification results are shown in the table below.
[0079] Table 1: Accuracy of 1000 repeated trials
[0080] average value median Maximum value Minimum value mode variance PLCC 0.89 0.90 0.94 0.80 0.90 0.00 SRCC 0.89 0.89 0.93 0.80 0.90 0.00 KRCC 0.70 0.71 0.77 0.59 0.72 0.00 MAE 7.75 7.65 10.56 5.70 7.49 0.50 RMSE 9.89 9.78 13.14 7.36 9.47 0.77
[0081] This embodiment proposes for the first time a highly practical point cloud quality assessment method based on support vector machines (SVM) in real-world point cloud transmission systems. The method first extracts all features that may affect point cloud quality, including 19 features such as compression features, geometric position features, normal vector features, curvature features, and color features. To reduce the time cost of point cloud quality assessment algorithms in real-world point cloud transmission systems, this embodiment uses a Lasso predictor to further simplify the extracted 19 features, selecting the four most effective features to provide feature vector information for the subsequent training and testing processes of the SVM. Four effective feature vectors are extracted from all point clouds in the training set, and their MOS values and feature vector information are written into the training set. The model is then trained using the SVM and the training set information. Finally, the trained model is used to test the feature vector information in the test set to predict the quality of the distorted point cloud.
Claims
1. A method for evaluating the quality of reduced reference point clouds based on support vector machines, characterized in that, Specifically, the following steps are included: (1) Point cloud feature extraction: Extract point cloud compression features, point cloud geometric features, point cloud normal features, curvature features and color features respectively; (2) Feature selection: The most effective features are selected from the point cloud features extracted in step (1) using the minimum absolute shrinkage and selection operator predictor to form a feature set; specifically: the minimum absolute shrinkage and selection operator predictor is used to estimate the vector of linear regression coefficients by minimizing the sum of squared residuals. Minimizing the sum of squared residuals is constrained by the l1 norm of the coefficient vector. (13) In the formula, X represents the eigenvalue. This represents the coefficient corresponding to the feature; (3) Support Vector Machine Training and Prediction: First, the point cloud data is divided into a training set and a test set, each accounting for 50%, and the point cloud content of the training set and the test set do not overlap; then, each point cloud in the training set is input into the feature vector corresponding to the feature set selected in step (2). The support vector machine model is obtained by averaging the subjective opinion scores of the labels; then, the feature vector of each point cloud in the test set is input. The prediction quality of point clouds in the test set is directly calculated using the trained support vector machine model.
2. The method for evaluating the quality of reduced reference point clouds based on support vector machines according to claim 1, characterized in that, When extracting point cloud compression features in step (1), the geometric quantization parameters and color quantization parameters of the point cloud are extracted as point cloud compression features using video-based point cloud compression technology or geometry-based point cloud compression technology.
3. The method for evaluating the quality of reduced reference point clouds based on support vector machines according to claim 1, characterized in that, The specific process of extracting point cloud geometric features in step (1) is as follows: Assume Let N represent the geometric and color information of the i-th point in the point cloud P, and let N be the total number of points. Using the K-nearest neighbor method, select the set of the k nearest neighbors of the i-th point. For each point nearest neighbor set Midpoint, variance of relevant geometric and color information Median Mean absolute deviation absolute median and coefficient of variation The statistic is obtained using the following formula: (1) (2) (3) (4) (5) in This represents the value of the nth nearest neighbor of the i-th point. It represents all the nearest neighbors of the i-th point. The average value, It represents the nearest neighbor set of the i-th point. All points The value of the j-th point after sorting the values in ascending order. Represents the nearest neighbor set All K points Expected value; for Geometric features, values associated with all i-th points This refers to based on geometric information The Euclidean distance of each point is calculated, and the statistical average of all points is used as the geometric feature value of the final point cloud P.
4. The method for evaluating the quality of reduced reference point clouds based on support vector machines according to claim 3, characterized in that, The point cloud normal feature mentioned in step (1) is obtained by taking the normal of each point from the original normal information obtained from the acquisition device or the calculated value based on the coordinate information of all the nearest K points according to the existing algorithm. The normal feature is obtained by calculating the angular similarity between the normal vector of the i-th point and each of the adjacent points to check the uniformity of the local surface shape. The angular similarity between the i-th point and its j-th nearest neighbor point is calculated as follows: (6) For the normal feature, the values associated with all i-th points refer to those based on geometric information. The similarity between each point and its nearest neighbors is calculated, and then the statistical value of each point is calculated according to formulas (1)-(5). , , , as well as Formulas (1)-(5) use The value is the angle similarity value calculated according to formula (6), and the average of the statistics of all points is used as the normal feature value of the final point cloud P.
5. The method for evaluating the quality of reduced reference point clouds based on support vector machines according to claim 4, characterized in that, The specific process of extracting curvature features in step (1) is as follows: For each point nearest neighbor set All points in the region are represented by a local plane in the least-squares sense, i.e. (7) In the formula, For plane The normal vector, for The distance to the origin is calculated from this distance. The centroid passing through its K neighborhood points Then the covariance matrix M is, (8) Eigenvalue decomposition of the covariance matrix M yields E eigenvalues of the covariance matrix M. , then point Surface curvature for: (9) Calculate each point Surface curvature Then, the statistical curvature-related features are used as the extracted curvature features for each point. nearest neighbor set Given all K nearest neighbors, calculate the standard deviation of the curvature of the current point and its nearest neighbors. ,Right now, (10) In the formula, Point nearest neighbor set The j-th nearest neighbor curvature, express nearest neighbor set The average curvature of all nearest neighbor points; then based on each point standard deviation of curvature Further calculations are performed on the maximum value (Max), median value (med), and standard deviation (TwiStd) of the curvature standard deviation for all points in the point cloud. (11) In the formula, This indicates a standard deviation operation, ultimately using the standard deviation of curvature. , and Three statistical values are used as curvature features of the point cloud.
6. The method for evaluating the quality of reduced reference point clouds based on support vector machines according to claim 5, characterized in that, Step (1) The process of extracting color features is as follows: First, according to the standard BT.709 formulated by the International Telecommunication Union Wireless Communications Sector, each Color information Transform into Information, and then calculate color features according to formulas (10) and (11) for extracting curvature features, and use the formulas in the formulas. Use color brightness information Instead, we add the average value of the color measurement information to this, that is, （12） In the formula, This represents the operation of calculating the average. Point nearest neighbor set The j-th nearest neighbor Brightness information, express nearest neighbor set The average brightness of all nearest neighbor points.