A Color Point Cloud Quality Evaluation Method Based on the Joint Perception of Geometry and Color

Abstract

The present invention relates to a method for evaluating the quality of colored point clouds based on combined geometric and color perception. Without the information of the original three-dimensional colored point cloud, potential geometric structures and color attribute information are provided during the visualization process of the three-dimensional colored point cloud to be evaluated. By utilizing the mutual influence between geometric distortion and color distortion, the combined geometric and color features for measuring the hybrid distortion, the color features for measuring color distortion, and the color features for measuring color distortion are obtained from the three-dimensional colored point cloud to be evaluated, which can comprehensively characterize the features of the distorted point cloud, thereby comprehensively evaluating the features of the distorted three-dimensional colored point cloud, and further obtaining a subjective perception score with a high similarity to the objective evaluation result.

Description

Technical Field

[0001] The present invention relates to the technical field of color point cloud reference - free quality evaluation, and more particularly, to a method for evaluating the quality of color point clouds based on combined geometric and color perception. Background Art

[0002] As a representation method of a new medium, 3D point clouds have advantages in terms of easy acquisition, accurate modeling, and realistic rendering, and can meet the multimedia applications in fields such as intelligent construction and cultural relic protection. However, in order to represent visual information with high fidelity, a model may have millions or even billions of points, and these unstructured 3D points also include coordinates and their associated attributes. For the huge data volume of point clouds, lossy compression schemes are usually adopted. Compared with lossless compression schemes, they can achieve greater data reduction, but at the cost of perceptual distortion. In addition, considering that the signal of point clouds may also be disturbed during the acquisition, transmission, or rendering process, resulting in fluctuations in the perception of the Human Visual System (HVS). Therefore, it is crucial to propose a high - performance color point cloud quality evaluation that conforms to human eye perception.

[0003] To quantify this visual perception mechanism, people often conduct research from the perspectives of subjective and objective quality evaluations; subjective quality evaluation relies on human opinions and provides real visual perception scores for different degrees of distortion; although these methods are accurate, they are both time - consuming and costly in terms of labor. Objective quality metrics refer to algorithms that calculate the visual quality of predicted distortion degrees, which can be roughly divided into methods based on the projection domain and methods based on the spatial domain. Among them, methods based on the projection domain can directly apply relatively mature traditional 2D image quality evaluation methods. For example: ① Project the 3D point cloud onto a 2D plane and use classic image objective quality evaluation metrics such as PSNR, Visual Information Fidelity (VIFP), Structural Similarity (SSIM), and Multi - Scale Structural Similarity (MS - SSIM) in the pixel domain to construct a point cloud quality evaluation method; ② Predict the visual quality by comparing the LPB statistical data of the reference and distorted point clouds. The advantage is that it is easy to operate and there are many image - processing methods, but the disadvantage is that it ignores the unique spatial structure of the point cloud. Merely applying 2D image quality evaluation methods to each projection image cannot effectively quantify the quality of 3D point clouds.

[0004] In the spatial domain-based metrics, technicians have proposed the following methods: ① A point-to-plane metric method is proposed on a point-to-point basis; ② Considering that the classical Hausdorff distance is very sensitive to outliers, an improved metric method based on the generalized Hausdorff distance and based on PSNR is proposed; ③ A method for capturing the degradation of distorted point clouds based on angular similarity, which is both simple and efficient; ④ Using curvature statistical features to estimate point cloud distortion and then extending it to color distortion metrics; ⑤ Using a linear combination of geometric and histogram color statistical features to predict point cloud quality; ⑥ Using voxelization as a preprocessing step to achieve different scaling effects, and then extracting statistical data dependent on position, normal, curvature, and brightness as features of PointSSIM; ⑦ Before extracting the features of each region, the point cloud is evenly divided; Geometric features include statistical moments applied to Euclidean distance, angular information, and local density, and these statistical moments are weighted according to the roughness of the region; Texture features rely on the same statistics in the HSV color space; ⑧ There is also a method based on graph signal processing, which evaluates the color gradient statistical moments of the key points of the reference point cloud and performs high-pass filtering identification on its topological structure. The above methods are all called full-reference quality assessment. As the name implies, because they have a strong demand for the original content during operation, however, full-reference is difficult to apply in actual systems.

[0005] Currently, the exploration of semi / reference-free quality assessment methods for colored point clouds is only preliminary. Therefore, there are few existing semi / reference-free quality assessment methods. The semi-reference objective quality metric PCM-RR proposed by technicians depends on global features extracted from position and color data; there are also technicians who propose to calculate the statistical information of the Euclidean distance between each sample and the arithmetic mean of the point cloud using geometric coordinates and color values, called VQA-CPC.

[0006] Full-reference / semi-reference point cloud quality assessment methods require information of the original point cloud model. In some cases where the original point cloud information cannot be obtained, full-reference / semi-reference point cloud quality assessment methods will not be applicable; Reference-free point cloud quality assessment methods do not require information of the original point cloud, but more attention needs to be paid to its prediction performance. The consistency between the objective evaluation results and the subjective perception quality of the existing reference-free point cloud quality assessment methods remains to be improved. Summary of the Invention

[0007] The problem solved by the present invention is how to make the objective evaluation result of the reference-free point cloud quality assessment method similar to the subjective perception quality.

[0008] To solve the above problems, the present invention provides

[0009] A method for evaluating the quality of colored point clouds based on joint geometric and color perception, comprising the following steps:

[0010] Step 1: Obtain the three-dimensional color point cloud to be evaluated, as well as its coordinate attributes and color attributes;

[0011] Step 2: Jointly use the coordinate attributes, the fast point feature histogram corresponding to the coordinate attributes, and the color space to perform supervoxel segmentation on the three-dimensional color point cloud into N sub-point clouds;

[0012] Step 3: Extract the combined geometric and color features for measuring the mixed distortion based on the sub-point clouds;

[0013] Step 4: Extract the geometric features for measuring the geometric distortion based on the coordinate attributes of the three-dimensional color point cloud;

[0014] Step 5: Extract the color features for measuring the color distortion based on the color attributes of the three-dimensional color point cloud;

[0015] Step 6: Combine the combined geometric and color features, geometric features, and color features into the perceptual feature vector of the three-dimensional color point cloud, and use the perceptual feature vector as the input of the random forest learning model, and pool to obtain the final objective evaluation score of the three-dimensional color point cloud.

[0016] The beneficial effects of the present invention are as follows: Without the information of the original three-dimensional color point cloud, the potential geometric structure and color attribute information are provided during the visualization process of the three-dimensional color point cloud to be evaluated. Utilizing the mutual influence between geometric distortion and color distortion, the combined geometric and color features for measuring the mixed distortion, the color features for measuring the color distortion, and the geometric features for measuring the color distortion from the three-dimensional color point cloud to be evaluated can comprehensively represent the features of the distorted point cloud, thereby comprehensively evaluating the features of the distorted three-dimensional color point cloud, and further obtaining a subjective perception score with a high similarity to the objective evaluation result.

[0017] Preferably, in Step 1, the three-dimensional color point cloud P(x, y, z, R, G, B) to be evaluated is obtained, and the coordinate attribute P G =(x, y, z) and the color attribute P C =(R, G, B) of the three-dimensional color point cloud are extracted.

[0018] Preferably, Step 2 specifically includes:

[0019] Step 201: Calculate the fast point feature histogram based on the information of the coordinate attribute P G =(x, y, z);

[0020] Step 202: Jointly use the coordinate attributes, the color space Lab corresponding to the coordinate attributes, and the fast point feature histogram to perform supervoxel segmentation on the three-dimensional color point cloud, and obtain N sub-point clouds after segmentation, where P part (i) is the i-th sub-point cloud, and i = 1, 2,..., N G ;

[0021] Supervoxel segmentation provides a more natural and compact representation for 3D color point clouds, enabling subsequent calculations to be performed on regions rather than on scattered points. The segmentation principle utilizes the local information of each point, including geometric coordinate attributes and color attributes, belonging to joint perception segmentation. Subsequently, geometric and color joint features can be effectively extracted from local regions.

[0022] Preferably, step 3 specifically includes:

[0023] Step 301: Based on P part (i) Construct a k-NN graph signal matrix G(f) of joint geometry and color, and perform multi-scale spectral graph wavelet transform on the graph signal f of the graph signal matrix G(f) to obtain spectral graph wavelet function coefficients W f (t, j) and scaling function coefficients t represents the scale defined in the spectral domain, and j represents the j-th point serial number in the i-th sub-point cloud;

[0024] Step 302: Adopt the union operation of spectral graph wavelet function coefficients and scaling function coefficients to obtain a joint pyramid matrix Pym. Perform singular value decomposition on the pyramid matrix Pym to obtain φ non-negative eigenvalues, and arrange them in descending order to get Ψ1 ≥ Ψ2 ≥ … ≥ Ψ φ , thereby obtaining the deep attribute information of the color point cloud. Take the first k largest eigenvalues as the eigen-joint features of the i-th sub-point cloud, and de-mean the eigen-joint features of the N G sub-point clouds to obtain the eigen features F JV ;

[0025] Step 303: Calculate the entropy feature F JE and local energy feature F JQ of the spectral graph wavelet function coefficients and scaling function coefficients at each scale;

[0026] Step 304: Based on F JV obtain the geometric and color joint feature F J of the 3D color point cloud, F J = [F JE , F JQ , F JV ;

[0027] Preferably, step 4 specifically includes:

[0028] Step 401: Based on the coordinate attribute P G = (x, y, z) construct a geometric coordinate difference matrix Σ i , and perform operations on the geometric coordinate difference matrix Σ iPerform the discrete KL transform to obtain the decomposed eigenvalues λ1≥λ2≥λ3, and based on calculate the geometric structure descriptor f GG , and based on the geometric structure descriptor f GG calculate the mean and variance as the geometric structure features F of the distorted point cloud GG ;

[0029] Step 402: Based on the coordinate attribute P G =(x, y, z), calculate the high-frequency components (A x , A y , A z ) after the multi-scale one-dimensional wavelet decomposition of the three coordinate axes; then calculate the entropy F GE and energy F GQ of each high-frequency component as the geometric detail features F GH , and combine the geometric structure features F GG to obtain the geometric features F G , F G =[F GG , F GE , F GQ .

[0030] Preferably, the specific steps of step 5 include:

[0031] Step 501: Convert the color attribute P C =(R, G, B) to the grayscale channel, and calculate the luminance segmentation space LM k (x′, y′, z′, R′, G′, B′) using the luminance contrast threshold, where k represents the k-th luminance segmentation space, k = 1, 2,..., N C ; Strengthen the correlation between color attributes through the luminance segmentation space, and make the color attributes of the point cloud more correlated through the regions after luminance segmentation, which is convenient for extracting color consistency features;

[0032] Step 502: Calculate the mutual information feature of the RGB color components as the color consistency feature F k based on the luminance segmentation space LM CC ;

[0033] Step 503: Based on the color attribute P C =(R, G, B), calculate the high-frequency components (A R , A G , A B ) after the multi-scale one-dimensional wavelet decomposition of the three color components; Use the asymmetric generalized Gaussian distribution for (A R , A G , A B)Perform fitting to obtain the parameter F of each color channel CA , the parameter F CA includes shape α, mean μ, left variance σ and right variance θ; based on the parameter F CA calculate the standard deviation F of the high-frequency component CB , energy F CQ , peak F CP and peak index F CI to form the color detail feature F for describing color details CH , F CH = [F CA , F CB , F CQ , F CP , F CI ; Combine the color consistency feature F CC to obtain the color feature F of the three-dimensional color point cloud C , F C = [F CA , F CB , F CQ , F CP , F CI , F CC .

[0034] Preferably, the step 6 specifically includes:

[0035] Step 601, combine the joint feature F of the geometry and color of the three-dimensional color point cloud J , geometric feature F G , color feature F C into the perception feature vector F of the point cloud P, F = [F J , F G , F C ;

[0036] Step 602, use the perception feature vector F as the input of the random forest learning model, and pool to obtain the final objective quality evaluation score Q of the distorted color point cloud predict . Description of the Drawings

[0037] Figure 1 This is the flowchart of the present invention. Detailed Embodiment

[0038] To make the above objects, features and advantages of the present invention more obvious and understandable, the following detailed description of the specific embodiments of the present invention will be given with reference to the drawings.

[0039] A method for evaluating the quality of a color point cloud based on joint perception of geometry and color includes the following steps:

[0040] Step 1: Obtain the three-dimensional color point cloud P(x, y, z, R, G, B) to be evaluated and its coordinate attributes P G =(x, y, z) and color attributes P C =(R, G, B). (x, y, z) is the geometric coordinate information of the three-dimensional color point cloud at the three-dimensional coordinate axes, with a total of N×3-dimensional data. (R, G, B) is the color information, with N×3-dimensional data. Therefore, the three-dimensional color point cloud data consists of N×6-dimensional data; in this specific embodiment, the three-dimensional color point cloud to be evaluated is the distorted point cloud;

[0041] Step 2: Jointly use the coordinate attributes, the Fast Point Feature Histogram (FPFH) corresponding to the coordinate attributes, and the color space to perform supervoxel segmentation on the three-dimensional color point cloud into N sub-point clouds;

[0042] Step 201: Consider the positional influence and normal relationship between all points within a local range to facilitate the joint attribute segmentation of the three-dimensional color point cloud. In this specific embodiment, based on the coordinate attribute P G =(x, y, z), calculate a 33-dimensional Fast Point Feature Histogram;

[0043] Step 202: Jointly use the 3-dimensional coordinate attributes, the 3-dimensional color space Lab corresponding to the coordinate attributes, and the Fast Point Feature Histogram to perform supervoxel segmentation on the three-dimensional color point cloud. A total of 39-dimensional features are used as the segmentation basis. Calculate the standardized distance D based on the above features. Those with a distance less than the threshold D are clustered together to obtain N sub-point clouds, P part (i) is the i-th sub-point cloud, i = 1, 2,..., N G ;

[0044] Step 3: Extract the geometric and color joint features for measuring the mixed distortion based on the sub-point clouds; specifically including:

[0045] Step 301: Based on P part (i), construct a joint geometric and color k-NN graph signal matrix G(f), and perform multi-scale spectral graph wavelet transform (SGWT) on the graph signal f of the graph signal matrix G(f) to obtain the spectral graph wavelet function coefficients W f (t, j) and the scaling function coefficients t represents the scale defined in the spectral domain, and j represents the j-th point sequence number in the i-th sub-point cloud; in this specific embodiment, four-scale spectral graph wavelet function coefficients W f (t1, j), W f (t2, j), W f (t3, j), W f (t4, j);

[0046] Step 302: By using the union operation of the spectrogram wavelet function coefficients and the scaling function coefficients, the joint pyramid matrix Pym is obtained. The pyramid matrix Pym is subjected to singular value decomposition (SVD) to obtain φ non-negative eigenvalues, which are sorted in descending order to obtain Ψ1≥Ψ2≥…≥Ψ φ , so as to obtain the deep attribute information of the color point cloud. The first k largest eigenvalues are taken as the eigen-joint features of the i-th sub-point cloud, and the N G eigen-joint features of the sub-point clouds are de-meaned to obtain the eigen-feature F JV ; In this specific embodiment, the first 3 largest eigenvalues are selected as the 3-dimensional eigen-feature F JV ;

[0047] Step 303: Calculate the entropy feature F JE and the local energy feature F JQ of the spectrogram wavelet function coefficients and the scaling function coefficients at each scale; In this specific embodiment, the entropy of the coefficients at each scale is F JE (i):

[0048]

[0049] In the formula, is the entropy feature of the first scale of the spectrogram wavelet coefficients; is the entropy feature of the scaling function coefficients;

[0050] Similarly, the local energy feature of the coefficients at each scale is F JQ (i):

[0051]

[0052] In the formula, is the local energy feature of the first scale of the spectrogram wavelet coefficients; is the local energy feature of the scaling function coefficients;

[0053] By averaging the entropy features and the local energy features of the point sets in different segmentation regions, the entropy feature F JE and the local energy feature F JQ at different scales are obtained;

[0054] Step 304: Based on F JV obtain the geometric and color joint feature F J of the three-dimensional color point cloud, F J = [F JE , F JQ , F JV ;

[0055] Step 4: Based on the coordinate attributes of the three-dimensional color point cloud, extract geometric features for measuring geometric distortion, specifically including:

[0056] Step 401: Based on the coordinate attribute P G =(x, y, z), construct the geometric coordinate difference matrix Σ i ; In this specific embodiment, r-search is selected to calculate the adjacent area around the i-th point sample, r = 25; then, for each point and its neighborhood points in area i, construct the overall divergence matrix Σ according to the coordinate difference i ; Then, perform discrete KL transform on the geometric coordinate difference matrix Σ i to obtain the decomposed eigenvalues λ1≥λ2≥λ3. Based on calculate the geometric structure descriptor f GG , and use the three reduced eigenvalues to construct a geometric structure descriptor f that describes the diffusion characteristics of the lower surface of all main axes GG . Based on the geometric structure descriptor f GG calculate the mean and variance as the geometric structure features F of the distorted point cloud GG ;

[0057] Step 402: Based on the coordinate attribute P G =(x, y, z), calculate the high-frequency components (A x , A y , A z ) after multi-scale one-dimensional wavelet decomposition of the three coordinate axes; then calculate the entropy F GE and energy F GQ of each high-frequency component as the geometric detail features F GH . Combine the geometric structure features F GG to obtain the geometric features F of the three-dimensional color point cloud G , F G =[F GG , F GE , F GQ ;

[0058] Step 5: Based on the color attributes of the three-dimensional color point cloud, extract color features for measuring color distortion; specifically including:

[0059] Step 501: Convert the color attribute P C =(R, G, B) to the grayscale channel. Specifically, convert the RGB three-channel color components to the point cloud grayscale value according to the ratio of converting the 2D image to the grayscale channel, and then calculate the luminance segmentation space LM k (x′, y′, z′, R′, G′, B′), where k represents the k-th luminance segmentation space, k = 1, 2,..., N C; Strengthen the correlation between color attributes by dividing the space by brightness. The regions after brightness division make the color attributes of the point cloud more correlated, facilitating the extraction of color consistency features;

[0060] Step 502: Based on the brightness division space LM k (x′, y′, z′, R′, G′, B′) Calculate the mutual information feature of the RGB color components as the color consistency feature F CC ; Specifically, for each set of points with equal brightness values, there is a correlation between different color channels. For example, taking R and G as an example, first obtain the joint probability distribution function ρ(X R , X G ), then calculate the one-dimensional entropy between independent channels and the two-dimensional entropy between correlated channels. According to the mutual information (MI) formula, the calculated result is the color feature of the k-th brightness division space, denoted as F CC (k). Finally, average the color features of k brightness division spaces to obtain the color consistency feature F CC ;

[0061] Step 503: Based on the color attribute P C =(R, G, B), calculate the high-frequency components (A R , A G , A B ) after multi-scale one-dimensional wavelet decomposition of the three color components; Use the Asymmetric Generalized Gaussian Distribution (AGGD) to fit (A R , A G , A B ) to obtain the parameters F CA of each color channel. The parameter F CA includes the shape α, the mean μ, the left variance σ, and the right variance θ; Calculate the standard deviation F CA , the energy F CB , the peak value F CQ , and the peak index F CP composed of the color detail features F CI for describing color details, F CH = [F CH , F CA , F CB , F CQ , F CP , F CI ; Combine the color consistency feature F CC to obtain the color feature F C of the three-dimensional color point cloud, F C = [FCA , F CB , F CQ , F CP , F CI , F CC

[0062] Step 6. Combine the combined geometric and color features, geometric features, and color features into the perceptual feature vector of the 3D color point cloud, and use the perceptual feature vector as the input of the random forest learning model, and pool to obtain the final objective evaluation score of the 3D color point cloud, specifically including:

[0063] Step 601. Combine the combined geometric and color feature F J of the 3D color point cloud, the geometric feature F G , and the color feature F C into the perceptual feature vector F of the point cloud P, F = [F J , F G , F C ; in this specific embodiment, the dimension of F is 48;

[0064] Step 602. Use the perceptual feature vector F as the input of the random forest learning model, and pool to obtain the final objective quality evaluation score Q of the distorted color point cloud predict ; in this specific embodiment, the larger Q predict , the better the quality of the color point cloud corresponding to the input F; on the contrary, it means that the quality of the color point cloud corresponding to the input F is worse.

[0065] Experimental results show that:

[0066] ​First, the proposed method is tested on three publicly available subjective evaluation databases of color point clouds. Meanwhile, the results obtained by the present invention are compared with those of existing point cloud quality assessment algorithms. The three publicly available 3D color point cloud subjective evaluation databases used for the comparative experiment are the CPCD2.0 database of Ningbo University (He Z, Jiang G, Jiang Z, et al. Towards a colored point cloud quality assessment method using colored texture and curvature projection[C] / / 2021 IEEE International Conference on Image Processing(ICIP). IEEE, 2021:1444-1448.), the SJTU-PCQA database of Shanghai Jiao Tong University (Yang Q, Chen H, Ma Z, et al. Predicting the perceptual quality of point cloud: A 3d-to-2d projection-based exploration[J]. IEEE Transactions on Multimedia, 2020.) and the IRPC database (Javaheri A, Brites C, Pereira F M B, et al. Point cloud rendering after coding: Impacts on subjective and objective quality[J]. IEEE Transactions on Multimedia, 2020.). Among them, the CPCD2.0 database contains 10 original point cloud models and 360 distorted point cloud models. There are 4 types of distortions, including the latest point cloud coding standards proposed by the Moving Picture Experts Group MPEG (i.e., G-PCC-Octree, G-PCC-Trisoup, and V-PCC) and Gaussian noise. These 4 types of distortions are divided into 9 different distortion levels. Therefore, one original model corresponds to 36 distorted versions, totaling 360 distorted point cloud models. The database also provides the Mean Opinion Score (MOS). The SJTU-PCQA database contains 10 original color three-dimensional point clouds and 420 distorted color three-dimensional point clouds. One original color three-dimensional point cloud and its 42 distorted color three-dimensional point clouds are respectively used to train the subjects before subjective scoring. The types of distortions include octree compression, color noise, geometric noise, and scaling, without encoding distortion.The IRPC database contains a total of 6 original color three-dimensional point clouds and 54 distorted color three-dimensional point clouds. The library is compressed using three different codecs at three different rates, representing low, medium, and high quality respectively; the distortion types adopt the compression schemes of PCL, G-PCC-Octree, and V-PCC point cloud codecs, and the database provides MOS values.

[0067] Table 1 Comparison of various items of three color point cloud subjective databases

[0068] Database Name Sequence Set Time Quantity Distortion Type CPCD2.0 Portrait 2021 360 G-PCC, V-PCC, Gaussian Noise SJTU-PCQA Object, Portrait 2020 420 Octree Compression, Color Noise, Geometric Noise, Scaling IRPC Object, Portrait 2020 54 PCL, G-PCC, V-PCC

[0069] The verification of the algorithm performance is carried out by comparing MOS with the score Q predicted by the point cloud quality evaluation method. predict Specifically, the features extracted by the objective quality evaluation method are first mapped into the predicted scores of the method through machine learning, and then the predicted score Q predict is non-linearly fitted with the subjective MOS value.

[0070] In terms of the comparison evaluation indicators, three standard indicators provided by the Video Quality Experts Group (VQEG) are adopted in the experimental stage to compare the evaluation performance of the objective evaluation methods for point cloud quality. The three general indicators are the Spearman rank correlation coefficient (SROCC), the Pearson linear correlation coefficient (PLCC), and the root mean square error (RMSE). Among them, SROCC is used to measure the prediction monotonicity, and PLCC and RMSE are used to measure the accuracy of the method. The value range of SROCC is between [-1, 1], and the value range of PLCC is between [0, 1]. The closer SROCC and PLCC are to 1, and the closer RMSE is to 0, the better the performance of this objective quality evaluation method.

[0071] The method of the present invention will be compared with seven full-reference image quality evaluation methods in the spatial domain, including: P2pHausdorff, P2pMSE, AS MEAN AS RMS AS MSE PC-MSDM and PCQM; four full-reference image quality evaluation methods in the projection domain, including PSNR projection SSIM projection MS-SSIM projection VIFP projection and a no-reference color point cloud quality evaluation method BQE-CVP.

[0072] Table 2 presents the values of three prediction performance metrics, SROCC, PLCC, and RMSE, of the method of the present invention, eleven full-reference objective quality assessment methods, and one no-reference objective quality assessment method on three color point cloud databases. In Table 2, the objective quality assessment method with the best performance is marked in bold.

[0073] Table 2 Results of three performance metrics, SROCC, PLCC, and RMSE, of the method of the present invention, eleven full-reference objective quality assessment methods, and one no-reference objective quality assessment method on three color point cloud databases

[0074]

[0075]

[0076] As can be seen from Table 2, the method of the present invention has good prediction performance in all three color point cloud subjective databases, which indicates that the method of the present invention has good robustness. It is worth mentioning that the method of the present invention still has an advantage compared with the full-reference methods.

[0077] Although the present disclosure is disclosed as above, the protection scope of the present disclosure is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present disclosure, and these changes and modifications will all fall within the protection scope of the present invention.

Claims

1. A method for evaluating the quality of colored point clouds based on the combined perception of geometry and color, characterized in that, The method includes the following steps: Step 1: Obtain the three-dimensional color point cloud to be evaluated and its coordinate attributes and color attributes. Obtain the three-dimensional color point cloud P(x, y, z, R, G, B) to be evaluated, and extract the coordinate attribute P G =(x, y, z) and the color attribute P C =(R, G, B); Step 2: Supervoxel segment the three-dimensional color point cloud into N sub-point clouds by combining the coordinate attributes, the fast point feature histogram corresponding to the coordinate attributes, and the color space. Specifically, it includes: Step 201, calculate a fast point feature histogram based on the information of the coordinate attribute P G =(x, y, z); Step 202: Perform supervoxel segmentation on the three-dimensional color point cloud by combining the coordinate attributes, the corresponding color space Lab of the coordinate attributes, and the fast point feature histogram, and obtain N sub-point clouds, where P part (i) is the i-th sub-point cloud, i = 1, 2, …, N G ; Step 3: Extract the combined geometric and color features for measuring the hybrid distortion based on the sub-point clouds. Specifically, it includes: Step 301. Based on P part (i) Construct a k-NN graph signal matrix G(f) that combines geometry and color, and perform multi-scale spectrogram wavelet transform on the graph signal f of the graph signal matrix G(f) to obtain spectrogram wavelet function coefficients W f (t, j) and scaling function coefficients where t represents the scale defined in the spectral domain, and j represents the j-th point serial number in the i-th sub-point cloud; Step 302: By using the union operation of the spectral graph wavelet function coefficients and the scaling function coefficients, the joint pyramid matrix Pym is obtained. The pyramid matrix Pym is subjected to singular value decomposition to obtain φ non - negative eigenvalues, which are arranged in descending order to get Ψ1≥Ψ2≥…≥Ψ φ , and the first k largest eigenvalues are taken as the eigen - joint features of the i - th sub - point cloud. The eigen - joint features of N G sub - point clouds are averaged to obtain the eigen - feature F JV ; Step 303: Calculate the entropy feature F of the wavelet function coefficients and scaling function coefficients of the spectrogram at each scale JE and the local energy feature F JQ ; Step 304: Based on F JV Obtain the combined geometric and color features F of the three-dimensional color point cloud J , F J = [F JE , F JQ , F JV ; Step 4: Extract the geometric features for measuring the geometric distortion based on the coordinate attributes of the three-dimensional color point cloud. Specifically, it includes: Step 401: Based on the coordinate attribute P G =(x, y, z), construct the geometric coordinate difference matrix ∑i, and perform discrete KL transform on the geometric coordinate difference matrix ∑i to obtain the decomposed eigenvalues λ1≥λ2≥λ3. Based on calculate the geometric structure descriptor f GG , and based on the geometric structure descriptor f GG calculate the mean and variance as the geometric structure feature F of the distorted point cloud GG ; Step 402: Based on the coordinate attribute P G =(x, y, z), calculate the high-frequency components (A x , A y , A z ) after multi-scale one-dimensional wavelet decomposition of the three coordinate axes; then calculate the entropy F GE and energy F GQ of each high-frequency component as the geometric detail feature F GH , and combine with the geometric structure feature F GG to obtain the geometric feature F G of the three-dimensional color point cloud, F G = [F GG , F GE , F GQ ; Step 5: Extract the color features for measuring the color distortion based on the color attributes of the three-dimensional color point cloud. Specifically, it includes: Step 501: Convert the color attribute P C =(R, G, B) to the grayscale channel, and calculate the luminance segmentation space LM k (x', y', z', R', G', B') using the luminance contrast threshold, where k represents the k-th luminance segmentation space, k = 1, 2,..., N C ; Step 502, spatially segment based on luminance LM k (x', y', z', R', G', B') Calculate the mutual information feature of the RGB color components as the color consistency feature F CC ; Step 503: Based on the color attribute P C =(R, G, B), calculate the high-frequency components (A R , A G , A B ) after multi-scale one-dimensional wavelet decomposition of the three color components; Use the asymmetric generalized Gaussian distribution to fit (A R , A G , A B ) to obtain the parameters F CA of each color channel. The parameter F CA includes the shape α, the mean μ, the left variance σ, and the right variance θ; Calculate the standard deviation F CB , the energy F CQ , the peak value F CP , and the peak index F CI to obtain the color detail features F CH for describing the color details, F CH = [F CA , F CB , F CQ , F CP , F CI ; Combine the color consistency features F CC to obtain the color features F C of the three-dimensional color point cloud, F C = [F CA , F CB , F CQ , F CP , F CI , F CC ; Step 6: Combine the combined geometric and color features, the geometric features, and the color features into the perceptual feature vector of the three-dimensional color point cloud, and use the perceptual feature vector as the input of the random forest learning model, and pool to obtain the final objective evaluation score of the three-dimensional color point cloud.

2. The color point cloud quality evaluation method based on combined geometric and color perception according to claim 1, wherein Specifically, Step 6 includes: Step 601: Combine the joint feature F of the geometry and color of the three-dimensional color point cloud J , the geometric feature F G , and the color feature F C into the perceptual feature vector F of the point cloud P, F = [F J , F G , F C ; Step 602: Use the perceptual feature vector F as the input of the random forest learning model, and perform pooling to obtain the final objective quality evaluation score Q of the distorted color point cloud predict .

Images