A point cloud quality assessment method based on information weighted projection

Abstract

This invention belongs to the field of 3D point cloud quality assessment technology, and relates to a point cloud quality assessment method based on information-weighted projection. First, the point cloud coordinates are aligned with the origin. Then, the point cloud is rotated to obtain multiple viewpoints. Next, the point cloud coordinates are scaled to ensure that the 2D projection of the reference point cloud preserves details while maintaining a seamless appearance. Orthogonal projection and rasterization are then used to obtain the projected image. The above operations are performed on both the original and distorted point clouds to obtain projected images for each viewpoint. Paired images are then fed into the IW-SSIM algorithm to calculate the objective score of the distorted point cloud at each viewpoint. Finally, pooling the scores of each viewpoint yields the final objective score. The method is simple; point cloud quality can be assessed using projection and the existing image quality assessment method IW-SSIM. Information-weighted projection can better remove the influence of background areas on point cloud quality, more accurately assess the quality score of distorted point clouds, and achieve better consistency between objective and subjective quality scores.

Description

Technical fields:

[0001] This invention belongs to the field of 3D point cloud quality assessment technology, and relates to a point cloud quality assessment method based on information-weighted projection. Background technology:

[0002] 3D point clouds are collections of points used to represent 3D shapes, objects, and indoor / outdoor environments. Each point has three-dimensional coordinates and attribute information, such as color, reflection intensity, transparency, and normal vectors. In recent years, 3D point clouds have been widely used in immersive media communication as a new media format. Compared to traditional voice and video communication, immersive media communication can greatly improve the user experience, bring people closer together, and change their lifestyles. Internationally, MPEG and domestically, the AVS working groups have established dedicated teams to standardize point cloud compression, and Point Cloud Quality Assessment (PCQA) has also attracted the attention of VQEG.

[0003] Every step in point cloud processing, such as acquisition, compression, transmission, storage, and rendering, can introduce distortion and degrade point cloud quality. Therefore, effectively evaluating point cloud quality has become a pressing issue in the field. Point cloud quality can serve as a standard for comparing the merits of various point cloud processing algorithms, or as feedback to further adjust relevant algorithm parameters, achieving the goals of design, evaluation, and optimization.

[0004] Currently, in most 3D point cloud applications, the human eye is the final recipient of the point cloud image. Therefore, subjective evaluation is considered the most direct and accurate evaluation method. However, subjective quality evaluation is costly and lacks real-time performance. Therefore, objective quality evaluation methods are crucial. Based on whether the original reference point cloud is needed, objective point cloud quality evaluation can be divided into three modes: full-reference, partial-reference, and no-reference. Most existing algorithms are full-reference models. Based on whether the image is projected onto a 2D plane, algorithms are further divided into point-based algorithms and projection-based algorithms. Point-based algorithms are further divided into geometric distortion evaluation algorithms and "geometric + color" distortion evaluation algorithms. Since what the human eye sees is what it gets, projection-based algorithms are more consistent with the characteristics of the human visual system. However, existing projection-based algorithms still have the following shortcomings: background pixels are also involved in the calculation after projection, making it impossible to remove the influence of the background region on the algorithm. Summary of the Invention:

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies and to design a point cloud quality assessment method based on information-weighted projection. This method considers information weights to ensure good consistency between objective quality predictions and subjective quality scores, thereby guiding various processing steps such as 3D point cloud encoding and transmission.

[0006] To achieve the above objectives, the point cloud quality assessment method based on information-weighted projection described in this invention includes the following steps:

[0007] S1. Generate various distortion types from the original point cloud, select different combinations of geometric distortion and texture distortion to obtain a 3D point cloud quality assessment database, and conduct subjective assessment experiments to obtain the subjective quality score of each distorted point cloud.

[0008] S2. Align the center of the point cloud with the origin of the coordinate system to obtain the point cloud coordinates;

[0009] S3. Rotate the point cloud using a rotation vector to obtain multiple viewpoints and get the coordinates of the rotated point cloud;

[0010] S4. The scaled and rotated point cloud coordinates make the 2D projection of the original point cloud as seamless as possible while preserving details.

[0011] S5. Use orthogonal projection and rasterization to obtain the projected image from the point cloud coordinates obtained in S4.

[0012] S6. Repeat steps S2-S5 to obtain the projection images of the original point cloud and the distorted point cloud at each viewpoint;

[0013] S7: Input the paired images from each viewpoint into the IW-SSIM algorithm to obtain the objective score for each viewpoint and pool them to obtain the final objective score for the distorted point cloud, thus completing the point cloud quality assessment.

[0014] The operation of aligning the point cloud center to the origin in S2 of this invention is as follows:

[0015] g t =gt r

[0016] Where g t Let p = (gc) be the 3D coordinates of point p after transformation, g be the 3D coordinates of point p before transformation, c be the attribute color information, and t be the 3D coordinates of point p before transformation. r The transformation vector is defined by the geometric center coordinates of the corresponding reference point cloud.

[0017] The rotation vector rθ described in S3 of this invention is:

[0018]

[0019] θ = arccos(n) v ·n z )

[0020] Where n v Let n be the unit normal vector of point p. z For (0,0,1), ||n v ×n z|| represents the l2 norm of the orientation quantity, r is the rotation axis, and θ is the rotation axis angle. The rotation matrix R is obtained from r and θ, and then the coordinates of point p after rotation are obtained:

[0021] g r =g t R.

[0022] The point cloud coordinates obtained by S4 in this invention are: g s =s·g r , where g s This represents the scaled coordinates, where s is the scaling factor.

[0023] The specific process of using orthogonal projection on the point cloud coordinates obtained in S4 in S5 of this invention is as follows:

[0024]

[0025] in Let p be the projected coordinates. If there are identical points located at the same coordinates, the maximum value of the z component of g is taken; during rasterization, the attribute color information c is placed in its corresponding projection position, and the empty position c is (127,127,127), and finally the projected image I is obtained.

[0026] In step S6 of this invention, the above steps are repeated to obtain projection images I of the original point cloud and the distorted point cloud at each viewpoint. ref (n) and I dis (n).

[0027] The objective score of the distorted point cloud obtained in S7 of this invention is:

[0028]

[0029] Where I ref (n),I dis (n) represent the projection images of the original point cloud and the distorted point cloud at each viewpoint, respectively. v Number of viewpoints.

[0030] This invention first aligns the point cloud coordinates with the origin, then rotates the point cloud to obtain multiple viewpoints, and then scales the point cloud coordinates to make the 2D projection of the reference point cloud as seamless as possible while preserving details. Then, orthogonal projection and rasterization are used to obtain the projected image. The above operations are performed on the original point cloud and the distorted point cloud respectively to obtain the projected image of each viewpoint. The paired images are then fed into the IW-SSIM algorithm to calculate the objective score of the distorted point cloud at each viewpoint. Finally, pooling the scores of each viewpoint yields the final objective score.

[0031] Compared with existing technologies, this invention has the following advantages: First, it can accurately evaluate the quality of 3D point clouds, thereby guiding the processing of 3D point clouds and facilitating the improvement of the performance of subsequent applications; second, the method is simple, and the point cloud quality can be evaluated using projection and the existing image quality evaluation method IW-SSIM; third, information-weighted projection can better remove the influence of background areas on point cloud quality, more accurately evaluate the quality score of distorted point clouds, and make the objective quality score and subjective quality score have better consistency. Attached image description:

[0032] Figure 1 This is a flowchart of the point cloud quality assessment based on information-weighted projection as described in this invention.

[0033] Figure 2 The test set of this invention includes the original point cloud and the partially distorted point cloud, where (a), (c), (e), and (g) are the original point clouds cake, cauliflower, flower_pot, and glasses_case, respectively; (b), (d), (f), and (h) are the point clouds with different V-PCC encoding parameters corresponding to the above original point clouds: (b) GQP=38, TQP=44, MOS=64.02; (d) GQP=50, TQP=26, MOS=36.30; (f) GQP=38, TQP=38, MOS=65.28; (h) GQP=26, TQP=26, MOS=89.81. Detailed implementation method:

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Example:

[0036] The point cloud quality assessment method based on information-weighted projection described in this embodiment is as follows: Figure 1 As shown, it includes the following steps:

[0037] S1. Generate various distortion types from the original point cloud, select different combinations of geometric distortion and texture distortion to obtain a 3D point cloud quality assessment database, and conduct subjective assessment experiments to obtain the subjective quality score of each distorted point cloud.

[0038] S2. Align the center of the point cloud with the origin to obtain the point cloud coordinates g. t =gt r ,

[0039] Where g t Let p = (gc) be the 3D coordinates of point p after transformation, g be the 3D coordinates of point p before transformation, c be the attribute color information, and t be the 3D coordinates of point p before transformation. r The transformation vector is defined by the geometric center coordinates of the corresponding reference point cloud.

[0040] S3. Rotate the point cloud using a rotation vector to obtain multiple viewpoints, and then obtain the coordinates of the rotated point cloud; the rotation vector rθ is:

[0041]

[0042] θ = arccos(n) v ·n z )

[0043] Where n v Let n be the unit normal vector of point p. z For (0,0,1), ||n v ×n z || represents the l2 norm of the orientation quantity, r is the rotation axis, and θ is the rotation axis angle. The rotation matrix R is obtained from r and θ, and then the coordinates of point p after rotation are obtained:

[0044] g r =g t R;

[0045] S4. Obtain the scaled point cloud coordinates g after scaling and rotating. s =s·g r , where g s The coordinates are scaled, and s is the scaling factor, which makes the 2D projection of the original point cloud as seamless as possible while preserving details.

[0046] S5. First, apply orthogonal projection to the point cloud coordinates obtained in S4. in Let p be the projected coordinates. If there are identical points located at the same coordinates, the maximum value of the z component of g is taken; then rasterization is performed, the attribute color information c is placed into its corresponding projection position, and the empty position c is (127,127,127), and finally the projection image I is obtained.

[0047] S6. Repeat S2-S5 to obtain the projection images of the original point cloud and the distorted point cloud at each viewpoint. ref (n) and I dis (n);

[0048] S7: Input the paired images from each viewpoint into the IW-SSIM algorithm to obtain the objective score for each viewpoint and pool it to obtain the final objective score for the distorted point cloud, thus completing the point cloud quality assessment. The obtained objective score for the distorted point cloud is:

[0049]

[0050] Where I ref (n),I dis (n) represent the projection images of the original point cloud and the distorted point cloud at each viewpoint, respectively. v Number of viewpoints.

[0051] This embodiment evaluates the performance of the objective quality assessment model. For the cake, cauliflower, flower_pot, and glasses_case cases, the following metrics are used to calculate the consistency between objective and subjective quality scores: Spearman Rank-order Correlation Coefficient (SRCC), Pearson Linear Correlation Coefficient (PLCC), and Root Mean Square Error (RMSE). The results are as follows: Figure 2 As shown in Table 1, Table 1 fully illustrates the consistency between objective and subjective scores obtained for test point clouds using the quality assessment model of this embodiment.

[0052] Table 1: Performance Demonstration of the Quality Assessment Model in This Embodiment

[0053] Point cloud SRCC PLCC RMSE cake 0.6598 0.6743 16.62 cauliflower 0.8125 0.8578 11.52 flower_pot 0.9211 0.9368 8.322 glasses_case 0.7845 0.8077 13.34

[0054] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A point cloud quality assessment method based on information-weighted projection, characterized in that, Includes the following steps: S1. Generate various distortion types from the original point cloud, select different combinations of geometric distortion and texture distortion to obtain a 3D point cloud quality assessment database, and conduct subjective assessment experiments to obtain the subjective quality score of each distorted point cloud. S2. Align the center of the point cloud with the origin of the coordinate system to obtain the point cloud coordinates; S3. Rotate the point cloud using a rotation vector to obtain multiple viewpoints and get the coordinates of the rotated point cloud; S4. After scaling and rotating the point cloud coordinates, the point cloud coordinates are obtained as follows: , where g s This represents the scaled coordinates, where s is the scaling factor. Let p be the coordinates after rotation; S5. Apply orthogonal projection and rasterization to the point cloud coordinates obtained in S4 to obtain the projected image. The specific process is as follows: ,in Let p be the projected coordinates. If there are identical points located at the same coordinates, the maximum value of the z component of g is taken; during rasterization, the attribute color information c is placed into its corresponding projection position, and the empty position c is (127,127,127), and finally the projection image I is obtained. S6. Repeat steps S2-S5 to obtain the projection images of the original point cloud and the distorted point cloud at each viewpoint. The obtained projection images of the original point cloud and the distorted point cloud at each viewpoint are respectively... and ; S7: Input the paired images from each viewpoint into the IW-SSIM algorithm to obtain the objective score for each viewpoint and pool it to obtain the final objective score of the distorted point cloud, thus completing the point cloud quality assessment. The objective score of the distorted point cloud is: ， in These are projection images of the original point cloud and the distorted point cloud at each viewpoint, respectively. Number of viewpoints.

2. The point cloud quality assessment method based on information-weighted projection according to claim 1, characterized in that, The process of aligning the center of the point cloud to the origin in S2 is as follows: ,in Let p be the 3D coordinates of point p after transformation (gc), g be the 3D coordinates of point p before transformation, c be the attribute color information, and t be the 3D coordinates of point p before transformation (gc). r The transformation vector is the geometric center coordinate of the corresponding reference point cloud.

3. The point cloud quality assessment method based on information-weighted projection according to claim 2, characterized in that, The rotation vector rθ mentioned in S3 is: , , where n v Let n be the unit normal vector of point p. z (0,0,1) Let r be the l2 norm of the orientation vector, θ be the rotation axis, and θ be the rotation angle. The rotation matrix R is obtained from r and θ. Then, the coordinates of point p after rotation are obtained. .

Images