Point cloud quality assessment method, encoder, decoder and storage medium

Through the point cloud quality evaluation method, the subjective quality measurement value is calculated using feature parameters and model parameters, and the problem of independent calculation of geometric and color quality in the existing technology is solved, and the accurate evaluation and simplified calculation of the subjective quality of point clouds is achieved.

CN113784129BActive Publication Date: 2024-11-22GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010525713.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-06-10
Publication Date
2024-11-22
Estimated Expiration
2040-06-10

AI Technical Summary

Technical Problem

In the prior art, in point cloud quality evaluation, objective mass distortion measurements of geometry and color are independently calculated, which cannot accurately reflect the subjective visual experience of the human eye.

Method used

A point cloud quality evaluation method is proposed. By analyzing the code stream, the characteristic parameters of the point cloud to be evaluated are obtained, the model parameters of the quality evaluation model are determined, and the subjective quality measurement value is calculated using the quality evaluation model, which simplifies the computational complexity and improves the evaluation accuracy. .

Benefits of technology

Accurate evaluation of the subjective quality of point clouds is achieved, simplifying the calculation process, requiring only the original point cloud and its eigenvalues, and without distorting point clouds and matching point pairs, improving the accuracy and efficiency of the evaluation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113784129B_ABST
    Figure CN113784129B_ABST
Patent Text Reader

Abstract

An embodiment of the present application discloses a point cloud quality assessment method, an encoder, a decoder and a storage medium. The method includes: parsing a bit stream to obtain characteristic parameters of a point cloud to be evaluated; determining model parameters of a quality assessment model; and determining a subjective quality measurement value of the point cloud to be evaluated using the quality assessment model based on the model parameters and the characteristic parameters of the point cloud to be evaluated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of video image processing, and in particular to a point cloud quality assessment method, an encoder, a decoder and a storage medium. Background Art

[0002] In the video-based point cloud compression (V-PCC) encoder framework, the point cloud distortion metric (PC_error) technology is a reference algorithm for measuring the objective quality of point clouds. The PC_error technology calculates the geometry-based Peak Signal to Noise Ratio (PSNR) and the color-based (or attribute-based) PSNR respectively to characterize the objective quality levels of geometry and color.

[0003] In the current related technical solutions, the geometric PSNR and color PSNR of the point cloud are calculated independently. However, when the human visual system receives the point cloud, the geometric and color quality distortions are obtained at the same time and jointly affect the final human visual experience, resulting in the related technical solutions being unable to accurately reflect the subjective point cloud quality of the human eye. Summary of the invention

[0004] This application proposes a point cloud quality assessment method, encoder, decoder and storage medium, which can simplify the computational complexity of subjective quality assessment and improve the accuracy of subjective quality assessment.

[0005] To achieve the above purpose, the technical solution of this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a point cloud quality assessment method, which is applied to a decoder or a media data processing device, and the method includes:

[0007] Parse the code stream to obtain the characteristic parameters of the point cloud to be evaluated;

[0008] determining model parameters of the quality assessment model;

[0009] The subjective quality measurement value of the point cloud to be evaluated is determined using the quality evaluation model according to the model parameters and the characteristic parameters of the point cloud to be evaluated.

[0010] In a second aspect, an embodiment of the present application provides a point cloud quality assessment method, which is applied to an encoder or a media data processing device, and the method includes:

[0011] Determine the characteristic parameters of the point cloud to be evaluated;

[0012] determining model parameters of the quality assessment model;

[0013] The subjective quality measurement value of the point cloud to be evaluated is determined using the quality evaluation model according to the model parameters and the characteristic parameters of the point cloud to be evaluated.

[0014] In a third aspect, an embodiment of the present application provides a decoder, the decoder comprising a parsing unit, a first determining unit and a first calculating unit; wherein:

[0015] The parsing unit is configured to parse the code stream to obtain feature parameters of the point cloud to be evaluated;

[0016] The first determining unit is configured to determine model parameters of the quality assessment model;

[0017] The first calculation unit is configured to determine a subjective quality measurement value of the point cloud to be evaluated using the quality assessment model according to the model parameters and the feature parameters of the point cloud to be evaluated.

[0018] In a fourth aspect, an embodiment of the present application provides a decoder, the decoder comprising a first memory and a first processor; wherein:

[0019] The first memory is used to store a computer program that can be run on the first processor;

[0020] The first processor is configured to execute the method according to the first aspect when running the computer program.

[0021] In a fifth aspect, an embodiment of the present application provides an encoder, the encoder comprising a second determination unit and a second calculation unit; wherein,

[0022] The second determination unit is configured to determine characteristic parameters of the point cloud to be evaluated;

[0023] The second determining unit is further configured to determine model parameters of the quality assessment model;

[0024] The second calculation unit is configured to determine a subjective quality measurement value of the point cloud to be evaluated using the quality assessment model according to the model parameters and the feature parameters of the point cloud to be evaluated.

[0025] In a sixth aspect, an embodiment of the present application provides an encoder, the encoder comprising a second memory and a second processor; wherein:

[0026] The second memory is used to store a computer program that can be run on the second processor;

[0027] The second processor is used to execute the method described in the second aspect when running the computer program.

[0028] In a seventh aspect, an embodiment of the present application provides a computer storage medium storing a computer program, wherein the computer program implements the method described in the first aspect when executed by a first processor, or implements the method described in the second aspect when executed by a second processor.

[0029] The point cloud quality assessment method, encoder, decoder and storage medium provided in the embodiment of the present application determine the model parameters of the quality assessment model after obtaining the characteristic parameters of the point cloud to be assessed; and use the quality assessment model to determine the subjective quality measurement value of the point cloud to be assessed based on the model parameters and the characteristic parameters of the point cloud to be assessed. In this way, the accuracy of subjective quality assessment can be improved by using the quality assessment model; and in the quality assessment model, the technical solution of the present application only needs to use the original point cloud and the characteristic values ​​extracted from the original point cloud, and no longer needs the distorted point cloud and the matching points between the original point cloud and the distorted point cloud, thereby simplifying the computational complexity of the subjective quality assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of the positions of point-to-point distortion and point-to-plane distortion provided in an embodiment of the present application;

[0031] Figure 2 A schematic diagram of a V-PCC encoding process provided in an embodiment of the present application;

[0032] Figure 3 A schematic diagram of a V-PCC decoding process provided in an embodiment of the present application;

[0033] Figure 4 A schematic diagram of a process flow of a point cloud quality assessment method provided in an embodiment of the present application;

[0034] Figure 5 A schematic diagram of a flow chart of another point cloud quality assessment method provided in an embodiment of the present application;

[0035] Figure 6 A schematic diagram of the relative positions between a point and its neighboring points reflecting color fluctuations at a geometric distance provided in an embodiment of the present application;

[0036] Figure 7 A schematic diagram of the structure of a non-empty voxel block reflecting the average variance of a color block provided in an embodiment of the present application;

[0037] Figure 8 A flowchart of another point cloud quality assessment method provided in an embodiment of the present application;

[0038] Fig. 9 A schematic diagram of the structure of a decoder provided in an embodiment of the present application;

[0039] Fig.10 A schematic diagram of the hardware structure of a decoder provided in an embodiment of the present application;

[0040] Fig.11 A schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0041] Fig.12 A schematic diagram of the hardware structure of an encoder provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The following will be combined with the accompanying drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. It is understood that the specific embodiments described herein are only used to explain the related applications, rather than to limit the applications. It should also be noted that, for the convenience of description, only the parts related to the related applications are shown in the accompanying drawings.

[0043] It should be understood that the point data set of the product appearance surface obtained by measuring instruments in reverse engineering can be called a point cloud. Usually, the number of points obtained by using a 3D coordinate measuring machine is relatively small, and the distance between points is relatively large, which can be called a sparse point cloud; while the point cloud obtained by using a 3D laser scanner or a photographic scanner has a large number of points and is relatively dense, which can be called a dense point cloud. Here, each point can include geometric information (such as position information, i.e., x, y, z coordinates) and attribute information (such as color information, i.e., R, G, B information), and this description method is called a point cloud.

[0044] Specifically, the relevant technical solutions usually use the PC_error objective point cloud distortion calculation technology, which is a reference algorithm for measuring the objective quality of point clouds. Here, the PC_error technology calculates the geometric PSNR and color PSNR respectively to characterize the objective quality levels of geometry and color, which can be divided into geometric PSNR based on point-to-point geometric distortion (D1), geometric PSNR based on point-to-plane geometric distortion (D2), and color PSNR. The following will describe the PSNR calculations in detail for these three aspects.

[0045] (1) Geometric PSNR based on point-to-point geometric distortion (D1). This calculation model uses the distance difference between the geometric coordinates of the reference point cloud and the distorted point cloud matching point pair to calculate the geometric mean square error (MSE), and then calculates the geometric PSNR based on the calculated MSE. Figure 1 As shown, the specific implementation method is to let "A" and "B" represent the reference point cloud and the compressed point cloud respectively, and the evaluated compression error is e B,A, which is used to represent the compression error of point cloud "B" for reference point cloud "A". Here, for each point b in the compressed point cloud "B" i ,like Figure 1 The white filled points in the middle indicate the corresponding point a in the reference point cloud “A”. j ,like Figure 1 The black filled points in the figure are shown; the nearest neighbor is used to locate the corresponding point. In particular, a KD tree search is used to perform the nearest neighbor search to reduce the computational complexity. j Associated to point b in the compressed point cloud "B" i To determine the error vector E(i,j). Here, the length of the error vector is the point-to-point error, as shown in the following formula:

[0046]

[0047] According to N in the compressed point cloud “B” B b i ∈B point-to-point distance At this time, D1 of the entire point cloud can be defined as follows,

[0048]

[0049] Similarly, You can also refer to the above method to find the final

[0050] That is, the calculation of the PSNR value based on the geometric distortion (D1) of the point-to-plane is as follows,

[0051]

[0052] Wherein, p is the peak constant predefined by MPEG for each reference point cloud, and MSE is the mean square error between points (D1). It should be noted that, taking the test sequence 8iVFB-Long_dress as an example, p can be defined as 1023; but it is not specifically limited.

[0053] (2) Geometric PSNR based on geometric distortion (D2) of point-to-plane. This calculation model uses the dot product of the distance difference between the geometric coordinates of the matching point pairs of the reference point cloud and the distorted point cloud and their corresponding normal vectors to calculate the geometric MSE, and then calculates the geometric PSNR based on the calculated MSE. Figure 1 For example, the specific implementation method is to follow the normal direction N j By projecting the error vector E(i,j), we can get a new error vector In this way, the calculation formula of point-to-surface error is as follows:

[0054]

[0055] Furthermore, the point-to-surface error (D2) of the entire point cloud can be defined as follows,

[0056]

[0057] Similarly, You can also refer to the above method to find the final

[0058] That is, the calculation of the PSNR value based on the geometric distortion (D2) of the point-to-plane is as follows,

[0059]

[0060] Wherein, p is a peak constant predefined by MPEG for each reference point cloud, and MSE is the mean square error between a point and a surface (D2). Here, taking the test sequence 8iVFB-Long_dress as an example, p can also be defined as 1023; but no specific limitation is made.

[0061] (3) Color PSNR: This calculation model uses the color difference between the reference point cloud and the distorted point cloud to calculate the color MSE, and then calculates the color PSNR based on the calculated MSE. Here, for lossy attribute coding, the calculation of the color PSNR value is as follows:

[0062]

[0063] It should be noted that in video images, the first color component, the second color component and the third color component are generally used to characterize the color attribute of each point in the current point cloud. In the RGB space, these three color components are red (represented by R), green (represented by G) and blue (represented by B); in the YUV space, these three color components are the brightness component (represented by the Y component), the first chroma component (represented by the U component) and the second chroma component (represented by the V component).

[0064] In this way, for the color attribute, the MSE of each of the three color components can be calculated. Here, the ITU-R BT.709 standard is used to convert from RGB space to YUV space. Since the bit depth of each point of the color attribute of all test data is 8 bits, the peak value p of the PSNR calculation can be 255.

[0065] Therefore, the relevant technical solution calculates the PSNR of the geometry and color of the point cloud independently. However, when the human visual system receives the point cloud, the quality distortion of the geometry and color is obtained at the same time and together affects the final visual experience, resulting in the relevant technical solution being unable to accurately reflect the subjective point cloud quality of the human eye.

[0066] In order to accurately predict the point cloud quality of human vision, the embodiment of the present application proposes a point cloud quality assessment method. After obtaining the characteristic parameters of the point cloud to be evaluated, the model parameters of the quality assessment model are determined; according to the model parameters and the characteristic parameters of the point cloud to be evaluated, the subjective quality measurement value of the point cloud to be evaluated is determined using the quality assessment model. In this way, the accuracy of subjective quality assessment can be improved by using the quality assessment model; and in this quality assessment model, the technical solution of the present application only needs to use the original point cloud and the characteristic values ​​extracted from the original point cloud, and no longer needs the distorted point cloud and the matching points between the original point cloud and the distorted point cloud, thereby simplifying the computational complexity of the subjective quality assessment.

[0067] It should also be pointed out that in order to obtain good visual quality, a point cloud with sufficient density is required, which brings a large amount of point cloud data. At present, the Moving Pictures Expert Group (MPEG) has proposed two point cloud compression technologies, one is video-based point cloud compression (V-PCC) technology, and the other is geometry-based point cloud compression (G-PCC) technology; among them, the embodiment of this application will be described in detail using V-PCC technology as an example.

[0068] The main idea of ​​V-PCC is to use existing video codecs to compress the geometric and color information of dynamic point clouds. Figure 2 As shown, it shows a schematic diagram of the framework of a V-PCC encoding process provided by an embodiment of the present application. Figure 2In , the V-PCC coding process framework is applied to a point cloud encoder. For the input three-dimensional point cloud (3DPC), it is first decomposed by patches to generate a set of patches; these patches can be independently mapped to a two-dimensional grid by simple orthogonal projection without suffering from automatic occlusion and without resampling the point cloud geometry. Further, the extracted patches are mapped to a two-dimensional network through a packing process to generate an occupancy map; wherein the occupancy map consists of a binary map that indicates whether each cell of the grid belongs to an empty space or a point cloud. Then, a geometric image and a color image are generated using the patch information and the occupancy map information; and after the geometric image and the color image are image filled, a filled geometric image and a filled color image are obtained; here, the filling process aims to fill the empty space between patches in an attempt to generate a piecewise smooth image that may be more suitable for video coding; and the smoothing process aims to alleviate potential discontinuities that may appear at the patch boundaries due to compression distortion, and the point cloud geometry reconstruction process uses the occupancy map information to detect non-empty pixels in the geometric image. After video compression using an existing video encoder, such as H.265 / High Efficiency Video Coding (HEVC), compressed geometry video and compressed color video are obtained. In addition, the occupancy map and auxiliary patch information will also be compressed separately to obtain compressed occupancy map and compressed auxiliary patch information; these compressed information are then multiplexed together to generate the compressed bitstream of the final point cloud V-PCC.

[0069] That is to say, Figure 2 The V-PCC coding process framework shown in the figure is based on the use of a video encoder for point cloud compression. For the input 3D point cloud, it is basically achieved by decomposing each point cloud patch in the 3D point cloud sequence into a set of patches, which are independently mapped to a 2D grid composed of uniform blocks. The mapping is then used to store the geometric information and color information as a geometric image and a color image respectively. Then, the existing video encoder, such as H.265 / HEVC, is used to compress the geometric image sequence and the color image sequence corresponding to the dynamic point cloud, respectively. Finally, the dynamic 3D point cloud is reconstructed using the geometric video and the color video as well as metadata (occupancy map of the 2D grid, auxiliary patch / block information, etc.). Among them, the bitstream of the compressed 3D point cloud can consist of two parts: geometric information and color information. For a given platform, the size of each part is controlled by the quantization parameter, which can accept a large number of values; at the same time, quantization will introduce distortion, thereby affecting the reconstruction quality.

[0070] like Figure 3As shown, it shows a schematic diagram of a V-PCC decoding process provided by an embodiment of the present application. Figure 3 In , the V-PCC decoding process framework is applied to the point cloud decoder. After the point cloud decoder obtains the compressed bit stream, it is first separated into compressed geometry video, compressed color video, compressed occupancy map and compressed auxiliary patch information through demultiplexing, and then the compressed geometry video and compressed color video are processed by video decompression to obtain decompressed geometry video and decompressed color video; the compressed occupancy map is processed by occupancy map decompression to obtain decompressed occupancy map; the compressed auxiliary patch information is processed by auxiliary patch information decompression to obtain decompressed auxiliary patch information, and then geometric reconstruction, smoothing and color reconstruction are performed in sequence to restore the three-dimensional point cloud input by the point cloud encoder.

[0071] Here, a point cloud quality assessment method provided in the embodiment of the present application can be applied to both the point cloud encoder and the point cloud decoder, or even to both the point cloud encoder and the point cloud decoder. In this way, if a better prediction effect can be obtained in the point cloud encoder by using the point cloud quality assessment method provided in the embodiment of the present application, then, correspondingly, a better prediction effect can also be obtained in the point cloud decoder.

[0072] Based on this, the technical solution of the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. Before the detailed description, it should be noted that the "first", "second", "third", etc. mentioned throughout the specification are only for distinguishing different features and do not have the function of limiting priority, sequence, size relationship, etc.

[0073] The embodiment of the present application provides a point cloud quality assessment method, which is applied to a point cloud decoder, which can be referred to as a decoder. The function implemented by the method can be implemented by calling a computer program by a first processor in the decoder. Of course, the computer program can be stored in a first memory. It can be seen that the decoder includes at least a first processor and a first memory.

[0074] See also Figure 4 , which shows a schematic diagram of a process of a point cloud quality assessment method according to an embodiment of the present application. Figure 4 As shown, the method may include:

[0075] S401: Parse the code stream to obtain feature parameters of the point cloud to be evaluated.

[0076] It should be noted that the point cloud quality assessment method of the embodiment of the present application can be applied to a decoder or a media data processing device. In practical applications, such as network optimization, quality assessment, etc., media data processing devices in the network, such as content delivery networks (CDNs), etc., the embodiment of the present application is not limited to playback devices used on the traditional user side and including decoders, but can also be other devices including decoders, and no limitation is made here.

[0077] It should also be noted that in a point cloud, a point can be all points in the point cloud or some points in the point cloud, and these points are relatively concentrated in space.

[0078] In some embodiments, the characteristic parameters of the point cloud to be evaluated may include quantization parameters of the point cloud to be evaluated; wherein the quantization parameters may include geometric quantization parameters and color quantization parameters of the point cloud to be evaluated.

[0079] Here, the geometric quantization parameter can be represented by QSg, which is used to indicate the quantization step value of the geometric video sequence; the color quantization parameter can be represented by QSc, which is used to indicate the quantization step value of the color / attribute video sequence.

[0080] In the embodiment of the present application, both the geometric quantization parameter and the color quantization parameter are determined on the encoder side based on the coding parameters; once the coding parameters are determined, the geometric quantization parameter and the color quantization parameter can be obtained, and these two parameters can be written into the bitstream; thus, by parsing the bitstream on the decoder side, these two parameters can be obtained so as to subsequently evaluate the subjective quality of the point cloud to be evaluated.

[0081] S402: Determine model parameters of the quality assessment model.

[0082] In the embodiment of the present application, in order to accurately predict the point cloud quality of human vision, firstly, different levels of distorted point clouds generated by high-quality original point clouds can be used to construct a comprehensive and effective point cloud subjective quality test data set using the Double Stimulus Impairment Scale (DSIS) subjective test method, and then a simple and effective point cloud subjective quality assessment model is proposed based on this data set, which can be referred to as a quality assessment model. Here, the quality assessment model can represent the evaluation of the subjective quality of the point cloud to be evaluated according to the model parameters.

[0083] That is to say, firstly, a subjective quality test data set needs to be established. In some embodiments, the establishment of the subjective quality test data set may include:

[0084] Acquire at least one reference point cloud;

[0085] Compressing each reference point cloud in the at least one reference point cloud using different values ​​of the quantization parameter to obtain a plurality of distorted point clouds corresponding to each reference point cloud;

[0086] The subjective quality test data set is obtained based on the at least one reference point cloud and the observation data of each of the plurality of distorted point clouds obtained.

[0087] Exemplarily, 16 high-quality original point clouds (i.e., reference point clouds) are selected from the Waterloo point cloud data set, and the point cloud content may include fruits, vegetables, snacks, etc. These reference point clouds are compressed using the 7th version test module of V-PCC to obtain distorted point clouds. Among them, for each reference point cloud, 25 levels of distorted point clouds are generated by setting 5 geometric quantization parameters (such as 26, 32, 38, 44 and 50) and 5 color quantization parameters (such as 26, 32, 38, 44 and 50). In order to display the three-dimensional point cloud and the two-dimensional video as complete as possible, a horizontal circle and a vertical circle with a radius of 5000 are selected as the virtual camera path in turn, and the center of the circle is located at the geometric center of the object, and a viewpoint is generated every two degrees of rotation on these circles around the reference point cloud and the distorted point cloud, so that 360 image frames can be generated for each point cloud. Then, the fragments of the distorted point cloud and its reference point cloud are horizontally connected to a 10-second video sequence for demonstration. 30 testers (15 males and 15 females) sat in front of the screen at a distance of about 2 times the height of the screen. First, they conducted a training session to observe the various levels of distorted point clouds in advance to familiarize themselves with the quality of point clouds from poor to good. Then, they conducted a formal test session. The test session used the DSIS method to conduct subjective testing. The original reference point cloud and the distorted point cloud fragments were presented on the screen at the same time. The observers evaluated the degree of distortion by comparing the difference between the two and gave a mean subjective opinion score (MOS) in the range of 0-100; here, the higher the MOS score, the better the subjective quality. The observers needed to observe for a total of 2 hours, test 400 data, divided into 4 test sessions, and rest for 5 minutes between every 2 consecutive test sessions. Finally, the 400 test data of the 30 observers were summarized, and the average MOS score of the observers was used as the final MOS score of each observed data. In this way, we can obtain a subjective quality test dataset of point clouds, and use the data in this dataset as the reference standard (ground truth) for the subsequent establishment of a quality assessment model.

[0088] In this way, after the subjective quality test data set is established in advance, the quality assessment model can be constructed based on the subjective quality test data set. Assume that the variable (100-MOS) is defined as the compensation value of MOS.c To represent, then the point cloud subjective quality assessment model can be obtained according to the test data in the subjective quality test data set, that is, the quality assessment model described in the embodiment of the present application is as follows:

[0089] MOS c =p1QS g +p2QS c +p3 (8)

[0090] Among them, QS g and QS c They represent the geometric quantization parameter and the color quantization parameter, respectively, and are used to indicate the quantization step value of the geometric video sequence and the color video sequence; p1, p2, and p3 represent model parameters. In the subjective quality test dataset, the accuracy of the fitted quality assessment model is shown in Table 1 below.

[0091] Table 1

[0092] Point Cloud p1 p2 p3 SCC RMSE Bag 0.223 0.183 6.342 0.949 4.954 Banana 0.247 0.08 23.601 0.902 6.336 Biscuits 0.143 0.156 12.072 0.927 4.387 Cake 0.241 0.125 10.489 0.938 5.153 Cauliflower 0.246 0.177 9.773 0.916 6.782 Flowerpot 0.291 0.075 16.212 0.877 8.339 House 0.22 0.269 3.597 0.93 7.059 Litchi 0.195 0.266 3.874 0.914 7.488 Mushroom 0.164 0.225 18.579 0.89 7.262 Ping-pong_bat 0.24 0.221 14.24 0.872 9.243 Puer_tea 0.124 0.297 11.921 0.948 5.568 Pumpkin 0.131 0.223 7.424 0.939 4.898 Ship 0.268 0.068 16.756 0.91 6.438 Statue 0.254 0.142 18.777 0.852 9.011 Stone 0.17 0.291 4.555 0.945 6.026 Tool_box 0.117 0.266 15.152 0.914 6.63 Average - - - 0.914 6.598

[0093] In Table 1, p1, p2, and p3 are model parameters fitted according to the subjective quality test data set. The squared correlation coefficient (SCC) represents the correlation between the MOS value predicted by formula (8) and the MOS value actually tested, which is used to measure the deviation between the predicted MOS value and the actual MOS value. The root mean square error (RMSE) represents the root mean square error between the MOS value predicted by formula (8) and the MOS value actually tested, which is used to measure the deviation between the predicted MOS value and the actual MOS value.

[0094] Among them, the closer the SCC is to 1, the smaller the deviation between the predicted MOS value and the actual MOS value, the higher the prediction accuracy and the better the subjective quality observation effect; conversely, the more the SCC deviates from 1, the greater the deviation between the predicted MOS value and the actual MOS value, the lower the prediction accuracy and the worse the subjective quality observation effect. For RMSE, the smaller the RMSE value, the smaller the deviation between the predicted MOS value and the actual MOS value, the higher the prediction accuracy and the better the subjective quality observation effect; conversely, the larger the RMSE value, the greater the deviation between the predicted MOS value and the actual MOS value, the lower the prediction accuracy and the worse the subjective quality observation effect.

[0095] That is to say, Table 1 shows the error between the MOS value predicted by the quality assessment model and the actual MOS value after fitting the model parameters. According to the content of Table 1, the SCC value is basically high, and the RMSE value is low, indicating that the accuracy of the MOS value predicted by formula (8) is very high, that is, the prediction error of the quality assessment model is small.

[0096] It should also be noted that the model parameters refer to the model coefficients used to construct the quality assessment model. Generally speaking, the model parameters may include multiple ones, such as the first model parameter (represented by p1), the second model parameter (represented by p2) and the third model parameter (represented by p3).

[0097] The determination of the model parameters may be described below through several implementation methods.

[0098] In a possible implementation, the model parameters of the quality assessment model may be determined by fitting a subjective quality test data set. The model parameters of the quality assessment model may include:

[0099] Obtain a subjective quality test dataset;

[0100] Based on the subjective quality test data set, a model parameter function is fitted; wherein the model parameter function is used to reflect the corresponding relationship between the model parameters and the characteristic parameters;

[0101] The model parameters are calculated according to the acquired characteristic parameters and the model parameter function.

[0102] It should be noted that, taking QSg and QSc as examples of characteristic parameters, the subjective quality test data set is pre-established, and the subjective quality test data set at least includes multiple distorted point clouds and QSg, QSc and actual MOS values ​​corresponding to each distorted point cloud. After obtaining the subjective quality test data set, the model parameter function can be fitted; at this time, the model parameter function reflects the corresponding relationship between the model parameters and QSg and QSc. For a certain point cloud to be evaluated, after knowing QSg and QSc, the fitted model parameters can be obtained according to the model parameter function, such as p1, p2, and p3 shown in Table 1. Then, according to the quality assessment model shown in formula (8), the MOS value of the point cloud can also be predicted.

[0103] In another possible implementation, the model parameters do not necessarily need to be calculated on the spot, and may also be backup data obtained in advance. The model parameters of the quality assessment model may include:

[0104] Model parameters for the point cloud to be evaluated are selected from one or more preset groups of candidate quality assessment model parameters.

[0105] That is, one or more sets of candidate quality assessment model parameters are pre-stored in the decoder or media data processing device. At this time, the model parameters for the point cloud to be evaluated can be directly selected from the preset one or more sets of candidate quality assessment model parameters to obtain the quality assessment model.

[0106] Furthermore, in order to enable the above-mentioned quality assessment model to be used in an actual encoder or decoder, the embodiment of the present application also proposes a feature-based prediction method for model parameters.

[0107] In another possible implementation, model parameters can be determined based on features extracted from point clouds. Specifically, the prediction model parameters mainly use two original point cloud features, including: a first feature about color fluctuation in geometric distance (CFGD) and a second feature about color block mean variance (CBMV) to determine the model parameters. Figure 5 As shown, the method may include:

[0108] S501: extracting features from the point cloud to be evaluated using a first calculation sub-model to obtain a first feature value of the point cloud to be evaluated;

[0109] It should be noted that the first calculation sub-model characterizes the eigenvalues ​​related to the color fluctuations at the geometric distance extracted from the point cloud to be evaluated. Here, the average value of the color intensity difference with the N neighboring points in the field at a unit distance can be used as the first eigenvalue of the point cloud, that is, the CFGD value. Wherein, N is an integer greater than 0, such as N is equal to 7, but it is not specifically limited.

[0110] Regarding the extraction of the first eigenvalue, in some embodiments, the extracting features of the point cloud to be evaluated by using the first computing sub-model to obtain the first eigenvalue of the point cloud to be evaluated may include:

[0111] Calculating first eigenvalues ​​corresponding to one or more points in the point cloud to be evaluated;

[0112] A weighted mean is calculated for the first eigenvalues ​​corresponding to the one or more points, and the obtained weighted mean is determined as the first eigenvalue of the point cloud to be evaluated.

[0113] Further, the calculating the first eigenvalues ​​corresponding to one or more points in the point cloud to be evaluated may include:

[0114] For a current point in the point cloud to be evaluated, determine a set of neighboring points associated with the current point; wherein the set of neighboring points includes at least one neighboring point;

[0115] For the neighbor point set, calculating a color intensity difference between the current point and the at least one neighbor point at a unit distance to obtain a color intensity difference at least one unit distance;

[0116] A weighted average of the color intensity differences over the at least one unit distance is calculated to obtain a first eigenvalue corresponding to the current point.

[0117] Further, the calculating the color intensity difference between the current point and the at least one neighboring point at a unit distance may include:

[0118] Obtaining a first color intensity value of a first color component of the current point and a second color intensity value of a first color component of at least one neighboring point;

[0119] Calculating an absolute value of a difference between a first color intensity value of the current point and a second color intensity value of the at least one neighboring point to obtain a color intensity difference between the current point and the at least one neighboring point;

[0120] According to the color intensity difference between the current point and the at least one neighboring point and the distance value between the current point and the at least one neighboring point, the color intensity difference between the current point and the at least one neighboring point at a unit distance is obtained.

[0121] It should be noted that in the embodiment of the present application, for the data of the point cloud to be evaluated, such as the first color intensity value of the current point, the first color intensity value of the neighboring point, etc., on the decoder side, these data can be obtained by parsing the bit stream or decoding the bit stream.

[0122] It should also be noted that for the weighted mean, the weight values ​​may be the same or different. In the case of the same weight values, it can be regarded as an average value, that is, the average value of equal weight values ​​is a special weighted mean.

[0123] That is to say, Figure 6 As shown in the figure, p0 is the current point to be calculated, p1, p2, ..., p7 are the N points closest to p0, where N is 7. First, the average value of the absolute difference in color intensity between the current point to be calculated and the N neighboring points at adjacent geometric Euclidean distances is calculated as the CFGD value corresponding to the current point to be calculated, and then the average value of the CFGD values ​​corresponding to multiple points in the entire point cloud is calculated, and the obtained average value is determined as the CFGD value of the point cloud. Here, the specific calculation formula for CFGD of the entire point cloud is as follows:

[0124]

[0125] Among them, P represents the point cloud to be evaluated, S represents the point p i The set of neighboring points, T represents the number of points in the point cloud, and N represents the point p i The number of neighboring points in the neighboring point set. i ) represents point p i The value of the first color component in the color attribute, C(p j ) represents point p j The value of the first color component in the color attribute of i,j Represents point p i and point p j The distance between Represents point p i With point p j The absolute difference in color intensity per unit distance.

[0126] It should also be noted that the distance here can be a geometric Euclidean distance, or a distance calculated based on a Morton code, etc., which is not limited in the embodiments of the present application; in addition, the first color component can be a Y component, but can also be expanded to a U component or a V component, etc., which is not limited in the embodiments of the present application.

[0127] S502: extracting features from the point cloud to be evaluated by using a second calculation sub-model to obtain a second feature value of the point cloud to be evaluated;

[0128] It should be noted that the second calculation sub-model represents the extraction of eigenvalues ​​related to the color block average variance from the point cloud to be evaluated. Here, the mean of the color standard deviations of all points in the non-empty voxel block can be used as the second eigenvalue of the point cloud, that is, the CBMV value.

[0129] Regarding the extraction of the second eigenvalue, in some embodiments, the extracting features of the point cloud to be evaluated by using the second computing sub-model to obtain the second eigenvalue of the point cloud to be evaluated may include:

[0130] Calculating second eigenvalues ​​corresponding to one or more non-empty voxel blocks in the point cloud to be evaluated;

[0131] A weighted mean is calculated for the second eigenvalues ​​corresponding to the one or more non-empty voxel blocks, and the obtained weighted mean is determined as the second eigenvalue of the point cloud to be evaluated.

[0132] Furthermore, the calculating the second eigenvalues ​​corresponding to one or more non-empty voxel blocks in the point cloud to be evaluated may include:

[0133] For a current non-empty voxel block in the point cloud to be evaluated, obtaining a third color intensity value of a first color component of at least one point in the current non-empty voxel block;

[0134] Calculating a weighted average of the third color intensity values ​​of at least one point in the current non-empty voxel block to obtain a color intensity average value of the current non-empty voxel block;

[0135] For at least one point in the current non-empty voxel block, determine a color standard deviation of the at least one point by using the third color intensity value and an average color intensity value of the current non-empty voxel block;

[0136] A weighted mean of the color standard deviation of the at least one point is calculated to obtain a second eigenvalue corresponding to the current non-empty voxel block.

[0137] It should be noted that a non-empty voxel block indicates that the voxel block includes at least one point. In addition, in the embodiment of the present application, for the data of the point cloud to be evaluated, such as the third color intensity value of at least one point in the current non-empty voxel block, on the decoder side, these data can also be obtained by parsing the bitstream or decoding the bitstream.

[0138] It should also be noted that if Figure 7 As shown in the figure, the entire point cloud is first divided into multiple H×W×L voxel blocks, where H, W, and L are integers greater than 0, for example, it is divided into 8×8×8 voxel blocks. The white box represents the i-th non-empty voxel block, p i1 ,p i2 ,…,p ij represents all points in a non-empty voxel block. Then the specific calculation formula of the CBMV of the entire point cloud is as follows:

[0139]

[0140] Where K represents the number of non-empty voxel blocks in the point cloud (i.e., the voxel block contains at least one point in the point cloud), M represents the number of all points in the i-th non-empty voxel block, and C(p ij ) indicates the pth ij The value of the first color component in the color attribute of each point, μ represents the average value of the first color component of all points in the non-empty voxel block; here, (C(p ij )-μ) 2 represents the pth non-empty voxel block in the i-th non-empty voxel block ij The color standard deviation of each point.

[0141] In this way, the first eigenvalue (i.e., CFGD value) of the point cloud to be evaluated can be calculated according to formula (9), and the second eigenvalue (i.e., CBMV value) of the point cloud to be evaluated can be calculated according to formula (10).

[0142] S503: Determine the model parameters according to the first eigenvalue, the second eigenvalue and a preset vector matrix.

[0143] It should be noted that, before S503, a preset vector matrix needs to be obtained first. In some embodiments, the method may further include:

[0144] Obtain a subjective quality test dataset;

[0145] The subjective quality test data set is trained to obtain the preset vector matrix.

[0146] Here, the preset vector matrix is ​​also obtained based on the subjective quality test data set. Specifically, after obtaining the subjective quality test data set, the preset vector matrix can be obtained by training the subjective quality test data set. Assuming that H represents the preset vector matrix, the value of the vector matrix that can be trained based on the subjective quality test data set is as follows,

[0147]

[0148] It should also be noted that since the preset vector matrix is ​​obtained based on training of a large amount of test data, different point clouds can use the vector matrix shown in formula (11).

[0149] In this way, after obtaining the preset vector matrix, the model parameters can be determined by combining the above two eigenvalues. In some embodiments, for S503, the determining of the model parameters according to the first eigenvalue, the second eigenvalue and the preset vector matrix may include:

[0150] Constructing a eigenvector based on a preset constant value, the first eigenvalue, and the second eigenvalue;

[0151] Performing a multiplication operation on the feature vector and the preset vector matrix to obtain a model parameter vector; wherein the model parameter vector includes a first model parameter, a second model parameter and a third model parameter;

[0152] The first model parameter, the second model parameter and the third model parameter are determined as the model parameters.

[0153] Further, in some embodiments, the preset constant value is an integer. Generally speaking, the preset constant value may be equal to 1, but is not specifically limited thereto.

[0154] That is to say, after extracting the two eigenvalues ​​CFGD and CBMV from the point cloud to be evaluated, they are marked with two variables f1 and f2 respectively; here, the first eigenvalue f1, the second eigenvalue f2 and the preset constant value 1 can form a 1×3 matrix, that is, a row vector, represented by the eigenvector F, F=[1 f1 f2]. In addition, the model parameters p1, p2 and p3 can also form a 1×3 matrix, represented by the model parameter vector P, P=[p1 p2 p3]. The preset vector matrix is ​​H, then the calculation method of the model parameter vector P is as follows,

[0155] P=F·H (12)

[0156] It should also be noted that, assuming that a specific point cloud quality test data set is known, the model parameters can be determined by fitting first; then two eigenvalues ​​are extracted using equations (9) and (10) to form a eigenvector, so that the preset vector matrix can also be obtained according to equation (12).

[0157] In this way, after extracting the two eigenvalues ​​of CFGD and CBMV from the point cloud to be evaluated, since H is a preset vector matrix, the model parameters can be determined according to formula (12) so that the MOS value of the point cloud to be evaluated can be calculated using the quality assessment model later.

[0158] S403: Determine a subjective quality measurement value of the point cloud to be evaluated using the quality evaluation model according to the model parameters and the characteristic parameters of the point cloud to be evaluated.

[0159] It should be noted that the quality assessment model can be regarded as a corresponding relationship between model parameters, feature parameters and subjective quality measurement values ​​(ie, MOS values). g and QS c The quality assessment model can also be regarded as a model parameter, QS g , QS c There is a corresponding relationship between and the subjective quality measurement value, which is shown in formula (8). g and QS c Afterwards, the subjective quality measurement value of the point cloud to be evaluated can be determined based on the quality assessment model.

[0160] In some embodiments, the quality assessment model can also be in a more complex form. In this case, the model parameters include four, represented by a, b, c, and d respectively; the quality assessment model is as follows:

[0161] MOS c =aQS g QS c +bQSg +qS c +d (13)

[0162] Compared with formula (8), one more QS is added g QS c The cross term of , and a model parameter is added synchronously, which makes the form more complicated and may improve the accuracy, but it will introduce greater difficulty in determining the model parameters. Since the more complicated the form, the greater the difficulty in practical application, the embodiment of the present application preferably uses the quality assessment model shown in formula (8).

[0163] In addition, the quality assessment model proposed in the embodiment of the present application can be applied not only to the field of bit rate control, but also to the fields of encoding parameter optimization, data enhancement in the post-processing process of point cloud data, etc.

[0164] Specifically, (1) in the field of rate control (RC), the point cloud quality assessment method can be used to establish a rate-distortion (RD) model, which includes a rate model and a distortion model, such as for distortion measurement. The calculation formula for its application is as follows:

[0165]

[0166]

[0167] Among them, R T represents the target bit rate, and the MOS obtained by equation (14) or equation (15) is C The result is the minimum value, the MOS C The smallest result indicates the largest MOS value, that is, the best subjective quality. In other words, if there are multiple bitrate options, the bitrate used by the bitrate control needs to be less than or equal to the target bitrate, while also obtaining the best subjective quality.

[0168] (2) In the field of coding parameter optimization, the appropriate geometric quantization parameter (QS) can be selected according to the quality assessment model shown in formula (8) g ) and color quantization parameter (QS c ) to obtain the required point cloud quality.

[0169] (3) In the field of data enhancement during the post-processing of point cloud data, a certain data enhancement algorithm can be used to process the point cloud, and then the point cloud quality predicted by the quality assessment model can be compared with the required point cloud quality. When the predicted point cloud quality is not less than the required point cloud quality, the enhancement algorithm can stop the enhancement strategy. Alternatively, in the process of deep learning on a large amount of data in the construction of a subjective quality test data set, three levels of test sequences such as high quality, medium quality, and low quality can be selected as the target data of the point cloud quality according to the quality assessment model for enhancement processing to obtain the required point cloud quality.

[0170] In addition, after the quality assessment model proposed in the embodiment of the present application obtains the subjective quality measurement value, the subjective quality measurement value can also be used for network optimization at the decoder side.

[0171] (1) For media data processing devices in the network, network parameters can be optimized according to the subjective quality measurement value, for example, the priority and routing table of the transmission unit containing the point cloud code stream data can be adjusted.

[0172] (2) For the playback device on the user side, for example, the subjective quality measurement value (or the mapping value of the subjective quality measurement value) can be fed back to the media data processing device in the network, and then the media data processing device optimizes the transmission network according to the subjective quality measurement value fed back by the user.

[0173] (3) For the playback device on the user side, for example, the post-processing unit parameters of the playback device can be adjusted according to the subjective quality measurement value (or the mapping value of the subjective quality measurement value) to enhance the point cloud obtained after decoding the bitstream (such as post-processing filtering, etc., refer to the image / video post-processing process).

[0174] In summary, in order to obtain a more effective quality assessment model, the embodiment of the present application first establishes a comprehensive and effective point cloud subjective test data set based on the existing V-PCC encoder. A more accurate and simple quality assessment model is further constructed on this data set. In order to make the quality assessment model widely used in practical application fields, two eigenvalues ​​such as color fluctuation on geometric distance and average variance of color blocks can be extracted from the original point cloud at this time, and the eigenvector F is formed with the preset constant value 1, so that the model parameters can be predicted using P=F·H shown in formula (12).

[0175] In the embodiment of the present application, the quality assessment model shown in formula (8) is used to test the test sequence of the point cloud subjective test data set. Among them, the Pearson Linear Correlation Coefficient (PLCC) is used to measure whether two data sets are on the same line, that is, to measure the linear relationship between fixed-distance variables; and the Spearman Rank-order Correlation Coefficient (SRCC) is used to reflect the closeness of the connection between two groups of variables, also known as the "rank difference method"; that is, in the embodiment of the present application, PLCC and SRCC are both used to reflect the consistency between the MOS value predicted by the quality assessment model and the actual MOS value.

[0176] Here, according to the PC_error technology used by V-PCC in the related technical solution, the PLCC and SRCC of the PSNR and MOS values ​​of the Y component can be obtained to be only 0.3956 and 0.3926; and based on the quality assessment model shown in formula (8), the PLCC and SRCC data have reached 0.9167 and 0.9174 respectively, as shown in Table 2; this data clearly shows that the solution of the present application has improved the accuracy of subjective quality assessment. In addition, the PC_error technology in the related technical solution needs to use the original point cloud and the distorted point cloud after encoding / decoding to calculate the PSNR of the Y component, and the demand for point clouds is large; while the quality assessment model proposed in the present application only needs the original point cloud and the two eigenvalues ​​extracted from the original point cloud, and no longer needs the distorted point cloud and the matching points between the original point cloud and the distorted point cloud, thereby simplifying the computational complexity of subjective quality assessment.

[0177] Table 2

[0178] Model Type Model PLCC SRCC RMSE FR <![CDATA[PSNR Y ]]> 0.3956 0.3926 20.2058 RR proposed algorithm 0.9167 0.9174 8.7933

[0179] Among them, FR is the full reference (Full Reference, FR) method, and RR represents the reduced reference (Reduced Reference, RR) method. Here, FR is the method adopted by the relevant technical solution, and RR is the method adopted by the embodiment of the present application. From Table 2, it is obvious that the PLCC and SRCC of the embodiment of the present application are much higher than the PLCC and SRCC of the relevant technical solution, while the RMSE of the embodiment of the present application is much lower than the RMSE of the relevant technical solution, indicating that the accuracy of the subjective quality evaluation under the embodiment of the present application is high.

[0180] The embodiment of the present application proposes a point cloud quality assessment method, which is applied to a decoder or a media data processing device. By parsing the bitstream, the characteristic parameters of the point cloud to be evaluated are obtained; the model parameters of the quality assessment model are determined; according to the model parameters and the characteristic parameters of the point cloud to be evaluated, the subjective quality measurement value of the point cloud to be evaluated is determined using the quality assessment model. In this way, the accuracy of subjective quality assessment can be improved by using the quality assessment model; and in this quality assessment model, the technical solution of the present application only needs to use the original point cloud and the characteristic values ​​extracted from the original point cloud, and no longer requires the distorted point cloud and the matching points between the original point cloud and the distorted point cloud, thereby simplifying the computational complexity of the subjective quality assessment.

[0181] The embodiment of the present application provides a point cloud quality assessment method, which is applied to a point cloud encoder, i.e., an encoder. The function implemented by the method can be implemented by calling a computer program by a second processor in the encoder, and of course the computer program can be stored in a second memory. It can be seen that the encoder includes at least a second processor and a second memory.

[0182] See also Figure 8 , which shows a flow chart of another point cloud quality assessment method according to an embodiment of the present application. Figure 8 As shown, the method may include:

[0183] S801: Determine characteristic parameters of the point cloud to be evaluated.

[0184] It should be noted that the point cloud quality assessment method of the embodiment of the present application can be applied to an encoder or a media data processing device. In practical applications, such as network optimization, quality assessment, etc., media data processing devices in the network, such as content delivery networks (CDNs), etc., the embodiment of the present application is not limited to playback devices used on the traditional user side and including encoders, but can also be other devices including encoders, and no limitation is made here.

[0185] It should also be noted that in a point cloud, a point can be all points in the point cloud or some points in the point cloud, and these points are relatively concentrated in space.

[0186] In some embodiments, the characteristic parameters of the point cloud to be evaluated may include quantization parameters of the point cloud to be evaluated; wherein the quantization parameters may include geometric quantization parameters and color quantization parameters of the point cloud to be evaluated.

[0187] Here, the geometric quantization parameter can be represented by QSg, which is used to indicate the quantization step value of the geometric video sequence; the color quantization parameter can be represented by QSc, which is used to indicate the quantization step value of the color / attribute video sequence.

[0188] In addition, whether it is a geometric quantization parameter or a color quantization parameter, it is determined on the encoder side according to the encoding parameter. In some embodiments, for S801, determining the characteristic parameters of the point cloud to be evaluated may include:

[0189] Obtaining precoding parameters of the point cloud to be evaluated;

[0190] According to the pre-coding parameters and a preset lookup table, characteristic parameters of the point cloud to be evaluated are determined; wherein the preset lookup table is used to reflect the corresponding relationship between the coding parameters and the characteristic parameters.

[0191] Further, the obtaining of the precoding parameters of the point cloud to be evaluated may include:

[0192] Pre-encoding the current block using multiple prediction modes to obtain a rate-distortion cost value corresponding to each prediction mode; wherein different prediction modes correspond to different encoding parameters

[0193] A minimum rate-distortion cost value is selected from the obtained multiple rate-distortion cost values, and a coding parameter corresponding to the minimum rate-distortion cost value is determined as the precoding parameter.

[0194] It should be noted that, on the encoder side, a simple decision strategy can be used to determine the precoding parameters, such as determining according to the size of the distortion value; or a complex decision strategy can be used, such as determining according to the result of rate distortion optimization (RDO), which is not limited in any way in the embodiments of the present application. Generally speaking, the RDO method can be used to determine the precoding parameters of the current block.

[0195] In this way, since the encoder stores a preset lookup table, the preset lookup table is used to reflect the correspondence between the encoding parameters and the characteristic parameters. Taking QSg and QSc as an example of characteristic parameters, the preset lookup table can also be used to reflect the correspondence between the encoding parameters and QSg and QSc; in this way, after determining the pre-coding parameters for this time, QSg and QSc corresponding to the pre-coding parameters can be found from the preset lookup table. At this time, the found QSg and QSc are the characteristic parameters of the point cloud to be evaluated, so as to subsequently evaluate the subjective quality of the point cloud to be evaluated.

[0196] It should also be noted that on the encoder side, the characteristic parameters (such as QSg and QSc) need to be written into the bitstream and then transmitted from the encoder to the decoder so that the characteristic parameters such as QSg and QSc can be obtained by parsing the bitstream on the decoder side.

[0197] S802: Determine model parameters of the quality assessment model.

[0198] It should be noted that the model parameters refer to the parameters used to construct the quality assessment model. Generally speaking, the model parameters may include multiple ones, such as a first model parameter (denoted by p1), a second model parameter (denoted by p2) and a third model parameter (denoted by p3).

[0199] The determination of the model parameters may be described below through several implementation methods.

[0200] In a possible implementation, the model parameters of the quality assessment model may be determined by fitting a subjective quality test data set. The model parameters of the quality assessment model may include:

[0201] Obtain a subjective quality test dataset;

[0202] Based on the subjective quality test data set, a model parameter function is fitted; wherein the model parameter function is used to reflect the corresponding relationship between the model parameters and the characteristic parameters;

[0203] The model parameters are calculated according to the acquired characteristic parameters and the model parameter function.

[0204] It should be noted that, taking QSg and QSc as examples of characteristic parameters, the subjective quality test data set is pre-established, and the subjective quality test data set at least includes multiple distorted point clouds and QSg, QSc and actual MOS values ​​corresponding to each distorted point cloud. After obtaining the subjective quality test data set, the model parameter function can be fitted; at this time, the model parameter function reflects the corresponding relationship between the model parameters and QSg and QSc. For a certain point cloud to be evaluated, after knowing QSg and QSc, the fitted model parameters can be obtained according to the model parameter function, such as p1, p2, and p3 shown in Table 1. Then, according to the quality assessment model shown in formula (8), the MOS value of the point cloud can also be predicted.

[0205] In another possible implementation, the model parameters do not necessarily need to be calculated on the spot, and may also be backup data obtained in advance. The model parameters of the quality assessment model may include:

[0206] Model parameters for the point cloud to be evaluated are selected from one or more preset groups of candidate quality assessment model parameters.

[0207] That is, one or more sets of candidate quality assessment model parameters are pre-stored in the decoder or media data processing device. At this time, the model parameters for the point cloud to be evaluated can be directly selected from the preset one or more sets of candidate quality assessment model parameters to obtain the quality assessment model.

[0208] Furthermore, in order to enable the above-mentioned quality assessment model to be used in an actual encoder or decoder, the embodiment of the present application also proposes a feature-based prediction method for model parameters.

[0209] In another possible implementation, model parameters may be determined based on features extracted from the point cloud. Determining the model parameters of the quality assessment model may include:

[0210] Using the first calculation sub-model to perform feature extraction on the point cloud to be evaluated, to obtain a first feature value of the point cloud to be evaluated;

[0211] Using the second computing sub-model to perform feature extraction on the point cloud to be evaluated, to obtain a second feature value of the point cloud to be evaluated;

[0212] The model parameters are determined according to the first eigenvalue, the second eigenvalue and a preset vector matrix.

[0213] It should be noted that the first calculation sub-model characterizes the eigenvalues ​​related to the color fluctuations at the geometric distance extracted from the point cloud to be evaluated. Here, the average value of the color intensity difference with the N neighboring points in the field at a unit distance can be used as the first eigenvalue of the point cloud, that is, the CFGD value. Wherein, N is an integer greater than 0, such as N is equal to 7, but it is not specifically limited.

[0214] It should also be noted that the second calculation sub-model characterizes the extraction of eigenvalues ​​related to the color block average variance from the point cloud to be evaluated. Here, the mean of the color standard deviations of all points in the non-empty voxel block can be used as the second eigenvalue of the point cloud, that is, the CBMV value.

[0215] The following is a detailed description of the extraction process of the two eigenvalues.

[0216] Regarding the extraction of the first eigenvalue, in some embodiments, the extracting features of the point cloud to be evaluated by using the first computing sub-model to obtain the first eigenvalue of the point cloud to be evaluated may include:

[0217] Calculating first eigenvalues ​​corresponding to one or more points in the point cloud to be evaluated;

[0218] A weighted mean is calculated for the first eigenvalues ​​corresponding to the one or more points, and the obtained weighted mean is determined as the first eigenvalue of the point cloud to be evaluated.

[0219] Further, the calculating the first eigenvalues ​​corresponding to one or more points in the point cloud to be evaluated may include:

[0220] For a current point in the point cloud to be evaluated, determine a set of neighboring points associated with the current point; wherein the set of neighboring points includes at least one neighboring point;

[0221] For the neighbor point set, calculating a color intensity difference between the current point and the at least one neighbor point at a unit distance to obtain a color intensity difference at least one unit distance;

[0222] A weighted average of the color intensity differences over the at least one unit distance is calculated to obtain a first eigenvalue corresponding to the current point.

[0223] Further, the calculating the color intensity difference between the current point and the at least one neighboring point at a unit distance may include:

[0224] Obtaining a first color intensity value of a first color component of the current point and a second color intensity value of a first color component of at least one neighboring point;

[0225] Calculating an absolute value of a difference between a first color intensity value of the current point and a second color intensity value of the at least one neighboring point to obtain a color intensity difference between the current point and the at least one neighboring point;

[0226] According to the color intensity difference between the current point and the at least one neighboring point and the distance value between the current point and the at least one neighboring point, the color intensity difference between the current point and the at least one neighboring point at a unit distance is obtained.

[0227] It should be noted that in the embodiment of the present application, for the data of the point cloud to be evaluated, such as the first color intensity value of the current point, the first color intensity value of the neighboring point, etc., on the encoder side, these data can be obtained through the data of the point cloud to be evaluated, and these data also need to be written into the code stream for transmission from the encoder to the decoder.

[0228] It should also be noted that for the weighted mean, the weight values ​​may be the same or different. In the case of the same weight values, it can be regarded as an average value, that is, the average value of equal weight values ​​is a special weighted mean.

[0229] That is to say, Figure 6 As shown, p0 is the current point to be calculated, p1, p2, ..., p7 represent the N points closest to point p0, where N is 7. First, the average value of the absolute difference in color intensity between the current point to be calculated and the N neighboring points at adjacent geometric Euclidean distances is calculated as the CFGD value corresponding to the current point to be calculated, and then the CFGD values ​​corresponding to multiple points in the entire point cloud are averaged, and the obtained average is determined as the CFGD value of the point cloud. Here, the specific calculation formula of the CFGD of the entire point cloud is shown in the above formula (9), and the details are described in the decoder side.

[0230] Regarding the extraction of the second eigenvalue, in some embodiments, the extracting features of the point cloud to be evaluated by using the second computing sub-model to obtain the second eigenvalue of the point cloud to be evaluated may include:

[0231] Calculating second eigenvalues ​​corresponding to one or more non-empty voxel blocks in the point cloud to be evaluated;

[0232] A weighted mean is calculated for the second eigenvalues ​​corresponding to the one or more non-empty voxel blocks, and the obtained weighted mean is determined as the second eigenvalue of the point cloud to be evaluated.

[0233] Furthermore, the calculating the second eigenvalues ​​corresponding to one or more non-empty voxel blocks in the point cloud to be evaluated may include:

[0234] For a current non-empty voxel block in the point cloud to be evaluated, obtaining a third color intensity value of a first color component of at least one point in the current non-empty voxel block;

[0235] Calculating a weighted average of the third color intensity values ​​of at least one point in the current non-empty voxel block to obtain a color intensity average value of the current non-empty voxel block;

[0236] For at least one point in the current non-empty voxel block, determine a color standard deviation of the at least one point by using the third color intensity value and an average color intensity value of the current non-empty voxel block;

[0237] A weighted mean of the color standard deviation of the at least one point is calculated to obtain a second eigenvalue corresponding to the current non-empty voxel block.

[0238] It should be noted that a non-empty voxel block indicates that the voxel block includes at least one point. In addition, in the embodiment of the present application, for the data of the point cloud to be evaluated, such as the third color intensity value of at least one point in the current non-empty voxel block, on the encoder side, these data can be obtained through the data of the point cloud to be evaluated, and these data also need to be written into the bitstream to be transmitted from the encoder to the decoder.

[0239] That is to say, Figure 7 As shown in the figure, the entire point cloud is first divided into multiple H×W×L voxel blocks, where H, W, and L are integers greater than 0, for example, it is divided into 8×8×8 voxel blocks. The white box represents the i-th non-empty voxel block, p i1 ,p i2 ,…,p ij represents all points in a non-empty voxel block. Then the specific calculation formula of the CBMV of the entire point cloud is shown in the above formula (10), and the details are described in the decoder side.

[0240] In this way, the first eigenvalue (i.e., CFGD value) of the point cloud to be evaluated can be calculated according to the above formula (9), and the second eigenvalue (i.e., CBMV value) of the point cloud to be evaluated can be calculated according to the above formula (10).

[0241] It should be noted that before determining the model parameters, it is also necessary to obtain a preset vector matrix. In some embodiments, the method may also include:

[0242] Obtain a subjective quality test dataset;

[0243] The subjective quality test data set is trained to obtain the preset vector matrix.

[0244] Here, the preset vector matrix is ​​also obtained based on the subjective quality test data set. Specifically, after obtaining the subjective quality test data set, the preset vector matrix can be obtained by training the subjective quality test data set. Assuming that H represents the preset vector matrix, the value of the vector matrix that can be trained based on the subjective quality test data set is as shown in the above formula (11).

[0245] It should also be noted that since the preset vector matrix is ​​obtained based on training of a large amount of test data, different point clouds can use the vector matrix shown in formula (11).

[0246] In this way, after obtaining the preset vector matrix, the above two eigenvalues ​​can be combined to determine the model parameters. In some embodiments, determining the model parameters according to the first eigenvalue, the second eigenvalue and the preset vector matrix may include:

[0247] Constructing a eigenvector based on a preset constant value, the first eigenvalue, and the second eigenvalue;

[0248] Performing a multiplication operation on the feature vector and the preset vector matrix to obtain a model parameter vector; wherein the model parameter vector includes a first model parameter, a second model parameter and a third model parameter;

[0249] The first model parameter, the second model parameter and the third model parameter are determined as the model parameters.

[0250] Further, in some embodiments, the preset constant value is an integer. Generally speaking, the preset constant value may be equal to 1, but is not specifically limited thereto.

[0251] That is to say, after extracting the two eigenvalues ​​CFGD and CBMV from the point cloud to be evaluated, they are marked with two variables f1 and f2 respectively; here, the first eigenvalue f1, the second eigenvalue f2 and the preset constant value 1 can form a 1×3 matrix, that is, a row vector, represented by the eigenvector F, F = [1 f1 f2]. In addition, the model parameters p1, p2 and p3 can also form a 1×3 matrix, represented by the model parameter vector P, P = [p1 p2 p3], and the preset vector matrix is ​​H, then the calculation method of the model parameter vector P is shown in the above formula (12).

[0252] In this way, after extracting the two eigenvalues ​​CFGD and CBMV from the point cloud to be evaluated, since H is a preset vector matrix, the model parameters can be determined according to the above formula (12), so that the MOS value of the point cloud to be evaluated can be calculated using the quality assessment model later.

[0253] S803: Determine a subjective quality measurement value of the point cloud to be evaluated using the quality evaluation model according to the model parameters and the characteristic parameters of the point cloud to be evaluated.

[0254] It should be noted that the quality assessment model can be regarded as a corresponding relationship between model parameters, feature parameters and subjective quality measurement values ​​(ie, MOS values). g and QS c The quality assessment model can also be regarded as a model parameter, QS g , QS c There is a corresponding relationship between and the subjective quality measurement value, which is shown in formula (8). g and QS c Afterwards, the subjective quality measurement value of the point cloud to be evaluated can be determined based on the quality assessment model.

[0255] In addition, after the quality assessment model proposed in the embodiment of the present application obtains the subjective quality measurement value, the subjective quality measurement value can also be used for network optimization on the encoder side.

[0256] (1) The subjective quality measurement value (or the mapping value of the subjective quality measurement value) is transmitted to the encoder optimization module, and the encoder optimization module determines the encoding parameters of the encoder according to the subjective quality measurement value, for example, the rate-distortion optimization process, the bit rate control process, etc.

[0257] (2) From the point cloud code stream sending side, the subjective quality measurement value can also be sent to the media data processing device in the network to optimize the transmission network.

[0258] (3) The subjective quality measurement value may also be sent to the playback device on the user side. The playback device on the user side may then compare the subjective quality measurement value with its local evaluation value to evaluate the transmission quality of the network and feed the evaluation result back to the media data processing device in the network for network optimization.

[0259] The embodiment of the present application proposes a point cloud quality assessment method, which is applied to an encoder or a media data processing device. By determining the characteristic parameters of the point cloud to be evaluated; determining the model parameters of the quality assessment model; and then using the quality assessment model according to the model parameters and the characteristic parameters of the point cloud to be evaluated, determining the subjective quality measurement value of the point cloud to be evaluated. In this way, the accuracy of subjective quality assessment can be improved by using the quality assessment model; and in this quality assessment model, the technical solution of the present application only needs to use the original point cloud and the characteristic values ​​extracted from the original point cloud, and no longer needs the distorted point cloud and the matching points between the original point cloud and the distorted point cloud, thereby simplifying the computational complexity of the subjective quality assessment.

[0260] Based on the same inventive concept as the above embodiments, see Fig. 9 , which shows a schematic diagram of the structure of a decoder 90 provided in an embodiment of the present application. Fig. 9 As shown, the decoder 90 may include: a parsing unit 901, a first determining unit 902 and a first calculating unit 903; wherein,

[0261] The parsing unit 901 is configured to parse the code stream to obtain characteristic parameters of the point cloud to be evaluated;

[0262] A first determining unit 902 is configured to determine a model parameter of a quality assessment model;

[0263] The first calculation unit 903 is configured to determine a subjective quality measurement value of the point cloud to be evaluated using the quality assessment model according to the model parameters and the feature parameters of the point cloud to be evaluated.

[0264] In some embodiments, the characteristic parameters of the point cloud to be evaluated include quantization parameters of the point cloud to be evaluated; wherein the quantization parameters include geometric quantization parameters and color quantization parameters of the point cloud to be evaluated.

[0265] In some embodiments, see Fig. 9 The decoder 90 may further include a first fitting unit 904 configured to obtain a subjective quality test data set; based on the subjective quality test data set, fit a model parameter function; wherein the model parameter function is used to reflect the corresponding relationship between the model parameters and the feature parameters;

[0266] The first calculation unit 903 is further configured to calculate the model parameters according to the acquired characteristic parameters and the model parameter function.

[0267] In some embodiments, the first determining unit 902 is further configured to select model parameters for the point cloud to be evaluated from one or more preset groups of candidate quality assessment model parameters.

[0268] In some embodiments, the first computing unit 903 is further configured to perform feature extraction on the point cloud to be evaluated using the first computing sub-model to obtain a first feature value of the point cloud to be evaluated; perform feature extraction on the point cloud to be evaluated using the second computing sub-model to obtain a second feature value of the point cloud to be evaluated;

[0269] The first determining unit 902 is further configured to determine the model parameters according to the first eigenvalue, the second eigenvalue and a preset vector matrix;

[0270] The first calculation sub-model represents the extraction of eigenvalues ​​related to color fluctuations on geometric distances from the point cloud to be evaluated, and the second calculation sub-model represents the extraction of eigenvalues ​​related to the average variance of color blocks from the point cloud to be evaluated.

[0271] In some embodiments, the first calculation unit 903 is further configured to calculate the first eigenvalues ​​corresponding to one or more points in the point cloud to be evaluated; perform weighted mean calculation on the first eigenvalues ​​corresponding to the one or more points, and determine the obtained weighted mean as the first eigenvalue of the point cloud to be evaluated.

[0272] In some embodiments, the first determining unit 902 is further configured to determine, for a current point in the point cloud to be evaluated, a set of neighboring points associated with the current point; wherein the set of neighboring points includes at least one neighboring point;

[0273] The first calculation unit 903 is also configured to calculate, for the set of neighboring points, the color intensity difference between the current point and at least one neighboring point at a unit distance, to obtain the color intensity difference at at least one unit distance; and calculate the weighted mean of the color intensity difference at at least one unit distance to obtain the first eigenvalue corresponding to the current point.

[0274] In some embodiments, the first calculation unit 903 is further configured to obtain a first color intensity value of a first color component of the current point and a second color intensity value of a first color component of the at least one neighboring point; calculate an absolute value of a difference between the first color intensity value of the current point and the second color intensity value of the at least one neighboring point to obtain a color intensity difference between the current point and the at least one neighboring point;

[0275] The first determination unit 902 is further configured to determine the color intensity difference between the current point and the at least one neighboring point per unit distance based on the color intensity difference between the current point and the at least one neighboring point and the distance value between the current point and the at least one neighboring point.

[0276] In some embodiments, the first computing unit 903 is further configured to calculate the second eigenvalues ​​corresponding to one or more non-empty voxel blocks in the point cloud to be evaluated; and perform weighted mean calculation on the second eigenvalues ​​corresponding to the one or more non-empty voxel blocks, and determine the obtained weighted mean as the second eigenvalue of the point cloud to be evaluated.

[0277] In some embodiments, the first calculation unit 903 is further configured to obtain, for the current non-empty voxel block in the point cloud to be evaluated, a third color intensity value of the first color component of at least one point in the current non-empty voxel block; calculate a weighted average of the third color intensity values ​​of at least one point in the current non-empty voxel block to obtain the color intensity average of the current non-empty voxel block; and is further configured to determine, for at least one point in the current non-empty voxel block, a color standard deviation of the at least one point using the third color intensity value and the color intensity average of the current non-empty voxel block; calculate a weighted average of the color standard deviations of the at least one point to obtain a second eigenvalue corresponding to the current non-empty voxel block.

[0278] In some embodiments, see Fig. 9 The decoder 90 may further include a first training unit 905 configured to obtain a subjective quality test data set; and train the subjective quality test data set to obtain the preset vector matrix.

[0279] In some embodiments, the first determination unit 902 is further configured to select the preset vector matrix for determining the model parameters from one or more preset groups of candidate vector matrices.

[0280] In some embodiments, see Fig. 9 , the decoder 90 may further include a first construction unit 906 configured to construct a feature vector based on a preset constant value, the first feature value and the second feature value;

[0281] The first calculation unit 903 is further configured to perform a multiplication operation on the feature vector and the preset vector matrix to obtain a model parameter vector; wherein the model parameter vector includes a first model parameter, a second model parameter and a third model parameter;

[0282] The first determining unit 902 is further configured to determine the first model parameter, the second model parameter and the third model parameter as the model parameters.

[0283] In some embodiments, the preset constant value is an integer.

[0284] It can be understood that in this embodiment, a "unit" can be a part of a circuit, a part of a processor, a part of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional module.

[0285] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0286] Therefore, an embodiment of the present application provides a computer storage medium, which is applied to a decoder 90. The computer storage medium stores a computer program. When the computer program is executed by a first processor, the method described in the decoder side of the above embodiment is implemented.

[0287] Based on the above-mentioned components of the decoder 90 and the computer storage medium, see Fig.10 , which shows a specific hardware structure example of the decoder 90 provided in an embodiment of the present application, which may include: a first communication interface 1001, a first memory 1002 and a first processor 1003; each component is coupled together through a first bus system 1004. It can be understood that the first bus system 1004 is used to realize the connection and communication between these components. In addition to the data bus, the first bus system 1004 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, in Fig.10 In the figure, various buses are marked as the first bus system 1004. Among them,

[0288] The first communication interface 1001 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0289] A first memory 1002, used to store a computer program that can be run on the first processor 1003;

[0290] The first processor 1003 is configured to, when running the computer program, execute:

[0291] Parse the code stream to obtain the characteristic parameters of the point cloud to be evaluated;

[0292] determining model parameters of the quality assessment model;

[0293] The subjective quality measurement value of the point cloud to be evaluated is determined using the quality evaluation model according to the model parameters and the characteristic parameters of the point cloud to be evaluated.

[0294] It can be understood that the first memory 1002 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DRRAM). The first memory 1002 of the system and method described in the present application is intended to include but is not limited to these and any other suitable types of memory.

[0295] The first processor 1003 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the first processor 1003. The above-mentioned first processor 1003 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the first memory 1002, and the first processor 1003 reads the information in the first memory 1002 and completes the steps of the above method in combination with its hardware.

[0296] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processors (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic devices (Programmable Logic Device, PLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units or combinations thereof for performing functions described in the present application. For software implementation, the technology described in the present application can be implemented by a module (such as a process, function, etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in a processor or outside a processor.

[0297] Optionally, as another embodiment, the first processor 1003 is further configured to execute the method described in any one of the aforementioned embodiments when running the computer program.

[0298] The present embodiment provides a decoder, which may include a parsing unit, a first determining unit, and a first calculating unit. In the decoder, the quality assessment model can be used to improve the accuracy of subjective quality assessment; and in the quality assessment model, the technical solution of the present application only needs to use the original point cloud and the feature values ​​extracted from the original point cloud, and no longer needs the distorted point cloud and the matching point pairs of the original point cloud and the distorted point cloud, thereby simplifying the computational complexity of the subjective quality assessment.

[0299] Based on the same inventive concept as the above embodiments, see Fig.11 , which shows a schematic diagram of the structure of an encoder 110 provided in an embodiment of the present application. Fig.11 As shown, the encoder 110 may include: a second determining unit 1101 and a second calculating unit 1102; wherein,

[0300] A second determining unit 1101 is configured to determine characteristic parameters of the point cloud to be evaluated;

[0301] The second determining unit 1101 is further configured to determine model parameters of the quality assessment model;

[0302] The second calculation unit 1102 is configured to determine a subjective quality measurement value of the point cloud to be evaluated using the quality assessment model according to the model parameters and the feature parameters of the point cloud to be evaluated.

[0303] In some embodiments, see Fig.11 The encoder 110 may further include a search unit 1103, configured to obtain pre-coding parameters of the point cloud to be evaluated; determine characteristic parameters of the point cloud to be evaluated according to the pre-coding parameters and a preset lookup table; wherein the preset lookup table is used to reflect the correspondence between the encoding parameters and the characteristic parameters.

[0304] In some embodiments, the characteristic parameters of the point cloud to be evaluated include quantization parameters of the point cloud to be evaluated; wherein the quantization parameters include geometric quantization parameters and color quantization parameters of the point cloud to be evaluated.

[0305] In some embodiments, see Fig.11 The encoder 110 may further include a second fitting unit 1104 configured to obtain a subjective quality test data set; based on the subjective quality test data set, fit a model parameter function; wherein the model parameter function is used to reflect the corresponding relationship between the model parameters and the feature parameters;

[0306] The second calculation unit 1102 is further configured to calculate the model parameters according to the acquired characteristic parameters and the model parameter function.

[0307] In some embodiments, the second determining unit 1101 is further configured to select model parameters for the point cloud to be evaluated from one or more preset groups of candidate quality assessment model parameters.

[0308] In some embodiments, the second computing unit 1102 is further configured to perform feature extraction on the point cloud to be evaluated using the first computing sub-model to obtain a first feature value of the point cloud to be evaluated; perform feature extraction on the point cloud to be evaluated using the second computing sub-model to obtain a second feature value of the point cloud to be evaluated;

[0309] The second determining unit 1101 is further configured to determine the model parameters according to the first eigenvalue, the second eigenvalue and a preset vector matrix;

[0310] The first calculation sub-model represents the extraction of eigenvalues ​​related to color fluctuations on geometric distances from the point cloud to be evaluated, and the second calculation sub-model represents the extraction of eigenvalues ​​related to the average variance of color blocks from the point cloud to be evaluated.

[0311] In some embodiments, the second computing unit 1102 is further configured to calculate the first eigenvalues ​​corresponding to one or more points in the point cloud to be evaluated; perform weighted mean calculation on the first eigenvalues ​​corresponding to the one or more points, and determine the obtained weighted mean as the first eigenvalue of the point cloud to be evaluated.

[0312] In some embodiments, the second determining unit 1101 is further configured to determine, for a current point in the point cloud to be evaluated, a set of neighboring points associated with the current point; wherein the set of neighboring points includes at least one neighboring point;

[0313] The second calculation unit 1102 is further configured to calculate, for the set of neighboring points, the color intensity difference between the current point and at least one neighboring point at a unit distance, to obtain the color intensity difference at at least one unit distance; and calculate the weighted mean of the color intensity difference at at least one unit distance to obtain the first eigenvalue corresponding to the current point.

[0314] In some embodiments, the second calculation unit 1102 is further configured to obtain a first color intensity value of a first color component of the current point and a second color intensity value of a first color component of the at least one neighboring point; calculate an absolute value of a difference between the first color intensity value of the current point and the second color intensity value of the at least one neighboring point, to obtain a color intensity difference between the current point and the at least one neighboring point;

[0315] The second determination unit 1101 is further configured to determine the color intensity difference between the current point and the at least one neighboring point per unit distance based on the color intensity difference between the current point and the at least one neighboring point and the distance value between the current point and the at least one neighboring point.

[0316] In some embodiments, the second computing unit 1102 is further configured to calculate the second eigenvalues ​​corresponding to one or more non-empty voxel blocks in the point cloud to be evaluated; and perform weighted mean calculation on the second eigenvalues ​​corresponding to the one or more non-empty voxel blocks, and determine the obtained weighted mean as the second eigenvalue of the point cloud to be evaluated.

[0317] In some embodiments, the second computing unit 1102 is further configured to, for a current non-empty voxel block in the point cloud to be evaluated, obtain a third color intensity value of a first color component of at least one point in the current non-empty voxel block; calculate a weighted average of the third color intensity values ​​of at least one point in the current non-empty voxel block to obtain the color intensity average of the current non-empty voxel block; and is further configured to, for at least one point in the current non-empty voxel block, determine the color standard deviation of the at least one point using the third color intensity value and the color intensity average of the current non-empty voxel block; calculate the weighted average of the color standard deviations of the at least one point to obtain a second eigenvalue corresponding to the current non-empty voxel block.

[0318] In some embodiments, see Fig.11 The encoder 110 may further include a second training unit 1105 configured to obtain a subjective quality test data set; and train the subjective quality test data set to obtain the preset vector matrix.

[0319] In some embodiments, the second determining unit 1101 is further configured to obtain a subjective quality test data set; and train the subjective quality test data set to obtain the preset vector matrix.

[0320] In some embodiments, see Fig.11 , the encoder 110 may further include a second construction unit 1106 configured to construct a feature vector based on a preset constant value, the first feature value and the second feature value;

[0321] The second calculation unit 1102 is further configured to perform a multiplication operation on the feature vector and the preset vector matrix to obtain a model parameter vector; wherein the model parameter vector includes a first model parameter, a second model parameter and a third model parameter;

[0322] The second determining unit 1101 is further configured to determine the first model parameter, the second model parameter and the third model parameter as the model parameters.

[0323] In some embodiments, the preset constant value is an integer.

[0324] It can be understood that in this embodiment, a "unit" can be a part of a circuit, a part of a processor, a part of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional module.

[0325] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer storage medium, which is applied to the encoder 110. The computer storage medium stores a computer program. When the computer program is executed by the second processor, the method described in the encoder side of the above embodiment is implemented.

[0326] Based on the composition of the encoder 110 and the computer storage medium, see Fig.12 , which shows a specific hardware structure example of the encoder 110 provided in an embodiment of the present application, which may include: a second communication interface 1201, a second memory 1202, and a second processor 1203; each component is coupled together via a second bus system 1204. It is understandable that the second bus system 1204 is used to realize the connection and communication between these components. In addition to the data bus, the second bus system 1204 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in Fig.12 In the figure, various buses are labeled as the second bus system 1204. Among them,

[0327] The second communication interface 1201 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;

[0328] The second memory 1202 is used to store a computer program that can be run on the second processor 1203;

[0329] The second processor 1203 is configured to, when running the computer program, execute:

[0330] Determine the characteristic parameters of the point cloud to be evaluated;

[0331] determining model parameters of the quality assessment model;

[0332] The subjective quality measurement value of the point cloud to be evaluated is determined using the quality evaluation model according to the model parameters and the characteristic parameters of the point cloud to be evaluated.

[0333] Optionally, as another embodiment, the second processor 1203 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0334] It can be understood that the hardware functions of the second memory 1202 and the first memory 1002 are similar, and the hardware functions of the second processor 1203 and the first processor 1003 are similar; they will not be described in detail here.

[0335] This embodiment provides an encoder, which may include a second determination unit and a second calculation unit. In the encoder, the quality assessment model can be used to improve the accuracy of subjective quality assessment; and in the quality assessment model, the technical solution of the present application only needs to use the original point cloud and the feature values ​​extracted from the original point cloud, and no longer needs the distorted point cloud and the matching point pairs of the original point cloud and the distorted point cloud, thereby simplifying the computational complexity of the subjective quality assessment.

[0336] It should be noted that, in this application, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0337] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0338] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0339] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0340] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0341] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A point cloud quality assessment method, characterized in that: Applied to a decoder or a media data processing device, the method comprises: Parse the code stream to obtain the characteristic parameters of the point cloud to be evaluated; determining model parameters of the quality assessment model; The subjective quality measurement value of the point cloud to be evaluated is determined using the quality evaluation model according to the model parameters and the characteristic parameters of the point cloud to be evaluated.

2. The method according to claim 1, characterized in that The characteristic parameters of the point cloud to be evaluated include quantization parameters of the point cloud to be evaluated; wherein the quantization parameters include geometric quantization parameters and color quantization parameters of the point cloud to be evaluated.

3. The method according to claim 1, characterized in that: The step of determining the model parameters of the quality assessment model comprises: Obtain a subjective quality test dataset; Based on the subjective quality test data set, a model parameter function is fitted; wherein the model parameter function is used to reflect the corresponding relationship between the model parameters and the characteristic parameters; The model parameters are calculated according to the acquired characteristic parameters and the model parameter function.

4. The method according to claim 1, characterized in that The step of determining the model parameters of the quality assessment model comprises: Model parameters for the point cloud to be evaluated are selected from one or more preset groups of candidate quality assessment model parameters.

5. The method according to claim 1, characterized in that The step of determining the model parameters of the quality assessment model comprises: Using the first calculation sub-model to perform feature extraction on the point cloud to be evaluated, to obtain a first feature value of the point cloud to be evaluated; Using the second computing sub-model to perform feature extraction on the point cloud to be evaluated, to obtain a second feature value of the point cloud to be evaluated; Determining the model parameters according to the first eigenvalue, the second eigenvalue and a preset vector matrix; The first calculation sub-model represents the extraction of eigenvalues ​​related to color fluctuations on geometric distances from the point cloud to be evaluated, and the second calculation sub-model represents the extraction of eigenvalues ​​related to the average variance of color blocks from the point cloud to be evaluated.

6. The method according to claim 5, characterized in that The step of extracting features from the point cloud to be evaluated by using the first computing sub-model to obtain a first feature value of the point cloud to be evaluated includes: Calculating first eigenvalues ​​corresponding to one or more points in the point cloud to be evaluated; A weighted mean is calculated for the first eigenvalues ​​corresponding to the one or more points, and the obtained weighted mean is determined as the first eigenvalue of the point cloud to be evaluated.

7. The method according to claim 6, characterized in that The calculating the first eigenvalues ​​corresponding to one or more points in the point cloud to be evaluated includes: For a current point in the point cloud to be evaluated, determine a set of neighboring points associated with the current point; wherein the set of neighboring points includes at least one neighboring point; For the neighbor point set, calculate the color intensity difference between the current point and the at least one neighbor point at a unit distance to obtain at least one color intensity difference at a unit distance; A weighted average of the color intensity differences over the at least one unit distance is calculated to obtain a first eigenvalue corresponding to the current point.

8. The method according to claim 7, characterized in that The calculating the color intensity difference between the current point and the at least one neighboring point at a unit distance includes: Obtaining a first color intensity value of a first color component of the current point and a second color intensity value of a first color component of at least one neighboring point; Calculating an absolute value of a difference between a first color intensity value of the current point and a second color intensity value of the at least one neighboring point to obtain a color intensity difference between the current point and the at least one neighboring point; The color intensity difference between the current point and the at least one neighboring point per unit distance is determined according to the color intensity difference between the current point and the at least one neighboring point and the distance between the current point and the at least one neighboring point.

9. The method according to claim 5, characterized in that The step of extracting features from the point cloud to be evaluated by using the second computing sub-model to obtain a second feature value of the point cloud to be evaluated includes: Calculating second eigenvalues ​​corresponding to one or more non-empty voxel blocks in the point cloud to be evaluated; A weighted mean is calculated for the second eigenvalues ​​corresponding to the one or more non-empty voxel blocks, and the obtained weighted mean is determined as the second eigenvalue of the point cloud to be evaluated.

10. The method according to claim 9, characterized in that The calculating the second eigenvalues ​​corresponding to one or more non-empty voxel blocks in the point cloud to be evaluated includes: For a current non-empty voxel block in the point cloud to be evaluated, obtaining a third color intensity value of a first color component of at least one point in the current non-empty voxel block; Calculating a weighted average of the third color intensity values ​​of at least one point in the current non-empty voxel block to obtain a color intensity average value of the current non-empty voxel block; For at least one point in the current non-empty voxel block, determine a color standard deviation of the at least one point by using the third color intensity value and an average color intensity value of the current non-empty voxel block; A weighted mean of the color standard deviation of the at least one point is calculated to obtain a second eigenvalue corresponding to the current non-empty voxel block.

11. The method according to claim 5, characterized in that The method further comprises: Obtain a subjective quality test dataset; The subjective quality test data set is trained to obtain the preset vector matrix.

12. The method according to claim 5, characterized in that The method further comprises: The preset vector matrix for determining the model parameters is selected from one or more preset sets of candidate vector matrices.

13. The method according to claim 5, characterized in that The determining the model parameters according to the first eigenvalue, the second eigenvalue and a preset vector matrix includes: Constructing a eigenvector based on a preset constant value, the first eigenvalue, and the second eigenvalue; Performing a multiplication operation on the feature vector and the preset vector matrix to obtain a model parameter vector; wherein the model parameter vector includes a first model parameter, a second model parameter and a third model parameter; The first model parameter, the second model parameter and the third model parameter are determined as the model parameters.

14. The method according to claim 13, characterized in that The preset constant value is an integer.

15. A point cloud quality assessment method, characterized in that: Applied to an encoder or a media data processing device, the method comprises: Determine the characteristic parameters of the point cloud to be evaluated; determining model parameters of the quality assessment model; The subjective quality measurement value of the point cloud to be evaluated is determined using the quality evaluation model according to the model parameters and the characteristic parameters of the point cloud to be evaluated.

16. The method according to claim 15, characterized in that The step of determining the characteristic parameters of the point cloud to be evaluated includes: Obtaining precoding parameters of the point cloud to be evaluated; According to the pre-coding parameters and a preset lookup table, characteristic parameters of the point cloud to be evaluated are determined; wherein the preset lookup table is used to reflect the corresponding relationship between the coding parameters and the characteristic parameters.

17. The method according to claim 15 or 16, characterized in that The characteristic parameters of the point cloud to be evaluated include quantization parameters of the point cloud to be evaluated; wherein the quantization parameters include geometric quantization parameters and color quantization parameters of the point cloud to be evaluated.

18. The method according to claim 15, characterized in that The step of determining the model parameters of the quality assessment model comprises: Obtain a subjective quality test dataset; Based on the subjective quality test data set, a model parameter function is fitted; wherein the model parameter function is used to reflect the corresponding relationship between the model parameters and the characteristic parameters; The model parameters are calculated according to the acquired characteristic parameters and the model parameter function.

19. The method according to claim 15, characterized in that The step of determining the model parameters of the quality assessment model comprises: Model parameters for the point cloud to be evaluated are selected from one or more preset groups of candidate quality assessment model parameters.

20. The method according to claim 15, characterized in that The step of determining the model parameters of the quality assessment model comprises: Using the first calculation sub-model to perform feature extraction on the point cloud to be evaluated, to obtain a first feature value of the point cloud to be evaluated; Using the second computing sub-model to perform feature extraction on the point cloud to be evaluated, to obtain a second feature value of the point cloud to be evaluated; Determining the model parameters according to the first eigenvalue, the second eigenvalue and a preset vector matrix; The first calculation sub-model represents the extraction of eigenvalues ​​related to color fluctuations on geometric distances from the point cloud to be evaluated, and the second calculation sub-model represents the extraction of eigenvalues ​​related to the average variance of color blocks from the point cloud to be evaluated.

21. The method according to claim 20, characterized in that The step of extracting features from the point cloud to be evaluated by using the first computing sub-model to obtain a first feature value of the point cloud to be evaluated includes: Calculating first eigenvalues ​​corresponding to one or more points in the point cloud to be evaluated; A weighted mean is calculated for the first eigenvalues ​​corresponding to the one or more points, and the obtained weighted mean is determined as the first eigenvalue of the point cloud to be evaluated.

22. The method according to claim 21, characterized in that The calculating the first eigenvalues ​​corresponding to one or more points in the point cloud to be evaluated includes: For a current point in the point cloud to be evaluated, determine a set of neighboring points associated with the current point; wherein the set of neighboring points includes at least one neighboring point; For the neighbor point set, calculate the color intensity difference between the current point and the at least one neighbor point at a unit distance to obtain at least one color intensity difference at a unit distance; A weighted average of the color intensity differences over the at least one unit distance is calculated to obtain a first eigenvalue corresponding to the current point.

23. The method according to claim 22, characterized in that The calculating the color intensity difference between the current point and the at least one neighboring point at a unit distance includes: Obtaining a first color intensity value of a first color component of the current point and a second color intensity value of a first color component of at least one neighboring point; Calculating an absolute value of a difference between a first color intensity value of the current point and a second color intensity value of the at least one neighboring point to obtain a color intensity difference between the current point and the at least one neighboring point; The color intensity difference between the current point and the at least one neighboring point per unit distance is determined according to the color intensity difference between the current point and the at least one neighboring point and the distance between the current point and the at least one neighboring point.

24. The method according to claim 20, characterized in that The step of extracting features from the point cloud to be evaluated by using the second computing sub-model to obtain a second feature value of the point cloud to be evaluated includes: Calculating second eigenvalues ​​corresponding to one or more non-empty voxel blocks in the point cloud to be evaluated; A weighted mean is calculated for the second eigenvalues ​​corresponding to the one or more non-empty voxel blocks, and the obtained weighted mean is determined as the second eigenvalue of the point cloud to be evaluated.

25. The method according to claim 24, characterized in that The calculating the second eigenvalues ​​corresponding to one or more non-empty voxel blocks in the point cloud to be evaluated includes: For a current non-empty voxel block in the point cloud to be evaluated, obtaining a third color intensity value of a first color component of at least one point in the current non-empty voxel block; Calculating a weighted average of the third color intensity values ​​of at least one point in the current non-empty voxel block to obtain a color intensity average value of the current non-empty voxel block; For at least one point in the current non-empty voxel block, determine a color standard deviation of the at least one point by using the third color intensity value and an average color intensity value of the current non-empty voxel block; A weighted mean of the color standard deviation of the at least one point is calculated to obtain a second eigenvalue corresponding to the current non-empty voxel block.

26. The method according to claim 20, characterized in that The method further comprises: Obtain a subjective quality test dataset; The subjective quality test data set is trained to obtain the preset vector matrix.

27. The method according to claim 20, characterized in that The method further comprises: The preset vector matrix for determining the model parameters is selected from one or more preset sets of candidate vector matrices.

28. The method according to claim 20, characterized in that The determining the model parameters according to the first eigenvalue, the second eigenvalue and a preset vector matrix includes: Constructing a eigenvector based on a preset constant value, the first eigenvalue, and the second eigenvalue; Performing a multiplication operation on the feature vector and the preset vector matrix to obtain a model parameter vector; wherein the model parameter vector includes a first model parameter, a second model parameter and a third model parameter; The first model parameter, the second model parameter and the third model parameter are determined as the model parameters.

29. The method according to claim 28, characterized in that The preset constant value is an integer.

30. A decoder, characterized in that: The decoder comprises a parsing unit, a first determining unit and a first calculating unit; wherein, The parsing unit is configured to parse the code stream to obtain feature parameters of the point cloud to be evaluated; The first determining unit is configured to determine a model parameter of a quality assessment model; The first calculation unit is configured to determine a subjective quality measurement value of the point cloud to be evaluated using the quality assessment model according to the model parameters and the feature parameters of the point cloud to be evaluated.

31. A decoder, characterized in that: The decoder comprises a first memory and a first processor; wherein, The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 1 to 14 when running the computer program.

32. An encoder, characterized in that The encoder comprises a second determining unit and a second calculating unit; wherein, The second determination unit is configured to determine characteristic parameters of the point cloud to be evaluated; The second determining unit is further configured to determine model parameters of the quality assessment model; The second calculation unit is configured to determine a subjective quality measurement value of the point cloud to be evaluated using the quality assessment model according to the model parameters and the feature parameters of the point cloud to be evaluated.

33. An encoder, characterized in that: The encoder includes a second memory and a second processor; wherein, The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 15 to 29 when running the computer program.

34. A computer storage medium, characterized in that The computer storage medium stores a computer program, which implements the method according to any one of claims 1 to 14 when executed by a first processor, or implements the method according to any one of claims 15 to 29 when executed by a second processor.