A method and system for evaluating the quality of a three-dimensional point cloud

Abstract

The application discloses a three-dimensional point cloud quality evaluation method and system, comprising the following steps: S1, acquiring point cloud information, inputting to a preset noise simulation function, and generating random point cloud noise; S2, constructing a point cloud scoring database based on the generated random point cloud noise and a preset point cloud quality evaluation method; S3, constructing a point cloud quality evaluation model based on the point cloud scoring database; S4, inputting the point cloud information into the point cloud quality evaluation model, and outputting the quality score of the point cloud as a point cloud quality evaluation result. Compared with the traditional technology, the application can quickly and accurately approximate the quality score of the point cloud under the simplified framework of the graph neural network, can generate a large-scale point cloud quality evaluation database, and can better train the deep model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multimedia information processing, and more particularly, to a three-dimensional point cloud quality evaluation method and system. BACKGROUND

[0002] Due to the widespread use of high-performance handheld depth sensor devices, especially with the development of virtual reality (VR) and augmented reality (AR) applications, the demand for 3D content acquisition, processing and display has increased dramatically. Among existing 3D content solutions, point cloud-based 3D content expression methods have attracted much attention. Point cloud is a flexible and efficient digital representation of objects in the real physical world. It not only contains the three-dimensional spatial position information of points on the surface of a real object, but also carries other physical attributes of the object, such as point normal information, point color information and texture coordinate information, etc. Essentially, point cloud can be regarded as an organized or unorganized data structure, which can be obtained from the surface of a regular or irregular three-dimensional model through sampling. Discrete point cloud data is a highly flexible way to describe three-dimensional object representation, which can edit point cloud models through point-by-point adjustment. As a kind of unstructured and non-connected discrete data, point cloud has great potential in various real-world applications, such as autonomous driving, augmented / virtual reality, scene understanding and immersive communication, etc.

[0003] Point cloud quality evaluation methods can be divided into subjective evaluation methods and objective evaluation methods according to the way of point cloud data quality scoring. From the perspective of evaluation function calculation, all methods can be classified into full-reference model evaluation methods, half-reference model evaluation methods and no-reference model evaluation methods. In recent years, JPEG and MPEG standardization organizations have launched several discussions on point cloud visual data, and MPEG has given the first point cloud data compression standard. However, the objective point cloud quality evaluation methods and no-reference model evaluation methods developed so far are limited, and these methods all have their own weaknesses, so the actual application needs are accelerating the development of researchers to develop better performance no-reference model objective color point cloud quality evaluation algorithm.

[0004] Influenced by the great success of deep learning in image and video processing, many researchers consider introducing deep learning methods into the field of geometry processing. However, the digital representation of geometric objects is either mesh or point cloud, which are unstructured and unordered non-Euclidean data organization methods, so it is difficult to process directly. Su et al. project 3D models from different angles to a plane, and then extract the features of 3D models under different viewing angles based on the CNN method of the plane image to achieve the purpose of identification. The main disadvantage of this method is that the information will be lost due to the occlusion problem, which cannot fully extract the features, and the calculation cost is huge. The core idea of the voxelization method of three-dimensional point cloud data is to convert the input three-dimensional shape data representation into a standard three-dimensional solid data representation, and then perform convolution operations on regular, fixed-size 3D voxels to perform model recognition and segmentation tasks. Multi-view technology and voxelization technology both require cutting three-dimensional models, and point-based deep learning methods directly operate on point set positions, which can extract better features and thus obtain better classification and recognition results. The representative methods among them are Pointnet, Pointnet++, and PointCNN, but the results of these methods are closely related to the sampling method of points, and generally require the number of points in the 3D model to be strictly consistent, thus limiting its application.

[0005] Graph convolution network (GCN) is another flexible method for processing non-Euclidean data. Generally, for discrete three-dimensional point cloud data, a graph needs to be constructed through K-neighbor method, and then various tasks in point cloud are processed based on graph convolution network. The first graph-based point cloud processing method directly processes on the graph, and its main disadvantage is that the extracted features are local. Another graph-based point cloud processing method is to process in the spectral domain, that is, in order to process the underlying graph data structure, the graph Fourier transform based on the theory of graph signal processing is proposed, which aims to convert the signal in the spatial domain to the spectral domain, and then design algorithms in the spectral domain to complete various graph processing tasks. Bruna et al. first proposed the spectral graph convolution network algorithm on the graph, and then ChebyNets and graph convolution network based on first-order Chebyshev polynomial were proposed to reduce the inefficiency problem of Laplacian matrix eigenvalue decomposition in spectral graph network. More efficient variants of GCN such as importance sampling, importance sampling and aggregation combined method, high-order propagation of multi-scale information, and simplified GCN have been proposed. One of the great advantages of converting point cloud data to the spectral domain is that point cloud is free from various constraints in the spatial domain, especially the processing difficulty of the unordered nature and rotation invariance of point cloud in the spatial domain, which provides a new computing tool for the processing of non-Euclidean geometric data.

[0006] There are many related researches on point cloud quality evaluation, such as evaluation protocol, user interaction mechanism, distortion loss and objective metric, etc. From the way of point cloud data quality scoring, all methods can be classified into subjective evaluation method and objective evaluation method; from the perspective of evaluation function calculation, they can be divided into full-reference model evaluation method, semi-reference model evaluation method and no-reference model evaluation method. The following mainly introduces the point cloud quality evaluation algorithm based on the angle of evaluation function. The full-reference model evaluation method is the most widely studied technology, and very good results have been achieved. Some representative methods have been added to the MPEG compression standard, including a series of evaluation methods such as point to point (p2point), point to plane (p2plane) and point to mesh (p2mesh), and the basic idea is to calculate the Hausdorff distance or calculate the PSNR value through MSE. AngSim method measures the quality by measuring the difference of point normal, and Javaheri et al. propose a generalized Hausdorff distance to calculate the point cloud quality. These methods only consider the deformation of spatial position, and do not consider that the human eye system is more sensitive to color and model structure deformation, so PCQM and GraphSim are proposed to better solve the quality evaluation problem. Some methods project the model to the plane in a multi-view way, so as to obtain multiple model photos at different angles, and then complete the quality evaluation of point cloud based on the image quality evaluation method, but the projection method generally causes information loss, which reduces the reliability of the quality evaluation method. In addition, there are also methods for calculating the features of points by calculating the geodesic feature histogram, and then comparing the histogram features of the corresponding points of the reference point cloud and the target point cloud to calculate the quality of the point cloud, but this method calculates the local features and does not consider the global features, so that the quality evaluation score is unreliable. The research on semi-reference model evaluation method is relatively less, which needs to use part of the information of the model and the whole model for matching. The no-reference model point cloud quality evaluation method generally needs to establish a database, and then build an evaluation function based on the regression method. The point cloud quality score database currently published includes SJTU-PCQA and IRPC of Shanghai Jiaotong University, but the data contained is less. Based on the large-scale point cloud quality score database, the sparse CNN regression point cloud quality evaluation method proposed by Liu et al. is currently the most representative no-model evaluation method.

[0007] The prior art discloses a full-reference 3D point cloud quality evaluation method based on point feature histogram geodesic distance. The method calculates the features of points by calculating the geodesic feature histogram, and then compares the histogram features of the corresponding points of the reference point cloud and the target point cloud to calculate the quality of the point cloud. The defect of this method is that the local features are calculated and the global features are not considered, so that the quality evaluation score is unreliable.

[0008] To this end, in combination with the above needs and defects of the prior art, the present application proposes a three-dimensional point cloud quality evaluation method and system. SUMMARY

[0009] The present application provides a three-dimensional point cloud quality evaluation method and system, establishes a large-scale point cloud quality scoring database, which can support the training of a large-scale graph convolution network, and trains a fast dynamic graph convolution network function, which can accurately approximate the quality score of the point cloud model to avoid designing a manual evaluation function.

[0010] The primary object of the present application is to solve the above technical problems, and the technical solution of the present application is as follows:

[0011] The present application provides a three-dimensional point cloud quality evaluation method, which comprises the following steps:

[0012] S1, obtaining point cloud information and inputting it to a preset noise simulation function to generate random point cloud noise.

[0013] S2, constructing a point cloud scoring database based on the generated random point cloud noise and a preset point cloud quality evaluation method.

[0014] S3, constructing a point cloud quality evaluation model based on the point cloud scoring database.

[0015] S4, inputting the point cloud information into the point cloud quality evaluation model and outputting the quality score of the point cloud as the point cloud quality evaluation result.

[0016] Further, the point cloud information is obtained from a public color database.

[0017] Further, the preset noise simulation function is a Gaussian mixture model, and the specific expression is:

[0018]

[0019]

[0020] where g(z) represents the Gaussian mixture model, K is the number of Gaussian functions used, z is the data point, with mean μ k , covariance matrix σ k is a Gaussian distribution function in d-dimensional space, α k is a non-negative weight, θ=(μ1,σ2,...,μ K ,σ K ), T represents the transpose symbol, and d represents the space dimension.

[0021] where (z1,z2,...,z n), according to the selected K value, (z1, z2,..., z n ) are aggregated into K clusters by K-means clustering method, and then the mean and variance of the K clusters are calculated as the values of (μ k ,σ k ), and then α k is randomly assigned on the interval [0, 1] and ∑α k = 1. Based on this method, various unknown distribution function noises can be simulated, and the noise simulation function design is completed.

[0022] Further, the preset point cloud quality evaluation method includes a full reference model point cloud quality evaluation method and a no reference model point cloud quality evaluation method.

[0023] The full reference model point cloud quality evaluation method is a public point cloud quality evaluation method.

[0024] Further, the no reference model point cloud quality evaluation method specifically is: the Mahalanobis distance is used to measure the distance between the vertices with color, and the K-neighbor method is used to construct a graph matrix, so as to fuse the position information and color information of the point cloud; the expression of the Mahalanobis distance d M is as follows:

[0025] d M (P i ,P j ) = (P i -P j ) T M(P i -P j )

[0026] P i = (x i ,y i ,z i ,r i ,g i ,b i )

[0027] P j = x j ,y j ,z j ,r j ,g j ,b j )

[0028] Wherein, M represents the Mahalanobis matrix, P i and P j are any two vertices with color in the point cloud, x i , y i , z i and xj y j z j are position information of vertexes P i and P j respectively, r i , g i , b i and r j , g j , b j are color information of vertexes P i and P j respectively.

[0029] According to the above technical features, the geometric information and color information of the point cloud ultimately determine the subjective score result of the point cloud model, and the objective score result of the quality evaluation of the point cloud model should be consistent with the subjective score result, so the color information should be fully considered when designing the quality evaluation method of the point cloud. The no-reference model point cloud quality evaluation method adopted by the application solves the problem that the public no-reference model point cloud quality evaluation method cannot fully consider the color information.

[0030] Further, the process of constructing the point cloud scoring database is: inputting the public point cloud information into the noise simulation function, substituting the random noise generated by the function into the position information and color information of the point cloud quality evaluation method respectively, calculating the signal-to-noise ratio value of the output result and the original point cloud information as the quality score of the noise point cloud information, thereby obtaining the point cloud scoring database.

[0031] Further, the point cloud quality evaluation model is a graph convolutional neural network, a multi-layer perception network or a graph neural network, and the process of constructing the point cloud quality evaluation model based on the point cloud scoring database includes updating the nodes of the network, specifically including: feature propagation, linear transformation and point-by-point activation.

[0032] Further, the feature propagation is specifically:

[0033] At the beginning of each layer of the point cloud quality evaluation model, the feature of node v i is the mean value of the local neighborhood node feature vector of v i :

[0034]

[0035] wherein d i denotes the degree of vertex v i , i = 1, 2,..., n, a ij denotes the weight of the edge composed of v i and v j , denotes the activation output of the k-1th graph convolutional layer, The feature propagation output result of the k-1th graph convolution layer is represented.

[0036] The expression of the Laplacian matrix S normalized to the graph matrix constructed based on the K-neighbor method of the input point cloud is as follows:

[0037]

[0038]

[0039] Wherein, A represents the graph matrix of the input point cloud, I is a unit matrix, is the degree matrix of , and The output after the feature propagation of the kth graph neural network is represented.

[0040] According to the above technical features, for the kth graph convolution layer, H (k) represents the feature representation of all nodes, and the input is the output H (k-1) of the k-1th graph convolution layer, (0) wherein H (k) = X, X represents the input of the network. The graph convolution neural network with k layers is similar to the k-layer multi-layer perception network, and the main difference is that the implicit representation of each layer node is the average value of the neighborhood of the node.

[0041] Further, the expression of the linear transformation and the point-by-point activation is as follows:

[0042]

[0043] Wherein, after omitting the activation operation, the graph convolution neural network with k layers is represented as:

[0044] H (k) = SS…SXθ (0) θ (1) …θ (k-1) = S k Xθ

[0045] Wherein X represents the input feature vector of the network.

[0046] The process of integrating the Laplacian matrix S and the Mahalanobis distance d M is as follows: the Mahalanobis matrix M (k) is constructed after learning θ of each layer, the Laplacian matrix is dynamically constructed through the Mahalanobis distance, the position information and the color information are fused, and the specific expression is as follows:

[0047] H (k) = S (k-1) S (k-2) ...S (0) Xθ

[0048] H (k) =S (k-1) S (k-2) ...S (0) Xθ (0) θ (1) ...θ (k-1)

[0049] M (k) =θ (k) (θ (k) ) T

[0050] Among them, S (k) It is based on the learned H (k-1) Dynamically generated normalized Laplace matrix.

[0051] Furthermore, when the number of vertices in the point cloud input to the point cloud quality assessment model is inconsistent with the number of vertices in the model network, the model executes a multi-resolution point cloud processing procedure.

[0052] Furthermore, the multi-resolution point cloud processing flow is specifically as follows:

[0053] When the number of vertices in the input point cloud is less than the number of vertices in the model network, an implicit representation of the point cloud is constructed using the signed distance function:

[0054]

[0055]

[0056] Where, n p Let r be the unit normal vector of point p, and r > 0 represent the bandwidth. Let w represent the gradient operator. i These are weight parameters; for a pre-defined set of n discrete points (p1, p2, ..., p... n The parameter w is determined by minimizing the following objective function. i :

[0057]

[0058] Once parameter w i Once determined, multiple sets of points are collected from the implicit surface using the method of sampling from the farthest point. Each set of points contains n points. A quality score is calculated for each set of points, and the average is taken as the final evaluation quality.

[0059] When the number of vertices in the input point cloud is greater than the number of vertices in the model network, the farthest point sampling method is used to collect multiple sets of points. The number of points in each set is n. Then, the quality score is calculated for each set of points, and the average is used as the final evaluation quality.

[0060] The second aspect of the present application provides a three-dimensional point cloud quality evaluation system, which is used for the three-dimensional point cloud quality evaluation method, and comprises a point cloud noise generation module, a data processing module and a point cloud quality evaluation module.

[0061] The point cloud information is input into the point cloud noise generation module, the generated point cloud noise is input into the data processing module, the data processing module constructs a point cloud score database based on the random point cloud noise and a preset point cloud quality evaluation method, a point cloud quality evaluation model is constructed according to the point cloud score database and saved to the point cloud quality evaluation module, the point cloud information is input into the point cloud quality evaluation module, and the quality score of the point cloud is output as a point cloud quality evaluation result.

[0062] Compared with the prior art, the beneficial effects of the technical scheme of the present application are:

[0063] The present application provides a three-dimensional point cloud quality evaluation method and system, a point cloud score database constructed by a preset point cloud quality evaluation method can achieve fusion of position information and color information, a point cloud quality evaluation model constructed based on the point cloud score database can quickly and accurately approximate the quality score of the point cloud, and simultaneously supports point cloud quality evaluation of any vertex number. BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 A flowchart of the three-dimensional point cloud quality evaluation method of the present application.

[0065] Figure 2 A flowchart in an embodiment of the present application.

[0066] Figure 3 A schematic diagram of the three-dimensional point cloud quality evaluation system of the present application. DETAILED DESCRIPTION

[0067] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application will be further described in detail below in combination with the drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0068] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, however, the present application can also be implemented in other ways different from those described herein, therefore, the protection scope of the present application is not limited by the specific embodiments disclosed below.

[0069] Embodiment 1

[0070] As shown in Figure 1 The present application provides a three-dimensional point cloud quality evaluation method, which comprises the following steps:

[0071] S1, acquire point cloud information, input to a preset noise simulation function, generate random point cloud noise.

[0072] S2, based on the generated random point cloud noise and the preset point cloud quality evaluation method, construct a point cloud score database.

[0073] S3, based on the point cloud score database, construct a point cloud quality evaluation model.

[0074] S4, input the point cloud information to the point cloud quality evaluation model, output the quality score of the point cloud as the point cloud quality evaluation result.

[0075] Further, the point cloud information is obtained from a public color database.

[0076] It should be noted that all the point clouds collected by sensors or processed point clouds have noise, mainly including point cloud position disturbance and abnormal points. That is, any point on the point cloud has a certain probability of appearing in the area near the original position, or appearing in any position in space (abnormal points). Because there are many mature methods (such as statistical filtering method) to remove abnormal value points, this project does not consider point cloud models with abnormal values.

[0077] Point cloud noise is random, that is, the distribution and intensity of noise is irregular; noise is superimposable, that is, the noise introduced by different stages of processing can be power-added, which will cause the signal-to-noise ratio to decrease in turn. Point cloud noise will drown the point cloud features, making it difficult to analyze the point cloud. Currently, point cloud analysis mainly focuses on modeling noise in the acquisition process, that is, the noise distribution caused by device precision. For example, existing research 1 considers that the noise introduced by Kinect sensor is Gauss noise, while the noise of KonicaMinolta Vivid 910 sensor is not Gauss noise and does not have an independent distribution, so the modeling of this kind of noise is based on the Gauss-like distribution of Fourier spectrum of 3D data. However, the noise in the process of point cloud processing and transmission does not necessarily follow a Gaussian distribution, so this study models the noise as follows: Gauss noise, Rayleigh noise, Gamma noise, exponential distribution noise, uniform noise and multiplicative noise, etc.

[0078] In order to model these situations, actual noise needs to be analyzed. Assuming that the above noise generation function cannot express the data, the noise can be modeled as a Gaussian mixture model g(z), so that the probability function that cannot be represented by the above distribution can be modeled, and the theoretical basis is that the Gaussian mixture model can approximate any probability distribution function.

[0079] Further, the preset noise simulation function is a Gaussian mixture model, and the specific expression is:

[0080]

[0081]

[0082] where g(z) denotes a Gaussian mixture model, K is the number of Gaussian functions adopted, z is a data point, is a d-dimensional Gaussian distribution function with mean μ k and covariance matrix σ k , α k is a non-negative weight, θ = (μ1, σ2,..., μ K , σ K ), T denotes a transpose symbol, and d denotes a space dimension.

[0083] where (z1, z2,..., z n ) are observed data, (z1, z2,..., z n ) are aggregated into K clusters using a K-means clustering method according to a selected K value, and then the mean and variance of the K clusters are calculated as values of (μ k , σ k ). α k is randomly assigned on an interval [0, 1] and ∑α k = 1. Based on this method, various unknown distribution function noises can be simulated, and a noise simulation function design is completed.

[0084] In a specific embodiment, once the noise generation simulation function is modeled, a required point cloud score database can be generated from an existing database. Currently, a publicly available high-resolution geometric color database is a HighResGC database, which includes MPEG, JPEG, Pleno, PointXR, VSENSE, M-PCCD, SJTU-PCQA, and IRPC, etc. Based on these databases, random noise can be generated using the above-mentioned probability distribution function, and then the random noise is added to the position and color components of the model. Then, a point cloud quality evaluation method based on an existing full reference model can be used to model a point cloud score database under a multi-resolution condition.

[0085] Further, the preset point cloud quality evaluation method includes a full reference model point cloud quality evaluation method and a no reference model point cloud quality evaluation method.

[0086] where the full reference model point cloud quality evaluation method is a publicly available point cloud quality evaluation method.

[0087] Further, the no-reference model point cloud quality evaluation method is specifically: Mahalanobis distance is used to measure the distance between vertices with color, and K-neighbor method is used to construct a graph matrix, so as to fuse the position information and color information of the point cloud; the expression of the Mahalanobis distance d M is as follows:

[0088] d M (P i ,P j )=(P i -P j ) T M(P i -P j )

[0089] P i =(x i ,y i ,z i ,r i ,g i ,b i )

[0090] P j =(x j ,y j ,z j ,r j ,g j ,b j )

[0091] wherein M represents a Mahalanobis matrix, P i and P j are any two vertices with color in the point cloud, x i , y i , z i and x j , y j , z j are position information of the vertices P i and P j respectively, and r i , g i , b i and r j , g j , b j are color information of the vertices P i and P j respectively.

[0092] Based on the aforementioned technical characteristics, the geometric and color information of a point cloud ultimately determines the subjective scoring result of the point cloud model. The objective scoring result of the point cloud model's quality assessment should be consistent with the subjective scoring result. Therefore, color information must be fully considered when designing a point cloud quality assessment method. The reference-free model point cloud quality assessment method adopted in this invention solves the problem that publicly available reference-free model point cloud quality assessment methods cannot fully consider color information.

[0093] Furthermore, the process of constructing the point cloud scoring database is as follows: inputting the publicly available point cloud information into the noise simulation function, substituting the random noise generated by the function into the position information and color information of the point cloud quality assessment method, calculating the signal-to-noise ratio of the output result and the original point cloud information as the quality score of the noisy point cloud information, thereby obtaining the point cloud scoring database.

[0094] Furthermore, the point cloud quality assessment model is a graph convolutional neural network, a multilayer perceptron network, or a graph neural network. The process of constructing the point cloud quality assessment model based on the point cloud scoring database includes updating the network nodes, specifically including feature propagation, linear transformation, and point-by-point activation.

[0095] Furthermore, the feature propagation specifically includes:

[0096] At the beginning of each layer of the point cloud quality assessment model, node v i The characteristic of a is the mean of the feature vectors of its local neighborhood nodes:

[0097]

[0098] Where, d i Represents vertex v i The degree, i = 1, 2, ..., n, a ij Indicates by v i and v j The weight of the edges formed, This represents the activation output of the (k-1)th graph convolutional layer. This represents the feature propagation output of the (k-1)th graph convolutional layer.

[0099] The expression for the normalized Laplacian matrix S of the graph matrix constructed based on the K-nearest neighbor method from the input point cloud is:

[0100]

[0101]

[0102] Where A represents the graph matrix of the input point cloud, I is the identity matrix. yes degree matrix, represents the output of the k-th layer graph neural network after feature propagation.

[0103] According to the above technical features, for the k-th graph convolution layer, H (k) represents the feature representation of all nodes, and the input is the output H (k-1) of the k-1-th graph convolution layer. (0) = X, X represents the input of the network. The graph convolution neural network with k layers is similar to the k-layer multi-layer perception network, and the main difference is that the implicit representation of each layer node is the average value of the neighborhood of the node.

[0104] Further, the expression of the linear transformation and the point-by-point activation is:

[0105]

[0106] wherein the graph convolution neural network with k layers is represented as:

[0107] H (k) = SS…SXθ (0) θ (1) …θ (k-1) = S k Xθ

[0108] wherein X represents the input feature vector of the network.

[0109] The process of integrating the Laplacian matrix S and the Mahalanobis distance d M is as follows: after learning θ of each layer, the Mahalanobis matrix M (k) is constructed, the Laplacian matrix is dynamically constructed through the Mahalanobis distance, and the position information and color information are fused, and the specific expression is as follows:

[0110] H (k) = S (k-1) S (k-2) ...S (0) Xθ

[0111] H (k) = S (k-1) S (k-2) ...S (0) Xθ (0) θ (1) ...θ (k-1)

[0112] M (k) = θ (k) (θ (k) ) T

[0113] wherein S (k) is according to the learned H(k-1) The dynamically established normalized Laplacian matrix.

[0114] Further, when the number of vertices of the point cloud input to the point cloud quality evaluation model is inconsistent with the number of vertices of the model network, the model performs a multi-resolution point cloud processing procedure.

[0115] Further, the multi-resolution point cloud processing procedure specifically includes:

[0116] When the number of vertices of the input point cloud is less than the number of vertices of the model network, an implicit representation of the point cloud is constructed using a signed distance function:

[0117]

[0118]

[0119] where n p is the unit normal vector of point p, r>0 represents the bandwidth, represents the gradient operator, w i is a weight parameter; for a preset n discrete point set (p1, p2,..., pn), the parameter w n is determined by minimizing the following objective function: i

[0120]

[0121] Once the parameter w i is determined, a plurality of point sets are collected from the implicit surface according to the farthest point sampling method, the number of points in each point set is n, the quality score of each point set is calculated, and the average is taken as the final evaluation quality.

[0122] When the number of vertices of the input point cloud is greater than the number of vertices of the model network, a plurality of point sets are collected using the farthest point sampling method, the number of points in each point set is n, and then the quality score of each point is calculated and averaged as the final evaluation quality.

[0123] Embodiment 2

[0124] Based on the above embodiment 1, in combination Figure 2 with the above description, this embodiment specifically describes the specific process of the point cloud quality evaluation of the present application.

[0125] For example Figure 2 ​As shown, for the quality evaluation of point cloud data, firstly, various point cloud models are collected from the network to form a basic point cloud database, and the point cloud is normalized; then based on the mechanism of point cloud distortion, point cloud noise is simulated, and various noises are added to the original model, and then a point cloud quality score database is constructed based on the point cloud quality evaluation method with reference model; the spatial position information and color information of the point cloud are fused, and then a dynamic graph convolution network is designed, and a point cloud quality evaluation model is trained based on the created database.

[0126] Embodiment 3

[0127] As Figure 3 shown, the application also provides a three-dimensional point cloud quality evaluation system, which is used for the three-dimensional point cloud quality evaluation method, and comprises a point cloud noise generation module, a data processing module and a point cloud quality evaluation module.

[0128] The point cloud information is input to the point cloud noise generation module, the generated point cloud noise is input to the data processing module, the data processing module constructs a point cloud score database based on the random point cloud noise and the preset point cloud quality evaluation method, and a point cloud quality evaluation model is constructed according to the point cloud score database and saved to the point cloud quality evaluation module; the point cloud information is input to the point cloud quality evaluation module, and the quality score of the point cloud is output as the point cloud quality evaluation result.

[0129] In the embodiments provided in the present application, it should be understood that the disclosed system and method can be implemented in other ways. Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by a program instructing related hardware, the foregoing program can be stored in a computer readable storage medium, and the program executes the steps including the above-mentioned method embodiments when executed; and the foregoing storage medium includes a mobile storage device, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk and various media capable of storing program codes.

[0130] Alternatively, the above-mentioned embodiments of the present application can be stored in a computer readable storage medium if implemented in the form of a software function module and sold or used as an independent product. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions for making a computer device.

[0131] All or part of the methods described in the various embodiments of the present application are executed. The aforementioned storage medium includes a mobile storage device, ROM, RAM, a magnetic or optical disk, and the like, and various media capable of storing program codes.

[0132] Obviously, the above-described embodiments of the present application are merely examples for clearly illustrating the present application, and are not intended to limit the implementation modes of the present application. The icons in the drawings for describing the positional relationship of the structures are merely for illustrative purposes, and should not be understood as limiting the present patent. Based on the above description, other different forms of changes or variations can be made by those of ordinary skill in the art. Here, it is not necessary and also impossible to exhaust all the implementation modes. Any modification, equivalent replacement, and improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the claims of the present application.

Claims

1. A method for evaluating the quality of three-dimensional point clouds, characterized in that, Includes the following steps: S1. Obtain point cloud information and input it into a preset noise simulation function to generate random point cloud noise; the point cloud information is obtained from a public color database. When the number of vertices of the point cloud input to the point cloud quality assessment model is inconsistent with the number of vertices of the model network, the model executes a multi-resolution point cloud processing flow. The preset point cloud quality assessment method includes a point cloud quality assessment method with a full reference model and a point cloud quality assessment method without a reference model. Specifically, the point cloud quality assessment method without a reference model involves using Mahalanobis distance to measure the distance between colored vertices and constructing a graph matrix using the K-nearest neighbor method, thereby fusing point cloud position information with color information. The Mahalanobis distance... The expression is: Where M represents the Markov matrix, and Let be any two colored vertices in the point cloud. and Vertices and Location information, and Vertices and Color information; S2. Construct a point cloud scoring database based on the generated random point cloud noise and the preset point cloud quality assessment method; the specific process of constructing the point cloud scoring database is as follows: input the publicly available point cloud information into the noise simulation function, substitute the random noise generated by the function into the position information and color information of the point cloud quality assessment method respectively, calculate the signal-to-noise ratio value of the output result and the original point cloud information as the quality score of the noise point cloud information, thereby obtaining the point cloud scoring database; S3. Construct a point cloud quality assessment model based on a point cloud scoring database; the point cloud quality assessment model is a graph convolutional neural network. The process of constructing the point cloud quality assessment model based on the point cloud scoring database includes updating the nodes of the network, specifically including: feature propagation, linear transformation and point-by-point activation. S4. Input the point cloud information into the point cloud quality assessment model and output the point cloud quality score as the point cloud quality assessment result.

2. The method for evaluating the quality of a three-dimensional point cloud according to claim 1, characterized in that, The preset noise simulation function is a Gaussian mixture model, and its specific expression is as follows: Where g(z) represents the Gaussian mixture model, K is the number of Gaussian functions used, and z is the number of data points. The mean is The covariance matrix is The Gaussian distribution function in d-dimensional space, It is a non-negative weight. =( ) T represents the transpose symbol, and d represents the spatial dimension.

3. The method for evaluating the quality of a three-dimensional point cloud according to claim 2, characterized in that, The feature propagation specifically refers to: At the beginning of each layer of the point cloud quality assessment model, the nodes... The characteristic of a is the mean of the feature vectors of its local neighborhood nodes: in, , Representing vertices respectively and The degree, i, j = 1,2,...,n, Indicates by and The weight of the edges formed, and Representing vertices respectively and The activation output of the (k-1)th graph convolutional layer This represents the feature propagation output of the (k-1)th graph convolutional layer; The expression for the normalized Laplacian matrix S of the graph matrix constructed based on the K-nearest neighbor method from the input point cloud is: Where A represents the graph matrix of the input point cloud, I is the identity matrix. yes The degree matrix, This represents the output of the k-th layer graph neural network after feature propagation.

4. The three-dimensional point cloud quality assessment method according to claim 3, characterized in that, The expressions for the linear transformation and pointwise activation are: With the activation operation omitted, a graph convolutional neural network with k layers is represented as follows: Where X represents the input feature vector of the network; Integrating the Laplacian matrix S and Mahalanobis distance The process is as follows: through learning at each layer... Then construct the Markov matrix The Laplacian matrix is ​​dynamically constructed using Mahalanobis distance to fuse position and color information. The specific expression is shown below: in, It is based on what was learned Dynamically generated normalized Laplace matrix.

5. A three-dimensional point cloud quality assessment according to claim 4, characterized in that, The multi-resolution point cloud processing flow is as follows: When the number of vertices in the input point cloud is less than the number of vertices in the model network, an implicit representation of the point cloud is constructed using the signed distance function: in, It is a point p The unit normal vector, r>0 represents the bandwidth. Represents the gradient operator. These are weight parameters; for a preset set of n discrete points The parameters are determined by minimizing the following objective function. : once parameter Once determined, multiple sets of points are collected from the implicit surface using the method of sampling from the farthest point. The number of points in each set is n. The quality score is calculated for each set of points, and the average is taken as the final evaluation quality. When the number of vertices in the input point cloud is greater than the number of vertices in the model network, the farthest point sampling method is used to collect multiple sets of points. The number of points in each set is n. Then, the quality score is calculated for each set of points, and the average is used as the final evaluation quality.

6. A three-dimensional point cloud quality assessment system, wherein the system is used in the three-dimensional point cloud quality assessment method according to any one of claims 1-5, characterized in that, It includes: a point cloud noise generation module, a data processing module, and a point cloud quality assessment module; Point cloud information is input into the point cloud noise generation module, and the generated point cloud noise is input into the data processing module. The data processing module constructs a point cloud scoring database based on random point cloud noise and a preset point cloud quality assessment method. It constructs a point cloud quality assessment model based on the point cloud scoring database and saves it to the point cloud quality assessment module. Point cloud information is input into the point cloud quality assessment module, and the point cloud quality score is output as the point cloud quality assessment result.

Images