A point cloud quality determination method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of image processing, in particular to a point cloud quality determination method and device, electronic equipment and a storage medium. The method comprises the following steps: a global graph of a to-be-evaluated point cloud is established; the global structure integrity of the point cloud has a certain influence on visual quality; therefore, key points capable of representing the overall structure of the point cloud are determined from the to-be-evaluated point cloud; the global quality feature and the local quality feature of the to-be-evaluated point cloud are determined according to the key points; and the quality score of the to-be-evaluated point cloud is determined according to the global quality feature and the local quality feature corresponding to the to-be-evaluated point cloud, so that the overall-to-local attention feature is met, the consistency between the quality of the to-be-evaluated point cloud and the human eye perception quality is improved, and the objectivity of the perception quality of the evaluated point cloud is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, electronic device and storage medium for determining point cloud quality. Background Technology

[0002] In recent years, the development of acquisition and reconstruction technologies has made it increasingly easier to acquire 3D point clouds. A 3D point cloud is usually composed of tens of thousands of disordered points, each of which consists of 3D geometric coordinates and related attributes (color and normal, etc.).

[0003] Because 3D point clouds can contain tens of thousands of points, quality issues such as noise and distortion may arise during point cloud processing and analysis. Furthermore, to efficiently transmit 3D point clouds, compression encoding is typically required. However, this compression inevitably leads to distortions such as blurred textures and structural holes, directly impacting the accuracy and efficiency of processing and analysis, and reducing the accuracy and reliability of the point cloud data. To address low-quality point cloud data promptly, the perceived quality of 3D point clouds can be evaluated to measure its overall quality. Therefore, objectively evaluating the perceived quality of 3D point clouds is a crucial problem that needs to be solved. Summary of the Invention

[0004] To improve the objectivity of evaluating the perceived quality of point clouds, this application provides a method, apparatus, electronic device, and storage medium for determining point cloud quality.

[0005] Firstly, this application provides a method for determining point cloud quality, employing the following technical solution:

[0006] A method for determining point cloud quality, comprising:

[0007] Establish a global graph of the point cloud to be evaluated;

[0008] Determine the key points corresponding to the global graph, where the key points represent the overall structure of the point cloud;

[0009] Based on the key points, determine the global quality features and local quality features corresponding to the point cloud to be evaluated;

[0010] Based on the global and local quality features of the point cloud to be evaluated, the quality score of the point cloud to be evaluated is determined.

[0011] By adopting the above technical solution, a global map of the point cloud to be evaluated is established. The integrity of the global structure of the point cloud has a certain impact on visual quality. Therefore, key points that can characterize the overall structure of the point cloud are identified from the point cloud to be evaluated. Based on the key points, the global quality features and local quality features of the point cloud to be evaluated are determined. Based on the corresponding global and local quality features, the quality score of the point cloud to be evaluated is determined, thereby satisfying the attention characteristics from the global to the local and improving the consistency between the determined quality of the point cloud to be evaluated and the perceived quality of the human eye, thus improving the objectivity of evaluating the perceived quality of the point cloud.

[0012] In one possible implementation, establishing a global graph of the point cloud to be evaluated includes:

[0013] Obtain the multidimensional vector of each point in the point cloud to be evaluated. The multidimensional vector consists of the three-dimensional spatial coordinates of the point and the color attribute of the point.

[0014] Each point in the point cloud to be evaluated is treated as a node;

[0015] Based on the three-dimensional spatial coordinates of each node, determine the distance between every two nodes;

[0016] Determine the edges based on the distance between any two nodes;

[0017] A global graph of the point cloud to be evaluated is established based on nodes and edges.

[0018] In one possible implementation, determining the key points corresponding to the global graph includes:

[0019] Based on the multidimensional vector of the node, the difference value corresponding to each node is determined. The difference value is used to characterize the degree of difference between the node and its neighboring nodes.

[0020] The difference values ​​corresponding to each node are sorted in descending order to obtain the target sequence;

[0021] Starting from the first difference value in the target sequence, a preset number of difference values ​​are selected as the target set;

[0022] The node corresponding to each difference value in the target set is determined as the key point corresponding to the global graph of the point cloud to be evaluated.

[0023] In one possible implementation, the node-based multidimensional vector, determining the difference value corresponding to each node, includes:

[0024] Determine the degree matrix of the global graph;

[0025] Based on the distance between any two nodes, determine the edge weight of the edge corresponding to each pair of nodes, and determine the edge weight matrix.

[0026] Based on the degree matrix of the global graph and the edge weight matrix, the edge weight matrix is ​​normalized to obtain a normalized edge weight matrix.

[0027] Based on the normalized edge weight matrix and identity matrix, the difference value corresponding to each node is determined.

[0028] In one possible implementation, the global graph contains multiple nodes and multiple edges. Based on the key points, the local quality features corresponding to the point cloud to be evaluated are determined, including:

[0029] The key points are sampled to obtain multiple target key points;

[0030] Determine the target location of each target key point in the point cloud to be evaluated, and based on each target location, cluster the target key points corresponding to each target location as the center to obtain local clusters of the target number. Each target key point corresponds to a local cluster, and each local cluster contains a target key point and at least one node.

[0031] Determine the feature encoding between the target key points and their corresponding edges in each local cluster;

[0032] The features of each target key point are aggregated with the features of the other key points in the corresponding local cluster to obtain the target features of each target key point;

[0033] Multiple depth processing steps are performed on the target features of the target key points to obtain the depth features of each target key point;

[0034] Pooling operations are performed on each depth feature to obtain the local quality features of the local clusters corresponding to each target key point, thereby obtaining the local quality features of the point cloud to be evaluated.

[0035] In one possible implementation, determining the feature encoding between the target keypoint and the corresponding edge in each local cluster includes:

[0036] Construct a local directed graph for each local cluster, wherein the local directed graph can represent the directional relationship between nodes and target key points;

[0037] Based on the local directed graph, the edges between the target key point and each node in the corresponding local cluster are determined, so as to obtain the edges between the target key point and each node in the corresponding local cluster in each local cluster.

[0038] The edges between the target key point in each local cluster and each node in the corresponding local cluster are encoded to obtain the feature encoding between the target key point and the corresponding edge in each local cluster.

[0039] In one possible implementation, determining the quality score of the point cloud to be evaluated based on its global and local quality features includes:

[0040] Each of the local quality features is concatenated with the global quality feature to obtain multiple concatenated features;

[0041] Each concatenated feature is encoded, and each encoded feature is weighted to obtain multiple weighted features;

[0042] By fusing all weighted features, the comprehensive quality features of the point cloud to be evaluated are obtained;

[0043] The comprehensive quality features are input into the quality regression model, and the output quality score is obtained to obtain the quality score of the point cloud to be evaluated.

[0044] Secondly, this application provides a point cloud quality determination device, which adopts the following technical solution:

[0045] A point cloud quality determination device, comprising:

[0046] A module is created to build a global graph of the point cloud to be evaluated;

[0047] The key point determination module is used to determine the key points corresponding to the global graph, and the key points are used to characterize the overall structure of the point cloud.

[0048] The feature determination module is used to determine the global quality features and local quality features corresponding to the point cloud to be evaluated based on the key points.

[0049] The score determination module is used to determine the quality score of the point cloud to be evaluated based on the global quality features and local quality features corresponding to the point cloud to be evaluated.

[0050] Thirdly, this application provides an electronic device that adopts the following technical solution:

[0051] An electronic device comprising:

[0052] At least one processor;

[0053] Memory;

[0054] At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, the at least one application being configured to: perform the method described in the first aspect above.

[0055] Fourthly, this application provides a computer-readable storage medium, which adopts the following technical solution:

[0056] A computer-readable storage medium includes: a computer program stored thereon that can be loaded by a processor and execute the method described in the first aspect above.

[0057] In summary, this application includes the following beneficial technical effects:

[0058] A global graph of the point cloud to be evaluated is established. The integrity of the global structure of the point cloud has a certain impact on visual quality. Therefore, key points that can characterize the overall structure of the point cloud are identified from the point cloud to be evaluated. Based on the key points, the global quality features and local quality features of the point cloud to be evaluated are determined. Based on the corresponding global and local quality features, the quality score of the point cloud to be evaluated is determined, thereby satisfying the attention characteristics from the global to the local and improving the consistency between the determined quality of the point cloud to be evaluated and the perceived quality of the human eye, thus improving the objectivity of the evaluation of the perceived quality of the point cloud. Attached Figure Description

[0059] Figure 1 This is a flowchart illustrating a point cloud quality determination method provided in an embodiment of this application;

[0060] Figure 2 This is a schematic diagram of key points provided in an embodiment of this application;

[0061] Figure 3 This is a partial directed graph provided in the embodiments of this application;

[0062] Figure 4 This is a schematic diagram illustrating the process of determining feature codes provided in an embodiment of this application;

[0063] Figure 5 This is a block diagram of a point cloud quality determination device provided in an embodiment of this application;

[0064] Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0065] The following is in conjunction with the appendix Figure 1-6 This application will be described in further detail.

[0066] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0067] To facilitate understanding of the technical solutions proposed in this application, several elements that will be introduced in the description of this application are first presented here. It should be understood that the following description is only for the purpose of understanding these elements and the content of the embodiments of this application, and does not necessarily cover all possible situations.

[0068] A point cloud typically consists of tens of thousands of unordered points, each of which is composed of three-dimensional geometric coordinates and related attributes (color, normal, etc.). The data of each point is its corresponding three-dimensional geometric coordinates and related attributes.

[0069] Perceived quality refers to how users or humans subjectively evaluate the quality of a product, service, experience, or system.

[0070] In recent years, advancements in acquisition and reconstruction technologies have made point cloud acquisition increasingly simple, leading to its application in fields such as autonomous driving, digital animation, and virtual reality. However, since 3D point clouds can contain tens of thousands of points, quality issues such as noise and distortion may arise during point cloud processing and analysis. Furthermore, efficient transmission of 3D point clouds typically requires compression encoding, which inevitably introduces distortions such as texture blurring and structural voids. These problems directly impact the accuracy and efficiency of processing and analysis, reducing the accuracy and reliability of the point cloud data. Since subjective human experimentation is the most reliable method for evaluating the visual quality of point clouds, evaluating the perceptual quality of 3D point clouds can be used to measure the overall quality of point cloud data in order to address low-quality point cloud data promptly. Therefore, objectively evaluating the perceptual quality of 3D point clouds has become a pressing issue.

[0071] In view of this, embodiments of this application provide a point cloud quality determination method. By establishing a global graph of the point cloud to be evaluated, the global structural integrity of the point cloud has a certain impact on visual quality. Therefore, key points that can characterize the overall structure of the point cloud are identified from the point cloud to be evaluated. Based on the key points, the global quality features and local quality features of the point cloud to be evaluated are determined. Based on the global quality features and local quality features corresponding to the point cloud to be evaluated, the quality score of the point cloud to be evaluated is determined. This satisfies the attention characteristics from global to local and improves the consistency between the determined quality of the point cloud to be evaluated and the perceived quality of the human eye, thereby improving the objectivity of evaluating the perceived quality of the point cloud.

[0072] See Figure 1 This application provides a point cloud quality determination method, executed by an electronic device, the method comprising:

[0073] Step S101: Establish a global graph of the point cloud to be evaluated.

[0074] Specifically, a global graph of a point cloud is an abstract model representing the structure of the point cloud in spatial coordinates. A global graph contains multiple nodes and at least one edge. Nodes represent selected points or sets of points in the point cloud, and edges represent the spatial or geometric connections and topological relationships between nodes. Specifically, the process of constructing the global graph of the point cloud to be evaluated can be as follows: Each point in the point cloud to be evaluated is determined as a node. Based on the KNN nearest neighbor method, the node with the closest distance to any given node is selected as its nearest neighbor. An edge is then constructed between the node and its nearest neighbor to obtain the edges between each node and its nearest neighbor, thus constructing the global graph of the point cloud to be evaluated.

[0075] Step S102: Determine the key points corresponding to the global graph.

[0076] Among them, key points represent the overall structure of the point cloud, see [link to relevant documentation]. Figure 2 The extracted key points can roughly reflect the morphology, integrity, and noise level of the point cloud. Figure 2 a is the first physical image. Figure 2 b represents the key point in the point cloud corresponding to the first physical image; similarly... Figure 2 c is the second physical image. Figure 2 d represents the key point in the point cloud corresponding to the second physical image; Figure 2 e is the third physical image. Figure 2 f represents the key point in the point cloud corresponding to the third physical image.

[0077] In point clouds, structural features are typically located in high-frequency space, and the human eye is more sensitive to high-frequency information, able to clearly perceive distortions in the overall structure of the point cloud. Therefore, nodes in the global graph can be sampled once to obtain high-frequency structural points, which can then be used as key points in the global graph. High-frequency structural points are points in the point cloud data that possess high-frequency signals, typically exhibiting high curvature or geometric feature variations. High-pass filters can be used to select these high-frequency signals.

[0078] Specifically, the process of extracting high-frequency structure points from a global graph may include: filtering each node in the point cloud using a filter, determining whether each node is a high-frequency structure point based on its features and attributes (curvature and orientation, etc.), and performing post-processing on the extracted high-frequency structure points to improve their quality and reliability. The filter may be a Gaussian filter or a median filter, and the filtering method may be smoothing or edge detection, etc. This application does not limit the specific methods used.

[0079] Step S103: Based on the key points, determine the global quality features and local quality features corresponding to the point cloud to be evaluated.

[0080] Global quality features can characterize the overall properties of a point cloud, such as its shape, distribution, or density. Local quality features can characterize the local properties of a point cloud and the local properties of each node. The local properties of a node can include normal vectors, curvature, and surface roughness.

[0081] After identifying the key points corresponding to the global map, the global structural distortion features of the point cloud can be determined based on these key points, thereby improving the objectivity of determining the perceptual quality of the point cloud to be evaluated. Specifically, a key point set can be established based on the extracted key points corresponding to the global map, and this key point set can be input into a global feature extraction model to obtain the global quality features output by the model. More specifically, the process of obtaining the global quality features described above can be represented as follows: ,in, For global quality characteristics, This represents a global feature extraction model. For the set of key points, This represents max pooling, used to aggregate features from all keypoints while maintaining permutation invariance. The global feature extraction model is a pre-trained model consisting of multiple layers of one-dimensional convolutions, batch normalization, and a linear rectified activation function repeated four times.

[0082] Because the human eye has a global-to-local focus characteristic, and distortions such as texture blurring and structural voids typically occur in local areas, to improve the objectivity of evaluating the perceptual quality of point clouds, we can focus on distortion phenomena in local areas. Specifically, based on nodes within a preset range from each keypoint, local regions are established. Each local region contains one keypoint and at least one node. Based on the local region corresponding to each keypoint, feature descriptors are determined for each local region, and the local quality of the point cloud is evaluated using these local feature descriptors to obtain local quality features. These feature descriptors may include point feature histograms, discovery vectors, flatness, curvature 3D shape context, and local surface features, etc.

[0083] Step S104: Determine the quality score of the point cloud to be evaluated based on the global quality features and local quality features corresponding to the point cloud to be evaluated.

[0084] Because the human eye is more sensitive to structural features, and structural quality significantly influences the geometry, topology, and accuracy of feature extraction in point clouds, the overall quality level of a point cloud is greatly affected. Therefore, global and local quality features can be combined to enhance the perception of overall structure by local quality features. Specifically, global quality features can be used as contextual information and concatenated with each local quality feature. More specifically, global quality features are concatenated with each local quality feature to obtain the concatenated features corresponding to each local quality feature.

[0085] Since each local quality feature is derived from its own local region, but local quality may be influenced by other local regions, different local quality features can be weighted and fused to enhance the objectivity of determining the quality score of the point cloud to be evaluated. Specifically, a feature sequence concatenated with the global quality feature and each local quality feature is established. A Tranformer model is used to weight and fuse the local quality features of different local regions to obtain weighted quality features. Then, average pooling is used to fuse the weighted quality features of all local regions to obtain a comprehensive quality feature. This comprehensive quality feature is input into a quality regression model to obtain the output quality score, thus determining the quality score of the point cloud to be evaluated. The quality regression model is a pre-trained model containing two hidden layers.

[0086] One possible implementation of this application embodiment is that, in step S101 above, establishing a global graph of the point cloud to be evaluated may specifically include: obtaining the multi-dimensional vector of each point in the point cloud to be evaluated; treating each point in the point cloud to be evaluated as a node; determining the distance between every two nodes based on the three-dimensional spatial coordinates of each node; determining the edge based on the distance between every two nodes; and establishing a global graph of the point cloud to be evaluated based on the nodes and the edges.

[0087] The multidimensional vector consists of the three-dimensional spatial coordinates of a point and its color attribute. Specifically, the three-dimensional spatial coordinates and color attribute of each point in the point cloud to be evaluated are used to form a multidimensional vector with a multidimensional dimension of D for that point. For example, if the three-dimensional spatial coordinates of a point are (X,Y,Z) and the color attribute is (R,G,B), then the multidimensional vector of that point is [X,Y,Z,R,G,B].

[0088] A point cloud contains tens of thousands of points. To construct a global graph of the point cloud to be evaluated, a point set can be established, and the 3D spatial coordinates and color attributes of each point in the point cloud to be evaluated can be obtained. These are then used to construct a multidimensional vector for each point. The global graph of the point cloud to be evaluated is then constructed based on the point set and the multidimensional vectors of each point, ensuring that the global graph can be constructed based on all the points in the point cloud. Specifically, the number of points N in the point cloud to be evaluated is obtained, and a point set P is constructed. , Let i be the i-th point in the set of points. Let be a set of D-dimensional integers.

[0089] Furthermore, after obtaining the point set of the point cloud to be evaluated, each point in the point cloud is treated as a node, and the Euclidean distance between any two nodes is calculated based on the three-dimensional spatial coordinates of each node. The connecting line between two nodes whose Euclidean distance is not greater than a distance threshold is called an edge, thus obtaining the nodes and edges of the global graph and establishing the global graph of the point cloud to be evaluated. The method for calculating the Euclidean distance is well known in the art, and will not be described in detail in this embodiment.

[0090] More specifically, make Represents the global graph, where Represents a node in the global graph. Let represent the edges in the global graph. Then the global graph of the point cloud P to be evaluated can be represented as follows: ,in, Represents nodes in the global graph. Represents the relationships between edges in the global graph, and It is an adjacency matrix.

[0091] One possible implementation of this application embodiment is that, in step S102 above, determining the key points corresponding to the global graph of the point cloud to be evaluated may specifically include: determining the difference value corresponding to each node based on the multidimensional vector of the node; sorting the difference values ​​corresponding to each node in descending order to obtain a target sequence; selecting a preset number of difference values ​​starting from the first difference value in the target sequence as a target set; and determining the node corresponding to each difference value in the target set as the key point corresponding to the global graph of the point cloud to be evaluated.

[0092] The difference value is used to characterize the degree of difference between a node and its neighboring nodes. The larger the difference value of a node, the more prominent the node is in the structure of the point cloud to be evaluated.

[0093] Because the human eye is highly sensitive to structural features, points that stand out more in a point cloud are more easily seen. Therefore, key points in the point cloud to be evaluated can be determined based on these prominent points. Specifically, Euclidean distance can be used to characterize the degree of difference between two points. Thus, the difference value for each node can be determined by comparing the Euclidean distances between the two nodes corresponding to an edge in the global graph. More specifically, the Euclidean distance between the two nodes corresponding to an edge in the global graph is calculated using their 3D spatial coordinates, and this distance is used as the difference value between the two nodes. Here, Euclidean distance = sqrt((x2-x1)^2+(y2-y1)^2+...+(z2-z1)^2), where (x1, y1, z1) and (x2, y2, z2) are the coordinates of the two nodes corresponding to the edge in the global graph, respectively.

[0094] Furthermore, after obtaining the difference value between the two nodes corresponding to each edge in the global graph, the calculated difference values ​​can be sorted in descending order (from largest to smallest) to obtain the target sequence. Here, one difference value corresponds to two nodes, and the target sequence stores the difference values ​​between two nodes. For example, if the difference value between node A and node B is 3.2, the difference value between node A and node C is 2.1, and the difference value between node B and node C is 3.7, then the target sequence is: the difference value between node B and node C is 3.7, the difference value between node A and node B is 3.2, and the difference value between node A and node C is 2.1.

[0095] Furthermore, after obtaining the target sequence, a preset number of difference values ​​can be selected, starting from the first difference value in the target sequence, as the target set. The node corresponding to each difference value in the target set is then identified as a key point in the global graph. Specifically, the first preset number of difference values ​​in the target sequence are directly obtained to form the target set, and the two nodes corresponding to each difference value in the target set are used as key points in the global graph. For example, in the above example, the preset number is 2. In the target sequence where the difference value between node B and node C is 3.7, the difference value between node A and node B is 3.2, and the difference value between node A and node C is 2.1, the first two difference values ​​are obtained: the difference value between node B and node C is 3.7, and the difference value between node A and node B is 3.2. The difference value between node B and node C and the difference value between node A and node B are then combined to form the target set, and the nodes B and C corresponding to the difference value 3.7 and the nodes A and B corresponding to the difference value 3.2 are used as key points in the global graph.

[0096] One possible implementation of this application embodiment is as follows: In the above embodiment, determining the difference value corresponding to each node based on the multidimensional vector of the node may specifically include: determining the degree matrix of the global graph based on the multidimensional vector of each node; determining the edge weight of the edge corresponding to each pair of nodes based on the distance between each pair of nodes, and determining the edge weight matrix; normalizing the edge weight matrix based on the degree matrix and the edge weight matrix of the global graph to obtain a normalized edge weight matrix; and determining the difference value corresponding to each node based on the normalized edge weight matrix and the identity matrix.

[0097] After establishing the global graph of the point cloud to be evaluated, a graph filter can be used to extract key points from the corresponding nodes in the global graph in order to filter the graph signal. Specifically, a graph filter is established based on the multi-dimensional vector of each node in the global graph, and the three-dimensional spatial coordinates of each node are input into the graph filter to obtain the difference value of each node in the output. More specifically, the process of establishing the graph filter can be as follows: since the global graph of the point cloud to be evaluated is an undirected graph, the number of diagonals on the degree matrix of the point cloud to be evaluated is the number of edges associated with each node, and the numbers on the diagonals are all 0 to obtain the degree matrix of the global graph; based on the adjacency matrix of the global graph of the point cloud to be evaluated, the edge weight between every two nodes is calculated to obtain the edge weight matrix of the global graph, and the edge weight matrix is ​​normalized using the inverse of the degree matrix.

[0098] More specifically, since the edge weight between two nodes is negatively correlated with the difference between the two nodes, the difference for each node can be determined by calculating the edge weight. Specifically, the edge weight between the two nodes is obtained by comparing the square of the Euclidean distance between them with the square of the first parameter, and then taking the negative of the ratio as a power of e. More specifically, based on the formula... The edge weight between the two nodes is calculated. Point and points Edge weights between them For point and points The Euclidean distance between nodes is defined by σ, the first parameter controlling the rate of edge weight decay, and τ, the second parameter limiting the distance threshold required to form an edge. Further, after obtaining the edge weights of the edges corresponding to every two nodes, these edge weights are substituted into the adjacency matrix. Determine the edge weight matrix .

[0099] Furthermore, after determining the edge weights, the edge weight matrix is ​​normalized using the inverse matrix of the global graph. Specifically, based on the formula... The edge weight matrix is ​​normalized. This is the optimized edge weight matrix. It is the inverse of the degree matrix. This is the edge weight matrix.

[0100] Furthermore, a graph filter is established based on the normalized edge weight matrix. More specifically, graph filters .in, It is the identity matrix. This is the optimized edge weight matrix. For matrix A diagonal matrix composed of eigenvectors, For matrix A diagonal matrix composed of the eigenvalues ​​of .

[0101] Furthermore, after obtaining the graph filter, the three-dimensional spatial coordinates of each store are input into the graph filter to obtain the output graph signal frequency response value of each node. The magnitude of the image signal frequency response value reflects the point's... and points In terms of structural differences, the larger the graph signal frequency response value, the more prominent the point is in the structure. Therefore, the graph signal frequency response value corresponding to each node can be used as the difference value of each node.

[0102] One possible implementation of this application embodiment is as follows: In the above embodiment, based on key points, the local quality features corresponding to the point cloud to be evaluated are determined. Specifically, this may include: sampling key points to obtain multiple target key points; determining the target position of each target key point in the point cloud to be evaluated, and clustering based on each target position with the target key point corresponding to each target position as the center to obtain local clusters of the target number; determining the feature encoding between the target key point and the corresponding edge in each local cluster; aggregating the features of each target key point and the remaining key points in the corresponding local cluster to obtain the target features of each target key point; performing multiple depth processing on the target features of the target key points to obtain the depth features of each target key point; and performing pooling operations on each depth feature to obtain the local quality features of the local cluster corresponding to each target key point, thereby obtaining the local quality features corresponding to the point cloud to be evaluated.

[0103] Each target key point corresponds to a local cluster, and each local cluster contains a target key point and at least one node.

[0104] Because the human visual system is more sensitive to structure and the human eye has a global-to-local focus characteristic, key points can be selected as cluster centers to establish clusters. Specifically, when the number of key points is too large, clusters may overlap, leading to duplicate sampling. Therefore, to ensure each cluster contains more information while avoiding duplicate sampling, key points are sampled a second time to reduce the number of clusters. Specifically, based on the global map of the point cloud to be evaluated, a preset number of key points is determined. One key point is selected as the initial sampling point. Then, based on the distance between the initial sampling point and all other nodes, the point farthest from the initial sampling point is selected as the next sampling point. This step is repeated until the preset number of sampling points is obtained, resulting in multiple target key points. The KNN clustering method is then used to cluster each target key point as the center, resulting in local clusters containing multiple nodes near the target key point. Since the number of nodes and the distances between nodes vary in the global map, the number of sampling points for different global maps can be determined based on density.

[0105] Specifically, edges between each node in a local cluster and the target keypoint are established. A feature concatenation operation is then used to encode the features between the target keypoint and the corresponding edge in each local cluster, yielding encoded features. After encoding, a graph feature extraction module is used to learn and aggregate the encoded features corresponding to that local cluster, resulting in aggregated features, which are the target features corresponding to the target keypoint in that local cluster. The graph feature extraction module is a neural network consisting of a feature extractor composed of a multilayer perceptron, batch normalization, a leaky linear rectified function, and max pooling.

[0106] Due to the multi-layered nature of the human visual system, to further improve the objectivity of evaluating the perceptual quality of point clouds, target features can undergo multiple deep learning processes to obtain quality features at different levels. Specifically, the target features corresponding to keypoints in local clusters are input into LFE layers of different depths to obtain the multi-layered perceptual quality output, which is the depth feature corresponding to the keypoint. Pooling is then performed on these depth features to obtain the local quality features of the local cluster corresponding to the keypoint. More specifically, the process of multiple depth processing and pooling is as follows: ,in, This refers to multi-level perceived quality, specifically the local quality features of local clusters corresponding to key target points. Indicates the cascading depth. This indicates a feature cascade operation. Indicates average pooling. For target features, This is a new feature dimension for network mapping.

[0107] One possible implementation of this application embodiment is as follows: In the above embodiment, determining the feature encoding between the target key point and the corresponding edge in each local cluster may specifically include: constructing a local directed graph for each local cluster; based on the local directed graph, determining the edge between the target key point and each node in the corresponding local cluster to obtain the edge between the target key point and each node in the corresponding local cluster; encoding the edge between the target key point and each node in the corresponding local cluster and the target key point to obtain the feature encoding between the target key point and the corresponding edge in each local cluster.

[0108] Locally directed graphs can represent the directional relationships between nodes and key target points. See [link to documentation] for details. Figure 3 In a locally directed graph, every node that has an edge to the target key point points to the target key point.

[0109] Specifically, construct the local directed graph corresponding to each local cluster C. ,in, This represents a node in a locally directed graph, that is, a vertex in a locally directed graph. Let D represent the edges in the local directed graph, where D=6, meaning each node in the local directed graph has a six-dimensional vector, i.e., [X,Y,Z,R,G,B], and K is the number of nodes in the local cluster excluding the target keypoint. Specifically, since the graph of the local cluster is a directed graph, the connection relationships of the edges in the local directed graph can be... ,in, Indicates the pointing relationship between edges. More specifically, the existence of an edge is determined based on the distance between each node and its corresponding target key point; or, the edges in each local directed graph are determined by the edges in the global graph.

[0110] Furthermore, after obtaining the local directed graph of the local clusters, nodes with pointing relationships to the target keypoints are designated as target nodes, and the edges between the target keypoints and target nodes are designated as edges in the local clusters. The edges between the target keypoints and each corresponding node in each local cluster are encoded to obtain the feature encoding between the target keypoints and their corresponding edges in each local cluster. Specifically, through... For each local cluster, feature encoding is performed between the target keypoint and its corresponding edge, where... This indicates a feature cascade operation. For target key points in a local directed graph, For nodes in a local directed graph, This represents the subtraction of the two feature vectors. The process of obtaining the feature encoding between the target keypoints and their corresponding edges in each local cluster is described in [link to documentation]. Figure 4 In the picture For feature extractor, For target features.

[0111] In one possible implementation of this application embodiment, in step S104 above, the quality score of the point cloud to be evaluated is determined based on the global quality features and local quality features corresponding to the point cloud to be evaluated. Specifically, this may include: concatenating each local quality feature with the global quality feature to obtain multiple concatenated features; encoding each concatenated feature and weighting each encoded feature to obtain multiple weighted features; fusing all weighted features to obtain the comprehensive quality features of the point cloud to be evaluated; inputting the comprehensive quality features into the quality regression model and obtaining the output quality score to obtain the quality score of the point cloud to be evaluated.

[0112] Specifically, the global quality features are concatenated with each local quality feature to obtain the concatenated features for each local quality feature. Further, a feature sequence resulting from the concatenation of the global and local quality features is established, and a Tranformer model is used to weightedly fuse the local quality features of different local regions to obtain weighted quality features. Then, average pooling is used to fuse the weighted quality features of all local regions to obtain a comprehensive quality feature. This comprehensive quality feature is input into a quality regression model, and the output quality score is obtained to determine the quality score of the point cloud to be evaluated. The quality regression model is a fully connected network with two hidden layers.

[0113] More specifically, through The global quality feature is concatenated with each local quality feature, where, Indicates the characteristics after cascading. This indicates a feature cascade operation. For global quality characteristics, Let be the local quality features of the c-th cluster. Further, the concatenated feature sequence is obtained. and based on The feature sequence is encoded as a d-dimensional matrix, where Q represents a Tranformer query, K represents the key, and V represents the value. , , ,and , as well as All are learnable matrices. Furthermore, based on... Calculate the weighted quality feature sequence, where, , Let T denote the normalized exponential function, and let T denote the matrix transpose operation. Let be the parameter matrix of the science department. Furthermore, a comprehensive quality feature is obtained by fusing the weighted quality features of all local clusters using average pooling.

[0114] The above embodiments describe a point cloud quality determination method from the perspective of process flow. The following embodiments describe a point cloud quality determination device from the perspective of virtual module or virtual unit. For details, please refer to the following embodiments.

[0115] See Figure 5 This application provides a point cloud quality determination device 50, including: a building module 510, a key point determination module 520, a feature determination module 530, and a score determination module 540.

[0116] A point cloud quality determination device 50 includes:

[0117] Module 510 is established to create a global graph of the point cloud to be evaluated;

[0118] The key point determination module 520 is used to determine the key points corresponding to the global graph. The key points are used to characterize the overall structure of the point cloud.

[0119] The feature determination module 530 is used to determine the global quality features and local quality features corresponding to the point cloud to be evaluated based on key points;

[0120] The score determination module 540 is used to determine the quality score of the point cloud to be evaluated based on the global quality features and local quality features corresponding to the point cloud to be evaluated.

[0121] In one possible implementation of this application embodiment, when establishing a global graph of the point cloud to be evaluated, the module 510 can specifically be used for:

[0122] Obtain the multidimensional vector of each point in the point cloud to be evaluated. The multidimensional vector consists of the point's three-dimensional spatial coordinates and the point's color attribute.

[0123] Each point in the point cloud to be evaluated is treated as a node;

[0124] Based on the three-dimensional spatial coordinates of each node, determine the distance between every two nodes;

[0125] Determine the edges based on the distance between any two nodes;

[0126] A global graph of the point cloud to be evaluated is constructed based on nodes and edges.

[0127] In one possible implementation of this application embodiment, when determining key points corresponding to the global graph, the key point determination module 520 may specifically:

[0128] Based on the multidimensional vector of nodes, the difference value corresponding to each node is determined. The difference value is used to characterize the degree of difference between the node and its neighboring nodes.

[0129] The difference values ​​corresponding to each node are sorted in descending order to obtain the target sequence;

[0130] Starting from the first difference value in the target sequence, a preset number of difference values ​​are selected as the target set;

[0131] The node corresponding to each difference value in the target set is identified as a key point in the global graph.

[0132] In one possible implementation of this application embodiment, the key point determination module 520, when determining the difference value corresponding to each node based on a node-based multidimensional vector, can specifically be used for:

[0133] Determine the degree matrix of the global graph;

[0134] Based on the distance between any two nodes, determine the edge weight of the edge corresponding to each pair of nodes, and determine the edge weight matrix.

[0135] Based on the degree matrix and edge weight matrix of the global graph, the edge weight matrix is ​​normalized to obtain a normalized edge weight matrix.

[0136] Based on the normalized edge weight matrix and identity matrix, the difference value corresponding to each node is determined.

[0137] In one possible implementation of this application embodiment, when the feature determination module 530 determines the local quality features corresponding to the point cloud to be evaluated based on key points, it can specifically be used for:

[0138] Sampling of key points yields multiple target key points;

[0139] Determine the target location of each target key point in the point cloud to be evaluated, and based on each target location, cluster the target key points corresponding to each target location as the center to obtain local clusters of the target number. Each target key point corresponds to a local cluster, and each local cluster contains a target key point and at least one node.

[0140] Determine the feature encoding between the target key points and their corresponding edges in each local cluster;

[0141] The feature codes corresponding to each target key point are aggregated to obtain the target features of each target key point;

[0142] Multiple depth processing steps are performed on the target features of the target key points to obtain the depth features of each target key point;

[0143] Pooling operations are performed on each depth feature to obtain the local quality features of the local clusters corresponding to each target key point, thereby obtaining the local quality features of the point cloud to be evaluated.

[0144] In one possible implementation of this application embodiment, when determining the feature encoding between the target key point and the corresponding edge in each local cluster, the feature determination module 530 may specifically be used for:

[0145] Construct a local directed graph for each local cluster. The local directed graph can represent the directional relationship between nodes and key target points.

[0146] Based on the local directed graph, the edges between the target key point and each node in the corresponding local cluster are determined, so as to obtain the edges between the target key point and each node in the corresponding local cluster in each local cluster.

[0147] The edges between the target keypoints in each local cluster and each node in the corresponding local cluster are encoded to obtain the feature encoding between the target keypoints and the corresponding edges in each local cluster.

[0148] In one possible implementation of this application embodiment, when the score determination module 540 determines the quality score of the point cloud to be evaluated based on the global quality features and local quality features corresponding to the point cloud to be evaluated, it can specifically be used for:

[0149] Each local quality feature is concatenated with the global quality feature to obtain multiple concatenated features;

[0150] Each concatenated feature is encoded, and each encoded feature is weighted to obtain multiple weighted features;

[0151] By fusing all weighted features, the comprehensive quality features of the point cloud to be evaluated are obtained;

[0152] The comprehensive quality characteristics are input into the quality regression model, and the output quality score is obtained to obtain the quality score of the point cloud to be evaluated.

[0153] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0154] This application also describes an electronic device from the perspective of a physical device, such as... Figure 6 As shown, Figure 6 The illustrated electronic device 600 includes a processor 601 and a memory 603. The processor 601 and the memory 603 are connected, for example, via a bus 602. Optionally, the electronic device 600 may also include a transceiver 604. It should be noted that in practical applications, the transceiver 604 is not limited to one type, and the structure of this electronic device 600 does not constitute a limitation on the embodiments of this application.

[0155] Processor 601 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 601 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0156] Bus 602 may include a pathway for transmitting information between the aforementioned components. Bus 602 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 602 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0157] The memory 603 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0158] The memory 603 stores application code that executes the scheme of this application, and its execution is controlled by the processor 601. The processor 601 executes the application code stored in the memory 603 to implement the content shown in the foregoing method embodiments.

[0159] Among them, electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers, and can also be servers, etc. Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0160] This application provides a computer-readable storage medium storing a computer program that, when run on a computer, enables the computer to execute the corresponding content in the aforementioned method embodiments.

[0161] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0162] The above are only some embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for determining point cloud quality, characterized in that, include: Establish a global graph of the point cloud to be evaluated; Determine the key points corresponding to the global graph, where the key points represent the overall structure of the point cloud; Based on the key points, determine the global quality features and local quality features corresponding to the point cloud to be evaluated; Based on the global and local quality features of the point cloud to be evaluated, the quality score of the point cloud to be evaluated is determined. Determining the key points corresponding to the global graph includes: Based on the multidimensional vector of the node, the difference value corresponding to each node is determined. The difference value is used to characterize the degree of difference between the node and its neighboring nodes. The difference values ​​corresponding to each node are sorted in descending order to obtain the target sequence; Starting from the first difference value in the target sequence, a preset number of difference values ​​are selected as the target set; Each node corresponding to a difference value in the target set is identified as a key point in the global graph. Based on the node-based multidimensional vector, determine the difference value corresponding to each node, including: Determine the degree matrix of the global graph; Based on the distance between any two nodes, determine the edge weight of the edge corresponding to each pair of nodes, and determine the edge weight matrix. Based on the degree matrix of the global graph and the edge weight matrix, the edge weight matrix is ​​normalized to obtain a normalized edge weight matrix. Based on the normalized edge weight matrix and identity matrix, the difference value corresponding to each node is determined. Based on the aforementioned key points, the local quality features corresponding to the point cloud to be evaluated are determined, including: The key points are sampled to obtain multiple target key points; Determine the target location of each target key point in the point cloud to be evaluated, and based on each target location, cluster the target key points corresponding to each target location as the center to obtain local clusters of the target number. Each target key point corresponds to a local cluster, and each local cluster contains a target key point and at least one node. Determine the feature encoding between the target key points and their corresponding edges in each local cluster; The feature codes corresponding to each target key point are aggregated to obtain the target features of each target key point; Multiple depth processing steps are performed on the target features of the target key points to obtain the depth features of each target key point; Pooling operations are performed on each depth feature to obtain the local quality features of the local clusters corresponding to each target key point, thereby obtaining the local quality features of the point cloud to be evaluated.

2. The point cloud quality determination method according to claim 1, characterized in that, The process of establishing a global graph of the point cloud to be evaluated includes: Obtain the multidimensional vector of each point in the point cloud to be evaluated. The multidimensional vector consists of the three-dimensional spatial coordinates of the point and the color attribute of the point. Each point in the point cloud to be evaluated is treated as a node; Based on the three-dimensional spatial coordinates of each node, determine the distance between every two nodes; Determine the edges based on the distance between any two nodes; A global graph of the point cloud to be evaluated is established based on nodes and edges.

3. The point cloud quality determination method according to claim 1, characterized in that, The step of determining the feature encoding between target key points and corresponding edges in each local cluster includes: Construct a local directed graph for each local cluster, wherein the local directed graph can represent the directional relationship between nodes and target key points; Based on the local directed graph, the edges between the target key point and each node in the corresponding local cluster are determined, so as to obtain the edges between the target key point and each node in the corresponding local cluster in each local cluster. The edges between the target key point in each local cluster and each node in the corresponding local cluster are encoded to obtain the feature encoding between the target key point and the corresponding edge in each local cluster.

4. The point cloud quality determination method according to claim 1, characterized in that, The process of determining the quality score of the point cloud to be evaluated based on its global and local quality features includes: Each of the local quality features is concatenated with the global quality feature to obtain multiple concatenated features; Each concatenated feature is encoded, and each encoded feature is weighted to obtain multiple weighted features; By fusing all weighted features, the comprehensive quality features of the point cloud to be evaluated are obtained; The comprehensive quality features are input into the quality regression model, and the output quality score is obtained to obtain the quality score of the point cloud to be evaluated.

5. A point cloud quality determination device, characterized in that, include: The apparatus is applied to the point cloud quality determination method according to any one of claims 1 to 4.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; Memory; At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, said at least one application being configured to: perform the point cloud quality determination method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed in the computer, the computer is instructed to perform the point cloud quality determination method according to any one of claims 1 to 4.

Images