Point cloud quality assessment method based on non-local geometric and color gradient aggregation graph

Abstract

The present invention discloses a point cloud quality assessment method based on a non-local geometry and color gradient aggregation graph. The method adopts a non-local geometry and color gradient aggregation graph model to evaluate the point cloud quality. The non-local geometry and color gradient aggregation graph model includes a geometry and color gradient extraction module, first to four local graph aggregation modules, first and second position-adaptive self-attention mechanism modules and a task learning module; the geometry and color gradient extraction module is mainly used to extract the geometry and color gradient features of the distorted point cloud, the local graph aggregation module is mainly used to construct the local spatial topological structure of the point cloud and aggregate local quality perception features, the position-adaptive self-attention mechanism module is mainly used to solve the long-distance dependency of the point cloud, and the task learning module mainly maps the quality perception features learned by the model to quality scores. The present invention can effectively extract the quality perception features of the distorted point cloud.

Description

Technical Field

[0001] The invention belongs to the technical field of point cloud data processing. Background Art

[0002] Point clouds are one of the most widely used data formats in applications such as augmented reality and virtual reality. However, during the transmission and processing of point clouds, corresponding noise is often introduced, resulting in deterioration in point cloud quality. Therefore, to better manage and control point cloud quality in point cloud applications, it is necessary to study and propose point cloud quality assessment methods. Based on the amount of reference information introduced, point cloud quality assessment (PCQA) algorithms are generally divided into three categories: full-reference assessment (FR-PCQA), semi-reference assessment (RR-PCQA), and no-reference assessment (NR-PCQA). However, in many practical applications, the original reference point cloud is often lacking. Therefore, compared with full-reference and semi-reference point cloud assessment, no-reference point cloud quality assessment has been more widely studied in recent years.

[0003] NR-PCQA aims to evaluate the perceptual quality of distorted point clouds without any reference information. Traditional NR-PCQA methods are typically carefully designed based on handcrafted features, which are mapped to MOSs via support vector regression (SVR). However, due to the high complexity of distortion types and point cloud content, the understanding of handcrafted features remains limited. Furthermore, the computation of handcrafted features is complex and time-consuming. In recent years, due to the strong representational capabilities of deep neural networks (DNNs), an increasing number of DNN-based NR-PCQA methods have been proposed.

[0004] Existing DNNs-based methods can generally be divided into two types: projection-based methods and point-based methods. Projection-based methods first project the point cloud into multiple views and use convolutional neural networks (CNNs) to extract features from the corresponding views. These features are then fused, and finally, the fused features are mapped to quality scores through a regressor. The main idea of ​​these methods is to convert the point cloud into a regular 2D grid structure and use the powerful representation learning ability of CNNs to extract quality-aware features. Although the point cloud is converted into a regular 2D grid structure, additional distortions (such as quantization artifacts) are inevitably introduced during the conversion process, and features related to geometric distortion cannot be effectively extracted. This is because projection-based methods cannot model spatial topological structures, which can effectively represent the geometric structure degradation of point clouds. Therefore, projection-based models are not the best choice for quality prediction.

[0005] In addition, point-based methods, such as ResSCNN, GPA, and PKT-PCQA, refer to using the attribute information of point clouds, namely geometric coordinates and color information, as the input of the network model, and then using DNNs to extract quality-aware features, and finally mapping these quality-aware features to quality scores through a regressor. First, although these works process point clouds directly and do not introduce additional distortion like projection-based methods, they directly process the spatial coordinates and color information of points, which cannot effectively represent local changes, namely quality degradation. Second, the ResSCNN model does not consider the local structural information of the points, which can better reflect fine-grained distortion degradation. Finally, although GPA and PKT-PCQA can model the relationship between local neighbor points, they ignore the long-term dependencies of point clouds. Summary of the Invention

[0006] Purpose of the invention: In order to solve the problems existing in the above-mentioned prior art, the present invention proposes a point cloud quality assessment method based on non-local geometric and color gradient aggregation maps.

[0007] Technical Solution: The present invention provides a point cloud quality assessment method based on non-local geometric and color gradient aggregation graphs. This method uses a non-local geometric and color gradient aggregation graph model to assess point cloud quality. The non-local geometric and color gradient aggregation graph model includes a geometric and color gradient extraction module, first to fourth local graph aggregation modules, first and second position-adaptive self-attention mechanism modules, and a task learning module. The local graph aggregation module is used to perform local graph aggregation on input features to extract deep features of the input features, and the position-adaptive self-attention mechanism module is used to optimize the weights of the input features.

[0008] The point cloud is input into the geometry and color gradient extraction module to obtain the geometric gradient features and color gradient features of the point cloud; the geometric gradient features are input into the first local graph aggregation module to obtain the aggregated geometric gradient features, and the color gradient features are input into the second local graph aggregation module to obtain the aggregated color gradient features; the aggregated geometric gradient features and the aggregated color gradient features are cascaded, and the cascaded features are input into the third local graph aggregation module, the features output by the third local graph aggregation module are sequentially subjected to the Relu activation function and the maximum pooling layer and then input into the first position adaptive self-attention mechanism module, the first position adaptive self-attention mechanism module inputs the weight-optimized features into the fourth local graph aggregation module, the features output by the fourth local graph aggregation module are subjected to the Relu activation function and then input into the second position adaptive self-attention mechanism module, the features output by the second position adaptive self-attention mechanism module are subjected to the global maximum pooling layer to obtain the quality perception features, and the learning module maps the quality perception features to the quality score to obtain the quality score of the point cloud.

[0009] Furthermore, the geometry and color gradient extraction module uses the KNN algorithm to construct the cth point v in the point cloud. c The spatial graph of , and find v c The m nearest neighbor points of , and then use the following formula to extract point v c Geometric gradient features c=1,2,...,N, where N represents the total number of points contained in any block of a point cloud;

[0010]

[0011] Among them, i, j and k are unit vectors. and Represents point v c Geometric gradients in the x, y, and z directions; and The expression is as follows:

[0012]

[0013]

[0014]

[0015] in, Represents point v c The x-coordinates of all neighboring points, Represents point v c The y coordinates of all neighboring points, Represents point v c The z coordinates of all neighboring points; c ,y c and z c Represents point v c The coordinate values ​​in the x-direction, y-direction, and z-direction; ∈ is a constant term; where the expressions of Δx, Δy, and Δz are as follows:

[0016]

[0017]

[0018]

[0019] in, Represents point v c The maximum x-coordinate among all neighboring points, Represents point v c The minimum x-coordinate of all neighboring points, Represents point v cThe maximum value of the y coordinate among all neighboring points, Represents point v c The minimum y coordinate of all neighboring points, Represents point v c The maximum value of the z coordinate among all neighboring points, Represents point v c The minimum value of the z coordinate of all neighboring points;

[0020] The geometry and color gradient extraction module uses the following formula to extract point v c Color gradient feature G c :

[0021]

[0022] d1=max{R m {||d k,c ||}}

[0023] Among them, C(v c ) represents point v c The color attribute, C(v k ) represents point v c The kth nearest neighbor point v k The color attribute, ||d k,c || represents v k With v c The Euclidean distance between m {||d k,c ||} represents the m nearest neighbor points and v c The Euclidean distance set of Used to weigh different distances to point v c The effect of color gradient, d k,c Represents point v c and the neighboring point v k The direction vector.

[0024] Furthermore, the local graph aggregation module performs the following operations on the input features:

[0025] Conv F = <F(v c ),w(v c )>+max <F(v k ),w(v k )>cosθ

[0026]

[0027] Among them, <.> represents the inner product operation, Conv F is the output of the local graph aggregation module, F(v c) is the input feature of the local graph aggregation module, which is the point v c The corresponding feature, w(v c ) is F(v c ) weight, F(v k ) is the neighboring point v k The characteristics of w(v k ) is F(v k ) weight; the first local graph aggregation module input feature F(v c ) is point v c The geometric gradient feature, F(v k ) is the neighboring point v k The geometric gradient features of the second local graph aggregation module input feature F(v c ) is point v c Color gradient features, F(v k ) is the neighboring point v k The color gradient feature of the third local graph aggregation module input feature F(v c ) is point v c Aggregate feature, the aggregate feature is the feature F(v k ) is the neighboring point v k Aggregation features of the fourth local graph aggregation module input feature F(v c ) is point v c The aggregated features after weight optimization, F(v k ) is point v k Aggregate features after weight optimization; d k,c Represents point v c and the neighboring point v k The direction vector, k s is the initialization direction vector, ||k s || is the Euclidean distance between the initialized direction vector and the origin.

[0028] Furthermore, the position-adaptive self-attention mechanism module introduces multi-head self-attention, which includes N h There are three self-attention blocks, each with the same structure. Any self-attention block uses three linear layers to generate the corresponding three feature sequences Q, K, and V:

[0029] Q=(F+PAE)W Q ; K=(F+PAE)W K ; V=(F+PAE)W V

[0030] Among them, F is the input feature of the self-attention block, PAE is the preset position adaptive embedding sequence, W Q 、W K and WV Both represent transformation matrices;

[0031] The feature self-attention (Q, K, V) output by the self-attention block is:

[0032]

[0033] Where d represents the dimension of the input feature and T is the matrix transpose;

[0034] The output feature MHSA of multi-head self-attention is:

[0035]

[0036] Among them, Q H ,K H and V H represents the feature sequence generated by the H-th self-attention block, W o is the output projection matrix.

[0037] Furthermore, the learning module also includes classifying the distortion type of the point cloud and classifying the distortion level of the point cloud.

[0038] Beneficial Effects: The method of the present invention can significantly improve the performance of point cloud quality assessment. It can extract geometric and color gradient features in local space using extended 3D convolutions and capture fine-grained structural information. It is an end-to-end deep learning training method that can effectively extract quality-perceived features from distorted point clouds. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a structural block diagram of the present invention;

[0040] Figure 2 It is a specific flow chart of the present invention;

[0041] Figure 3 A spatial diagram constructed for the present invention;

[0042] Figure 4 This is a structural block diagram of the position-adaptive self-attention mechanism module of the present invention. DETAILED DESCRIPTION

[0043] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0044] like Figure 1As shown, this embodiment proposes a point cloud quality assessment method based on a non-local geometric and color gradient aggregation graph, including a geometry and color gradient extraction module GCGEM, a local graph aggregation module LGAM, a position-adaptive self-attention mechanism module PASAM, and a task learning module. Specifically, the geometry and color gradient extraction module is mainly used to extract the geometry and color gradient features of the distorted point cloud, the local graph aggregation module is mainly used to construct the local spatial topology structure of the point cloud and aggregate local quality-aware features, the position-adaptive self-attention mechanism module is mainly used to resolve the long-distance dependencies of the point cloud, and the task learning module mainly maps the quality-aware features learned by the model to quality scores.

[0045] like Figure 2 As shown, the specific process of the present invention is: inputting the point cloud into the geometry and color gradient extraction module to obtain the geometric gradient features and color gradient features of the point cloud; inputting the geometric gradient features into the first local graph aggregation module to obtain the aggregated geometric gradient features, and inputting the color gradient features into the second local graph aggregation module to obtain the aggregated color gradient features; cascading the aggregated geometric gradient features and the aggregated color gradient features, and inputting the cascaded features into the third local graph aggregation module, and inputting the features output by the third local graph aggregation module into the first position adaptive self-attention mechanism module after passing through the Relu activation function and the maximum pooling layer in sequence, and the first position adaptive self-attention mechanism module inputs the weight-optimized features into the fourth local graph aggregation module, and the features output by the fourth local graph aggregation module are input into the second position adaptive self-attention mechanism module after passing through the Relu activation function, and the features output by the second position adaptive self-attention mechanism module are passed through the global maximum pooling layer to obtain quality perception features, and the learning module maps the quality perception features to quality scores to obtain the quality scores of the point clouds.

[0046] Figure 2 Max-pooling is maximum pooling, Global max-pooling is global maximum pooling, Distortion Type classification is distortion type classification, Distortion Level classification is distortion level classification, and Quality Regression is quality regression.

[0047] Geometry and color gradient extraction module

[0048] In the geometry and color gradient extraction module, the geometry and color gradient features of the distorted point cloud are extracted. Specifically, it can be divided into two submodules: the geometry gradient extraction module and the color gradient extraction module.

[0049] Geometric gradient extraction module

[0050] Constructing a spatial graph

[0051] A point cloud patch containing N points is represented as V = {v c |c=1,2,...,N}, where v c Represents the cth point in the point cloud patch. In order to describe the features of each point in the point cloud patch, F(v)∈R D In order to establish the topological structure between adjacent points and extract any point v in the patch c The local geometric and color gradient features of Figure 2 The v shown c Specifically, the KNN algorithm is used to find the m nearest neighbors for each point in the patch. The spatial graph is also used in the local graph aggregation module, using Represented as point v c The spatial diagram of:

[0052]

[0053] Among them, N(v c ,m) represents v c m neighboring points, direction vector d m,c =v m -v c , v m Indicates v c The mth neighboring point of Represents point v c Space diagram.

[0054] First, geometric coordinates only express the spatial positional relationship of the point cloud and cannot effectively express the local changes in the spatial structure. Second, in IQA, image gradient is an effective distortion characterization feature because it can capture the local structural information of the image, and the human visual system is very sensitive to structured information. Therefore, in order to describe the spatial structural changes of the point cloud, this embodiment introduces the concept of geometric gradient. The geometric gradient can be expressed as the rate of change of each point in the point cloud in the three directions of x, y, and z. The image gradient is defined as the difference between the pixels of the surrounding points and the pixels of the center point. In addition, the image is a regular 2D grid structure with clear up, down, left, and right positional relationships. However, it is very challenging to extract the geometric gradient of an irregular and disordered structure such as a point cloud. Specifically, the geometric structure of a point cloud is discrete, and its derivative cannot be obtained by analytical methods. Therefore, this embodiment directly processes discrete point clouds by applying the finite difference method (FDM) to approximate the geometric gradient, which can improve computational efficiency and avoid surface fitting processing of the point cloud.

[0055] Dot v c The geometric gradient The formula is:

[0056]

[0057] in, represents the partial derivative in the x direction, i represents the unit vector in the x direction, It is given by the following formula:

[0058]

[0059] f(x,y,z) represents a continuous function, and Definition and same.

[0060] Taking into account the irregular spatial structure of the point cloud and the arbitrariness of the directions between the center point and adjacent points, this embodiment approximates the above definition to the following formula:

[0061]

[0062]

[0063]

[0064] in, and Represents point v c Geometric gradients in the x, y, and z directions; Represents point v c The x-coordinates of all neighboring points, Represents point v c The y coordinates of all neighboring points, Represents point v c The z coordinates of all neighboring points; c ,y c and z c Represents point v c The coordinate values ​​in the x-, y-, and z-directions; ∈ is a constant term to prevent the denominator from being zero; the expressions for Δx, Δy, and Δz are as follows:

[0065]

[0066]

[0067]

[0068] in, Represents point v cThe maximum x-coordinate among all neighboring points, Represents point v c The minimum x-coordinate of all neighboring points, Represents point v c The maximum value of the y coordinate among all neighboring points, Represents point v c The minimum y coordinate of all neighboring points, Represents point v c The maximum value of the z coordinate among all neighboring points, Represents point v c The minimum z-coordinate of all neighboring points.

[0069] Finally, click v c The local maximum geometric gradient can be expressed as follows:

[0070]

[0071] Wherein, j and k both represent unit vectors.

[0072] Color gradient extraction module

[0073] As mentioned above, color gradient is an effective quality representation feature. This embodiment proposes a distance-weighted color gradient for point clouds. Assuming that the farther the neighboring points are from the center point, the less influence they have on the color of the center point. This is also consistent with the perception effect of HVS. c Color gradient feature G c It can be expressed as:

[0074]

[0075] d1=max{R m {||d k,c ||}}

[0076] Among them, C(v c ) represents point v c The color attribute, C(v k ) represents point v c The color attribute of the kth neighbor point, ||d k,c || represents v k With v c The Euclidean distance between m {||d k,c ||} represents the m nearest neighbor points and v c The Euclidean distance set of .

[0077] Local graph aggregation module

[0078] In the field of image quality assessment (IQA), the regular grid structure of an image enables the use of convolution kernels to extract deep features. However, for irregular structures such as point clouds, extracting deep features using convolution kernels is a challenge due to the arbitrariness of the directions between points. To solve this problem, a spatial graph was constructed in the previous section. In the CNN-based no-reference IQA method, feature extraction refers to the convolution between the convolution kernel and the feature map. Similarly, this embodiment uses a graph convolutional network (GCN) to extend the convolution operation and extract deep features from the point cloud. The aggregation operation process can be expressed as:

[0079] Conv F = <F(v c ),w(v c )>+max <F(v k ),w(v k )>cosθ

[0080]

[0081] Among them, <.> represents the inner product operation, Conv F is the output of the local graph aggregation module, F(v c ) and F(v k ) represent the points v input to the local graph aggregation module c 、v k Characteristics of d k,c Represents point v c and the neighboring point v k The direction vector, k s represents an arbitrarily initialized direction vector, w(v c ) and w(v k ) represent v c With v k The first local graph aggregation module inputs the feature F(v c ) is point v c The geometric gradient feature, F(v k ) is the neighboring point v k The geometric gradient features of the second local graph aggregation module input feature F(v c ) is point v c Color gradient features, F(v k ) is the neighboring point v k The color gradient feature of the third local graph aggregation module input feature F(v c ) is point v c Aggregate features, the aggregate features are the features after the color gradient features and the geometric gradient features are aggregated, F(v k ) is the neighboring point v k Aggregation features of the fourth local graph aggregation module input feature F(vc ) is point v c The aggregated features after weight optimization, F(v k ) is point v k Aggregate features after weight optimization; ||k s || is the Euclidean distance between the initialized direction vector and the origin.

[0082] Position-adaptive self-attention mechanism module

[0083] Transformer solves the long-term dependency problem that is difficult to solve in natural language processing, and it has subsequently been widely used in image processing. In order to expand the receptive field and model the global dependency of the point cloud, this embodiment introduces a position-adaptive self-attention mechanism module. Specifically, the self-attention mechanism processes sequence data by weighting the features at different positions to determine the importance of the features at that position. It allows the model to focus on different parts, thereby obtaining a stronger feature representation. In addition, considering the importance of position embedding to features at different positions, this embodiment proposes a position-adaptive embedding to enhance the understanding of global distortion features. Specifically, Figure 3 As shown in Figure 2, in PASAM, a linear layer liner is used to convert the input feature F into three different feature sequences: Q, K, V (Q refers to query, K refers to key, and V refers to value).

[0084] Q=(F+PAE)W Q ; K=(F+PAE)W K ; V=(F+PAE)W V ;

[0085] Among them, W Q 、W K and W V Both represent transformation matrices; d is the dimension of the input features.

[0086] d q =d k =d v =d m , d m Represents the dimension of the hidden layer of the position-adaptive self-attention mechanism module; PAE is the preset position-adaptive embedding sequence.

[0087] The output position-adaptive embedding and feature map. The attention weight self-attention(Q,K,V) is given by the following formula:

[0088]

[0089] T is the matrix transpose.

[0090] In order to model the various complex connections between features at different positions in the input point cloud, this embodiment further introduces multi-head self-attention (MHSA), which includes N h Self-attention blocks. MHSA linearly projects the input into multiple feature subspaces and is processed in parallel by several independent self-attention blocks. The vectors of multiple self-attention blocks are concatenated and mapped to the final output, which can be expressed as:

[0091]

[0092] Among them, Q H ,K H and V H represents the feature sequence generated by the H-th self-attention block, W o is the output projection matrix,

[0093] Task Learning Module

[0094] Task learning includes three types of tasks: quality regression, distortion type classification, and distortion level classification. The quality regression task maps the quality-aware features learned by the network model to quality scores. The distortion type classification and distortion level classification tasks help the network model learn more discriminative quality-aware features.

[0095] This example is implemented on the pytorch platform. The details are as follows:

[0096] Step 1: First, obtain K sampling points of the original point cloud through farthest point sampling. With these sampling points as the center, use the KNN algorithm to find N nearest neighbor points. Finally, a point cloud is divided into K patches.

[0097] Step 2: Input these patches into the network, build a KNN graph in 3D space for each point in the patch and extract its geometric and color gradient features;

[0098] Step 3: Each point in the patch is subjected to extended 3D convolution to extract local features, and the receptive field is gradually expanded in a layered hierarchy. After multiple layers of convolution operations, effective quality representation features are obtained.

[0099] Step 4: After passing through the position-adaptive self-attention mechanism module, the features are input into the multi-task learning module to obtain the quality score of each patch. The quality scores of all patches of a point cloud are averaged as the final quality score of the point cloud.

[0100] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction. To avoid unnecessary repetition, the present invention will not further describe various possible combinations.

Claims

1. A point cloud quality assessment method based on non-local geometric and color gradient aggregation maps, characterized by: This method uses a non-local geometric and color gradient aggregation graph model to evaluate point cloud quality. The non-local geometric and color gradient aggregation graph model includes a geometric and color gradient extraction module, first to fourth local graph aggregation modules, first and second position-adaptive self-attention mechanism modules, and a task learning module. The first to fourth local graph aggregation modules are used to perform local graph aggregation on input features to extract deep features of the input features, and the first and second position-adaptive self-attention mechanism modules are used to optimize the weights of the input features. The point cloud is input into the geometry and color gradient extraction module to obtain the geometric gradient features and color gradient features of the point cloud; the geometric gradient features are input into the first local graph aggregation module to obtain the aggregated geometric gradient features, and the color gradient features are input into the second local graph aggregation module to obtain the aggregated color gradient features; the aggregated geometric gradient features and the aggregated color gradient features are cascaded, and the cascaded features are input into the third local graph aggregation module, the features output by the third local graph aggregation module are sequentially subjected to the Relu activation function and the maximum pooling layer and then input into the first position adaptive self-attention mechanism module, the first position adaptive self-attention mechanism module inputs the weight-optimized features into the fourth local graph aggregation module, the features output by the fourth local graph aggregation module are subjected to the Relu activation function and then input into the second position adaptive self-attention mechanism module, the features output by the second position adaptive self-attention mechanism module are subjected to the global maximum pooling layer to obtain the quality perception features, and the task learning module maps the quality perception features to the quality score to obtain the quality score of the point cloud.

2. The point cloud quality assessment method based on non-local geometric and color gradient aggregation maps according to claim 1 is characterized in that: The geometry and color gradient extraction module uses the KNN algorithm to construct the cth point v in the point cloud. c The spatial graph of , and find v c The m nearest neighbor points of , and then use the following formula to extract point v c Geometric gradient features N represents the total number of points contained in any block of a point cloud; Among them, i, j and k are unit vectors. and Represents point v c Geometric gradients in the x, y, and z directions; and The expression is as follows: in, Represents point v c The x-coordinates of all neighboring points, Represents point v c The y coordinates of all neighboring points, Represents point v c The z coordinates of all neighboring points; c ,y c and z c Represents point v c The coordinate values ​​in the x-direction, y-direction, and z-direction; ∈ is a constant term; where the expressions of Δx, Δy, and Δz are as follows: in, Represents point v c The maximum x-coordinate of all neighboring points, Represents point v c The minimum x-coordinate value of all neighboring points, Represents point v c The maximum value of the y coordinate among all neighboring points, Represents point v c The minimum y coordinate of all neighboring points, Represents point v c The maximum value of the z coordinate among all neighboring points, Represents point v c The minimum value of the z coordinate of all neighboring points; The geometry and color gradient extraction module uses the following formula to extract point v c Color gradient feature G c : d1=max{R m {||d k,c ||}} Among them, C(v c ) represents point v c The color attribute, C(v k ) represents point v c The kth nearest neighbor point v k The color attribute, ||d k,c || represents v k With v c The Euclidean distance between m {||d k,c ||} represents the m nearest neighbor points and v c The Euclidean distance set of Used to weigh different distances to point v c The effect of color gradient, d k,c Represents point v c and the neighboring point v k The direction vector.

3. The point cloud quality assessment method based on non-local geometric and color gradient aggregation maps according to claim 2, characterized in that: The first to fourth local graph aggregation modules perform the following operations on the input features: Conv F =<F(v c ),w(v c )>+max<F(v k ),w(v k )>cosθ Among them, <.> represents the inner product operation, Conv F is the output of the first to fourth local graph aggregation modules, F(v c ) is the input feature of the first to fourth local graph aggregation modules, which is the point v c The corresponding feature, w(v c ) is F(v c ) weight, F(v k ) is the neighboring point v k The characteristics of w(v k ) is F(v k ) weight; the first local graph aggregation module input feature F(v c ) is point v c The geometric gradient feature, F(v k ) is the neighboring point v k The geometric gradient features of the second local graph aggregation module input feature F(v c ) is point v c Color gradient features, F(v k ) is the neighboring point v k The color gradient feature of the third local graph aggregation module input feature F(v c ) is point v c Aggregate feature, the aggregate feature is the feature F(v k ) is the neighboring point v k Aggregation features of the fourth local graph aggregation module input feature F(v c ) is point v c The aggregated features after weight optimization, F(v k ) is point v k Aggregate features after weight optimization; d k,c Represents point v c and the neighboring point v k The direction vector, k s is the initialization direction vector, ||k s || is the Euclidean distance between the initialized direction vector and the origin.

4. The point cloud quality assessment method based on non-local geometric and color gradient aggregation maps according to claim 1, characterized in that: The first and second position adaptive self-attention mechanism modules introduce multi-head self-attention, which includes N h There are three self-attention blocks, each with the same structure. Any self-attention block uses three linear layers to generate the corresponding three feature sequences Q, K, and V: Q=(F+PAE)W Q ;K=(F+PAE)W K ;V=(F+PAE)W V Among them, F is the input feature of the self-attention block, PAE is the preset position adaptive embedding sequence, W Q 、W K and W V Both represent transformation matrices; The feature self-attention (Q, K, V) output by the self-attention block is: Where d represents the dimension of the input feature and T is the matrix transpose; The output feature MHSA of multi-head self-attention is: Among them, Q H ,K H and V H represents the feature sequence generated by the H-th self-attention block, W o is the output projection matrix.

5. The point cloud quality assessment method based on non-local geometric and color gradient aggregation maps according to claim 1, characterized in that: The task learning module also includes classifying the distortion type of the point cloud and classifying the distortion level of the point cloud.

Images