Point cloud quality evaluation method, device, storage medium and equipment

Abstract

The present invention discloses a point cloud quality evaluation method, device, storage medium and equipment, belonging to the technical field of point cloud quality evaluation. The method comprises obtaining a 3D point cloud; inputting the 3D point cloud into a trained visual interaction model based on point cloud structural information features to obtain a quality score and complete point cloud quality evaluation; extracting quality perception features of local blocks of point clouds using a position vector weighting method based on self-attention, and training the model using a self-regulating learning method; wherein the position vector weighting method based on self-attention utilizes a self-attention mechanism to perform weight calculation on the relative positions of local block points, and then utilizes the weight to perform weight calculation on preprocessed features; and self-regulating learning helps to alleviate the noise label problem of the model during block sampling training, thereby improving the accuracy of model training.

Description

Technical Field

[0001] The present invention relates to a point cloud quality evaluation method, device, storage medium and equipment, and belongs to the technical field of point cloud quality evaluation. Background Art

[0002] In virtual reality and immersive communication applications, effective point cloud quality perception is key to monitoring and improving user experience. In recent years, researchers at home and abroad have conducted in-depth research on full-reference and no-reference point cloud quality perception modeling techniques.

[0003] Since full-reference point cloud quality assessment methods require reference point cloud data, raw data is often extremely difficult to obtain in practical applications, limiting their application in various fields. Therefore, relevant researchers have focused on no-reference point cloud quality assessment methods. These methods aim to assess the quality of distorted point clouds by building models when the reference point cloud is unknown. Zhang Zicheng et al. proposed projecting 3D models into quality-related geometric and color feature domains. They then extracted quality-perceptual features using 3D natural scene statistics and entropy, and regressed these features into visual quality scores using a support vector machine. Limited by domain knowledge, these handcrafted feature-based methods often have limited representation capabilities. Given the powerful representation learning capabilities of deep learning, researchers have attempted to use deep neural networks to learn point cloud quality-perceptual features. However, the unstructured nature of point clouds limits the application of convolutional neural networks on 3D point clouds. Therefore, researchers projected 3D point clouds from multiple angles into images or videos and then used convolutional neural networks to extract perceptual quality features. Liu Qi et al. proposed a multi-view deep learning method that jointly extracts features from multiple viewpoints of a point cloud and integrates them with a distortion type recognition module for quality prediction. Drawing on multi-view concepts, Fan Yu et al. obtained three video sequences by rotating a camera around a point cloud along three specific orbits. They then modified the ResNet3D model to extract the correlation between visual perceptual features and subjective quality scores in the video sequences. The core idea of ​​this approach is to project the point cloud into an image and then use an end-to-end convolutional neural network training method to extract quality-perceptual features and fit the quality score. However, this projection-based approach only considers color information and ignores geometric distortion, which is highly relevant to human perception.

[0004] Deep learning-based point cloud quality assessment is considered one of the most promising technologies in this field. However, due to the disorder and unstructured nature of point clouds, extracting quality-perceiving features directly from 3D point clouds using deep neural networks is a significant challenge. Current mainstream approaches project point clouds into images or videos and then use CNNs to extract perceptual quality features and fit quality scores to the projected images or videos. These methods attempt to leverage CNNs' powerful learning capabilities for image features. However, these projection-based methods only consider the color information of the point cloud and ignore the impact of geometric distortion on the perceptual quality of the point cloud, resulting in low accuracy in point cloud quality assessment. Summary of the Invention

[0005] The purpose of the present invention is to provide a point cloud quality evaluation method, device, storage medium and equipment to solve the problem of low accuracy existing in the prior art.

[0006] To achieve the above objectives, the present invention is implemented by adopting the following technical solutions:

[0007] In a first aspect, the present invention provides a point cloud quality evaluation method, comprising:

[0008] Get 3D point cloud;

[0009] The 3D point cloud is input into a trained visual interaction model based on point cloud structural information features to obtain a quality score and complete the point cloud quality evaluation.

[0010] In combination with the first aspect, further, the visual interaction model includes two branches, a feature interaction module and a fully connected layer;

[0011] The two branches are a content-aware stream and a distortion-aware stream based on point cloud structural information features, respectively, and content-aware features and distortion-aware features in the 3D point cloud are extracted through the content-aware stream and the distortion-aware stream respectively;

[0012] The feature interaction module performs a dot product operation on the content-aware feature and the distortion-aware feature, and outputs the operation result to the fully connected layer;

[0013] The fully connected layer converts the operation result into a quality score and outputs it.

[0014] In combination with the first aspect, further, after the 3D point cloud is input into the visual interaction model, the 3D point cloud is first preprocessed as follows:

[0015] Represent the 3D point cloud as , Represents the sampling point, and the coordinates of the sampling point are , It is the sampling point Axis coordinates, It is the sampling point Axis coordinates, It is the sampling point Axis coordinates, Represents the 1st to 2nd sampling points respectively. adjacent points, The coordinates of the adjacent points are , It is adjacent points Axis coordinates, It is adjacent points Axis coordinates, It is adjacent points axis coordinates;

[0016] No. The position vector of a neighboring point is calculated as the vector between the sampling point and the neighboring point:

[0017] ;

[0018] in, It is The position vectors of adjacent points, It is Between adjacent points and sampling points The difference between the axis coordinates, It is Between adjacent points and sampling points The difference between the axis coordinates, It is Between adjacent points and sampling points The difference between axis coordinates;

[0019] No. The distances of the neighboring points to the sample point are calculated as the modulus of the corresponding position vectors:

[0020] ;

[0021] in, It is The distance from the adjacent point to the sampling point;

[0022] Extract the brightness of each point in the 3D point cloud and calculate the brightness difference between each adjacent point and the sampling point;

[0023] The position vector of each adjacent point, the distance from each adjacent point to the sampling point, the brightness of each adjacent point, and the brightness difference between each adjacent point and the sampling point are used as preprocessing features. The preprocessing features are expressed as:

[0024] ;

[0025] ;

[0026] in, It is The preprocessed feature vectors of adjacent points, is the preprocessing feature, T represents transpose, It is The brightness of adjacent points, It is The brightness difference between adjacent points and the sampling point.

[0027] In combination with the first aspect, further, a point cloud structure information feature extraction module is provided in both the content perception stream and the distortion perception stream. The point cloud structure information feature extraction module is a self-attention module. The position vector of each adjacent point is mapped to the weighted weight of each adjacent point through the point cloud structure information feature extraction module to extract point cloud structure information features based on the position vector weighting.

[0028] In combination with the first aspect, further, the expression of the point cloud structure information feature is:

[0029] ;

[0030] in, is the point cloud structure information feature, It is the weight matrix obtained by concatenating the weight vectors output by all attention heads. represents matrix multiplication, It is the input of the point cloud structure information feature extraction module.

[0031] In combination with the first aspect, further, the weight matrix is ​​calculated by the following formula:

[0032] ;

[0033] in, Represents the 1st to the The weight vector output by the attention head, Represents a splicing operation;

[0034] ;

[0035] ;

[0036] ;

[0037] ;

[0038] in, Indicates the The weight vector output by the attention head, represents the attention head, is the position vector contained in the input of the point cloud structure information feature extraction module, represents transpose, It is The first linear mapping matrix of the attention head, It is The second linear mapping matrix of the attention head, It is The third linear mapping matrix of the attention heads, yes After the The linear output obtained by the first linear mapping of the attention head, yes After the The linear output obtained by the second linear mapping of the attention head, yes After the The linear output obtained by the third linear mapping of the attention head.

[0039] In combination with the first aspect, further, the visual interaction model is trained by the following method:

[0040] A label noise suppression method based on self-regulating learning is adopted. During the training of the visual interaction model, the label confidence is calculated using the model that has been trained to be stable, so as to achieve self-regulation of the model's loss function.

[0041] The process of training the visual interaction model includes:

[0042] The distortion perception stream is trained using the distortion classification task, and the cost function adopts the cross entropy loss between the predicted and actual distortion types;

[0043] The entire visual interaction model is trained by freezing the parameters of the distortion perception stream;

[0044] The following methods are used to determine whether the model has reached stability:

[0045] During the training process, the validation error of each epoch is recorded, and the variance of the validation error of L consecutive epochs is calculated using a sliding window.

[0046] When the variance is greater than the preset threshold, the current model has not reached stability;

[0047] When the variance is less than the preset threshold, the current model reaches stability;

[0048] The label confidence is calculated by the following method:

[0049] During each training parameter update iteration, the trained model is used to perform quality scoring on the samples of the current mini-batch to obtain a preliminary quality score, which is then normalized to obtain the label confidence:

[0050] ;

[0051] in, It is a sample The prediction score is used as the label confidence, is the preliminary quality score, is the mean of the preliminary quality scores, is the standard deviation of the preliminary quality scores;

[0052] The expression of the cost function is:

[0053] ;

[0054] in, is the cost function, and The samples The label scores and prediction scores of To control the hyperparameters of weight adjustment, is the sample size.

[0055] In a second aspect, the present invention further provides a point cloud quality evaluation device, comprising:

[0056] The point cloud acquisition module is configured to: acquire 3D point clouds;

[0057] The quality evaluation module is configured to: input the 3D point cloud into a trained visual interaction model based on point cloud structural information features, obtain a quality score, and complete point cloud quality evaluation.

[0058] In a third aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the point cloud quality assessment method as described in any one of the first aspects is implemented.

[0059] In a fourth aspect, the present invention further provides a device comprising:

[0060] a memory for storing instructions;

[0061] The processor is configured to execute the instructions so that the device performs operations to implement the point cloud quality assessment method as described in any one of the first aspects.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] The present invention provides a point cloud quality evaluation method, device, storage medium and equipment, which utilize a trained visual interaction model to extract local perception features (i.e., point cloud structural information features), thereby improving the accuracy of point cloud quality evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a flow chart of a point cloud quality evaluation method provided by an embodiment of the present invention;

[0065] Figure 2 is a schematic diagram of a visual interaction model provided by an embodiment of the present invention;

[0066] Figure 3 It is a schematic diagram of the optimization process of the non-uniform distortion suppression model based on self-adjusting learning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0068] Example 1

[0069] like Figure 1 As shown, the present invention provides a point cloud quality evaluation method, comprising the following steps:

[0070] S1. Obtain 3D point cloud.

[0071] S2. Input the 3D point cloud into a trained visual interaction model based on point cloud structural information features to obtain a quality score and complete point cloud quality evaluation.

[0072] The present invention adopts a point cloud structure information feature extraction module based on self-attention position vector weighting to extract quality perception features of 3D point clouds (i.e., point cloud structure information features). The point cloud structure information feature extraction module can simultaneously extract geometric and color structure information from point clouds.

[0073] The input of the point cloud structural information feature extraction module is a 3D point cloud. In order to better extract the structural information of irregular point clouds, the input 3D point cloud needs to be preprocessed, including:

[0074] Represent the 3D point cloud as , Represents the sampling point, and the coordinates of the sampling point are , It is the sampling point Axis coordinates, It is the sampling point Axis coordinates, It is the sampling point Axis coordinates, Represents the 1st to 2nd sampling points respectively. adjacent points, The coordinates of the adjacent points are , It is adjacent points Axis coordinates, It is adjacent points Axis coordinates, It is adjacent points axis coordinates;

[0075] No. The position vector of a neighboring point is calculated as the vector between the sampling point and the neighboring point:

[0076] ;

[0077] in, It is The position vectors of adjacent points, It is Between adjacent points and sampling points The difference between the axis coordinates, It is Between adjacent points and sampling points The difference between the axis coordinates, It is Between adjacent points and sampling points The difference between axis coordinates;

[0078] No. The distances of the neighboring points to the sample point are calculated as the modulus of the corresponding position vectors:

[0079] ;

[0080] in, It is The distance from the adjacent point to the sampling point;

[0081] Extract the brightness of each point in the 3D point cloud and calculate the brightness difference between each adjacent point and the sampling point;

[0082] The position vector of each adjacent point, the distance from each adjacent point to the sampling point, the brightness of each adjacent point, and the brightness difference between each adjacent point and the sampling point are used as preprocessing features. The preprocessing features are expressed as:

[0083] ;

[0084] ;

[0085] in, It is The preprocessed feature vectors of adjacent points, is the preprocessing feature, T represents transpose, It is The brightness of adjacent points, It is The brightness difference between adjacent points and the sampling point, .

[0086] Point cloud structural information is represented as the geometric and color changes of a 3D point cloud. The present invention calculates the position vector of each adjacent point in the 3D point cloud, the distance from each adjacent point to a sampling point, the brightness of each adjacent point, and the brightness difference between each adjacent point and the sampling point. The distance and position vector reflect the geometric information changes between local adjacent points in terms of magnitude and direction, respectively, while the brightness difference reflects the color-related structural information of the local block.

[0087] After preprocessing the features, the next step is to extract perceptual quality features. On the one hand, perceptual quality features must meet certain invariance requirements, including translational and rotational invariance. On the other hand, adjacent points have different relative positions with respect to the same sampling point, so local changes between adjacent points should be weighted differently when calculating the structural information of the 3D point cloud block. This method leverages the powerful learning representation capabilities of deep neural networks to extract point cloud structural information features (quality-perceptual features) based on weighted position vectors.

[0088] The idea of ​​extracting the structural information features of the point cloud in this invention is to treat the weighted weight as a nonlinear continuous function of the position vector. Specifically, the invention uses the self-attention module to fit the nonlinear continuous function f , that is, the position vector is mapped to the weighted weight of each point through the self-attention module.

[0089] The expression of point cloud structural information characteristics is:

[0090] ;

[0091] in, is the point cloud structure information feature, It is the weight matrix obtained by concatenating the weight vectors output by all attention heads. represents matrix multiplication, It is the input of the point cloud structure information feature extraction module. .

[0092] The weight matrix is ​​calculated by the following formula:

[0093] ;

[0094] in, Respectively represent the 1st to The weight vector output by the attention head, Represents a splicing operation;

[0095] ;

[0096] ;

[0097] ;

[0098] ;

[0099] in, Indicates the The weight vector output by the attention head, represents the attention head, is the position vector contained in the input of the point cloud structure information feature extraction module, represents transpose, It is The first linear mapping matrix of the attention head, It is The second linear mapping matrix of the attention head, It is The third linear mapping matrix of the attention heads, yes After the The linear output obtained by the first linear mapping of the attention head, yes After the The linear output obtained by the second linear mapping of the attention head, yes After the The linear output obtained by the third linear mapping of the attention head.

[0100] Previous studies have confirmed that the human visual system has a visual masking effect, that is, the same intensity distortion on different image contents appears to have different visibility when observed by the human eye. In other words, the quality perception of image distortion by the human eye is affected by the interaction between image content and distortion. This visual masking effect also exists when the human eye observes point clouds, so a visual interaction model based on the structural information characteristics of point clouds is adopted. Figure 2As shown in the figure, the model consists of a two-stream network based on a point cloud structural information feature module, including a content-aware feature stream and a distortion feature stream based on structural information. Both the content-aware feature stream and the distortion-aware feature stream use the point cloud structural information feature module and a multi-layer Transformer encoder for feature extraction, combining adaptive pooling and max pooling operations to obtain the corresponding features. Considering the differences in the properties of the two feature types, the two branch network streams can use Transformer encoders with different numbers of layers. Finally, feature interaction is achieved by performing a dot product operation on the content-aware features and distortion-aware features output by the two branch networks, and then regressing the quality score using a fully connected layer.

[0101] Due to the unstructured nature of point clouds, most currently proposed no-reference point cloud quality assessment methods employ a "local represents the whole" sampling approach. This involves sampling a large number of local point cloud blocks using the farthest point sampling method, and using the overall point cloud quality score as the quality score for the local point cloud blocks for model training. This approach offers a "divide and conquer" approach, reducing the complexity of processing individual samples during training. Furthermore, a large number of local blocks can be extracted from each point cloud, increasing the number and diversity of training samples, which in turn improves training accuracy and generalization.

[0102] During the training process of current mainstream methods, the sampling blocks of all point clouds used for training are usually shuffled to form a training set, and then samples from each mini-batch are randomly selected. In other words, the samples of each mini-batch may come from different point clouds; in addition, the same weight is used for all sample errors in the loss function. This type of method ignores the impact of non-uniform distortion on model training. The so-called "non-uniform distortion" means that the actual perceptual quality of different local point cloud blocks of the same point cloud varies greatly due to factors such as distortion intensity or changes in point cloud structure. These factors include contrast sensitivity and visual masking effects. Current methods assume that all sampling blocks of the same point cloud have the same quality score. Under the condition of non-uniform distortion, a large number of noise labels will be introduced, which will affect the accuracy of model training.

[0103] This method also employs a "part represents the whole" sampling approach. Unlike current mainstream methods, it builds on this foundation by proposing a label noise suppression method based on self-regulatory learning. Its key concept is to utilize a relatively stable model during training to calculate label confidence, thereby achieving self-regulation of the loss function. Specifically, during parameter update iterations, this method requires that samples within the same mini-batch are randomly selected from the same sampling patch within the point cloud. When the model training is relatively stable, the model is considered to have a certain degree of quality perception capability and can accurately assess the perceptual quality of different sample patches. Given the significant variation in perceptual quality within the same point cloud under non-uniform distortion conditions, the human visual system is often more sensitive to samples with lower perceptual quality. This means that within the same point cloud, the true perceptual quality of sample patches with lower model scores is closer to the overall point cloud MOS value. In other words, the label confidence of samples with lower model scores is higher than that of samples with higher scores. Therefore, this method uses the model score to measure sample label confidence and thereby adjust the weight of each sample in the loss function.

[0104] Stable model judgment based on sliding window:

[0105] During the training process, the validation error of each epoch is recorded, and the variance of the validation error of L consecutive epochs is calculated using a sliding window.

[0106] When the variance is greater than the preset threshold, the current model has not reached stability;

[0107] When the variance is less than the preset threshold, the current model is stable.

[0108] Self-regulating learning based on label confidence, such as Figure 3 Shown, including:

[0109] During each training parameter update iteration, the trained model is used to perform quality scoring on the samples of the current mini-batch to obtain a preliminary quality score, which is then normalized to obtain the label confidence:

[0110] ;

[0111] in, It is a sample The prediction score is used as the label confidence, is the preliminary quality score, is the mean of the preliminary quality scores, is the standard deviation of the preliminary quality scores.

[0112] Model training: The above model is trained in a staged manner. First, the distortion perception stream is trained using the distortion classification task, where the actual distortion type of each point cloud block is the same as the distortion type of the entire point cloud to which it belongs. The cost function uses a cross-entropy loss based on the predicted and actual distortion types; finally, the entire visual interaction model is trained by freezing the parameters of the distortion perception stream. During the iterative parameter update process, this method uses higher weights for samples with lower scores, and lower weights for samples with lower scores. The expression of the cost function is as follows:

[0113] ;

[0114] in, is the cost function, and For each sample The label scores and prediction scores of To control the hyperparameters of weight adjustment, is the sample size.

[0115] In the training phase, the sampled point cloud blocks are used as training samples, and the quality score of the entire point cloud is used as their score. In the testing phase, each sampled point cloud block is given a prediction score, and the average score of the point cloud blocks is used as the final score of the entire point cloud.

[0116] Example 2

[0117] An embodiment of the present invention further provides a point cloud quality evaluation device, comprising:

[0118] The point cloud acquisition module is configured to: acquire 3D point clouds;

[0119] The quality evaluation module is configured to: input the 3D point cloud into a trained visual interaction model based on point cloud structural information features, obtain a quality score, and complete point cloud quality evaluation.

[0120] Example 3

[0121] The present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the point cloud quality evaluation method provided in Example 1 is implemented:

[0122] Get 3D point cloud;

[0123] The 3D point cloud is input into a trained visual interaction model based on point cloud structural information features to obtain a quality score and complete the point cloud quality evaluation.

[0124] Example 4

[0125] The present invention provides a device comprising:

[0126] a memory for storing instructions;

[0127] A processor, configured to execute the instructions so that the device performs operations to implement the point cloud quality assessment method provided in Example 1:

[0128] Get 3D point cloud;

[0129] The 3D point cloud is input into a trained visual interaction model based on point cloud structural information features to obtain a quality score and complete the point cloud quality evaluation.

[0130] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0132] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0134] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A point cloud quality evaluation method, characterized in that: include: Get 3D point cloud; Input the 3D point cloud into a trained visual interaction model based on point cloud structural information features to obtain a quality score and complete the point cloud quality evaluation; The visual interaction model is trained by the following method: A label noise suppression method based on self-regulating learning is adopted. During the training of the visual interaction model, the label confidence is calculated using the model that has been trained to be stable, so as to achieve self-regulation of the model's loss function. The process of training the visual interaction model includes: The distortion perception stream is trained using the distortion classification task, and the cost function adopts the cross entropy loss between the predicted and actual distortion types; The entire visual interaction model is trained by freezing the parameters of the distortion perception stream; The following methods are used to determine whether the model has reached stability: During the training process, the validation error of each epoch is recorded, and the variance of the validation error of L consecutive epochs is calculated using a sliding window. When the variance is greater than the preset threshold, the current model has not reached stability; When the variance is less than the preset threshold, the current model reaches stability; The label confidence is calculated by the following method: During each training parameter update iteration, the trained model is used to perform quality scoring on the samples of the current mini-batch to obtain a preliminary quality score, which is then normalized to obtain the label confidence: ； in, It is a sample The prediction score is used as the label confidence, is the preliminary quality score, is the mean of the preliminary quality scores, is the standard deviation of the preliminary quality scores; The expression of the cost function is: ； in, is the cost function, and The samples The label scores and prediction scores of To control the hyperparameters of weight adjustment, is the sample size.

2. The point cloud quality evaluation method according to claim 1, characterized in that: The visual interaction model includes two branches, a feature interaction module and a fully connected layer; The two branches are a content-aware stream and a distortion-aware stream based on point cloud structural information features, respectively, and content-aware features and distortion-aware features in the 3D point cloud are extracted through the content-aware stream and the distortion-aware stream respectively; The feature interaction module performs a dot product operation on the content-aware feature and the distortion-aware feature, and outputs the operation result to the fully connected layer; The fully connected layer converts the calculation result into a quality score and outputs it.

3. The point cloud quality evaluation method according to claim 1, characterized in that: After the 3D point cloud is input into the visual interaction model, the 3D point cloud is preprocessed as follows: Represent the 3D point cloud as , Represents the sampling point, and the coordinates of the sampling point are , It is the sampling point Axis coordinates, It is the sampling point Axis coordinates, It is the sampling point Axis coordinates, Represents the 1st to 2nd sampling points respectively. adjacent points, The coordinates of the adjacent points are , It is adjacent points Axis coordinates, It is adjacent points Axis coordinates, It is adjacent points axis coordinates; No. The position vector of a neighboring point is calculated as the vector between the sampling point and the neighboring point: ； in, It is The position vectors of adjacent points, It is Between adjacent points and sampling points The difference between the axis coordinates, It is Between adjacent points and sampling points The difference between the axis coordinates, It is Between adjacent points and sampling points The difference between axis coordinates; No. The distances of the neighboring points to the sample point are calculated as the modulus of the corresponding position vectors: ； in, It is The distance from the adjacent point to the sampling point; Extract the brightness of each point in the 3D point cloud and calculate the brightness difference between each adjacent point and the sampling point; The position vector of each adjacent point, the distance from each adjacent point to the sampling point, the brightness of each adjacent point, and the brightness difference between each adjacent point and the sampling point are used as preprocessing features. The preprocessing features are expressed as: ； ； in, It is The preprocessed feature vectors of adjacent points, is the preprocessing feature, T represents transpose, It is The brightness of adjacent points, It is The brightness difference between adjacent points and the sampling point.

4. The point cloud quality evaluation method according to claim 2, characterized in that: Both the content-aware stream and the distortion-aware stream are provided with a point cloud structure information feature extraction module, which is a self-attention module. The point cloud structure information feature extraction module maps the position vector of each adjacent point to the weighted weight of each adjacent point to extract the point cloud structure information features based on the position vector weighting.

5. The point cloud quality evaluation method according to claim 4, characterized in that: The expression of the point cloud structure information feature is: ； in, is the point cloud structure information feature, It is the weight matrix obtained by concatenating the weight vectors output by all attention heads. represents matrix multiplication, It is the input of the point cloud structure information feature extraction module.

6. The point cloud quality evaluation method according to claim 5, characterized in that: The weight matrix is ​​calculated by the following formula: ； in, Respectively represent the 1st to The weight vector output by the attention head, Represents a splicing operation; ； ； ； ； in, Indicates the The weight vector output by the attention head, represents the attention head, is the position vector contained in the input of the point cloud structure information feature extraction module, represents transpose, It is The first linear mapping matrix of the attention head, It is The second linear mapping matrix of the attention head, It is The third linear mapping matrix of the attention heads, yes After the The linear output obtained by the first linear mapping of the attention head, yes After the The linear output obtained by the second linear mapping of the attention head, yes After the The linear output obtained by the third linear mapping of the attention head.

7. A point cloud quality evaluation device, characterized in that: include: The point cloud acquisition module is configured to: acquire 3D point clouds; The quality evaluation module is configured to: input the 3D point cloud into a trained visual interaction model based on point cloud structural information features to obtain a quality score and complete point cloud quality evaluation; The visual interaction model is trained by the following method: A label noise suppression method based on self-regulating learning is adopted. During the training of the visual interaction model, the label confidence is calculated using the model that has been trained to be stable, so as to achieve self-regulation of the model's loss function. The process of training the visual interaction model includes: The distortion perception stream is trained using the distortion classification task, and the cost function adopts the cross entropy loss between the predicted and actual distortion types; The entire visual interaction model is trained by freezing the parameters of the distortion perception stream; The following methods are used to determine whether the model has reached stability: During the training process, the validation error of each epoch is recorded, and the variance of the validation error of L consecutive epochs is calculated using a sliding window. When the variance is greater than the preset threshold, the current model has not reached stability; When the variance is less than the preset threshold, the current model reaches stability; The label confidence is calculated by the following method: During each training parameter update iteration, the trained model is used to perform quality scoring on the samples of the current mini-batch to obtain a preliminary quality score, which is then normalized to obtain the label confidence: ； in, It is a sample The prediction score is used as the label confidence, is the preliminary quality score, is the mean of the preliminary quality scores, is the standard deviation of the preliminary quality scores; The expression of the cost function is: ； in, is the cost function, and The samples The label scores and prediction scores of To control the hyperparameters of weight adjustment, is the sample size.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the point cloud quality assessment method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: include: a memory for storing instructions; The processor is configured to execute the instruction so that the electronic device performs the operation of implementing the point cloud quality assessment method according to any one of claims 1 to 6.

Images