Full-reference image quality evaluation method and system based on local area mask

Abstract

The invention discloses a full-reference image quality evaluation method and system based on a local area mask, and relates to the technical field of image quality evaluation, and the method comprises the specific steps: carrying out the local area information extraction of a distorted image and a reference image, and obtaining an initial local area feature map; performing multi-stage feature extraction on the depth feature map to obtain an initial depth feature map; generating a local area mask based on the initial local area feature map of the reference image, and generating a local area depth feature according to the mask; utilizing a self-attention mechanism and a cross attention mechanism to generate enhanced local area depth features according to the local area depth features; and performing weighted aggregation on the enhanced local area depth features by using the weight matrix to generate weighted fusion local area depth features, splicing the weighted fusion local area depth features with the initial local area feature map, and calculating a quality score. According to the method, the predicted quality score is closer to a perception result of human eyes, and an evaluation standard which better conforms to human visual characteristics is provided for image quality evaluation.

Description

Technical Field

[0001] The present invention relates to the technical field of image quality assessment, and more particularly to a full-reference image quality assessment method and system based on local area mask. Background Art

[0002] Image quality assessment is a basic and critical task in the field of image processing and computer vision, and its importance is self-evident. This technology is widely used in many fields such as image / video coding, super-resolution reconstruction, and image / video visual quality enhancement, providing strong support for the objective measurement and optimization of image quality. Its core lies in building a model that can simulate the perception mechanism of the human visual system, and quantitatively evaluate the overall quality of the image through digital calculation, thereby achieving objective and accurate image perception quality evaluation.

[0003] In the full-reference image quality assessment method, the algorithm estimates the quality score of the distorted image by comparing the human eye perception difference between the reference image (i.e. the original high-quality image) and the image to be evaluated (i.e. the distorted image that may have been processed or compressed). This method plays an irreplaceable role in the evaluation of image compression coding, the setting of test benchmarks for image processing algorithms, and the formulation of optimization goals for image processing tasks due to its intuitiveness and accuracy.

[0004] However, traditional full-reference image quality assessment methods often focus on global quality assessment and may not be sensitive enough to quality changes in local areas of the image. In practical applications, some areas of the image may have a more important impact on the overall quality perception, such as faces, text, or key objects. Therefore, how to effectively extract the features of important areas and improve the accuracy of image quality assessment is an urgent problem that technicians in this field need to solve. Summary of the invention

[0005] In view of this, the present invention provides a full-reference image quality evaluation method and system based on local area mask, which overcomes the above-mentioned defects.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A full-reference image quality evaluation method based on local area mask, the specific steps are:

[0008] Obtaining a distorted image and a reference image;

[0009] Extracting local area information from the distorted image and the reference image respectively to obtain an initial local area distortion feature map and an initial local area reference feature map;

[0010] Performing multi-stage feature extraction on the distorted image and the reference image respectively and generating an initial distorted depth feature map and an initial reference depth feature map;

[0011] Based on the initial local area reference feature map, a plurality of local area masks are generated, and the plurality of local area masks are respectively combined with the initial distorted depth feature map and the initial reference depth feature map to generate a plurality of local area distorted depth features and a plurality of local area reference depth features;

[0012] Based on the self-attention mechanism, multiple local area reference depth features are fused to generate an enhanced local area reference depth feature; based on the cross-attention mechanism, the enhanced local area reference depth feature is used as a guide feature to generate an enhanced local area distortion depth feature according to multiple local area distortion depth features;

[0013] A weight matrix is ​​generated according to the enhanced local area reference depth feature, and the enhanced local area reference depth feature and the enhanced local area distortion depth feature are weightedly aggregated by using the weight matrix to generate a weighted fused local area reference depth feature and a weighted fused local area distortion depth feature;

[0014] The weighted fused local area reference depth feature, the weighted fused local area distortion depth feature, the initial local area distortion feature map and the initial local area reference feature map are spliced, and a quality score is calculated.

[0015] Optionally, after acquiring the distorted image and the reference image, the distorted image and the reference image need to be preprocessed.

[0016] Optionally, the step of acquiring the initial local area distortion feature map or the initial local area reference feature map is:

[0017] The distorted image or the reference image is input into a local area sampling network, and the initial local area distortion feature map or the initial local area reference feature map is generated through multiple iterations.

[0018] Optionally, the step of acquiring the initial distorted depth feature map or the initial reference depth feature map is:

[0019] Extracting depth features of the distorted image or the reference image according to a depth extraction model, and selecting the depth features of a plurality of pre-set stages as a pre-depth feature map;

[0020] The pre-depth feature map is up-sampled to a size of the distorted image or the reference image to generate the initial distorted depth feature map or the initial reference depth feature map.

[0021] Optionally, the step of acquiring the local area distortion depth feature or the local area reference depth feature is:

[0022] Generating the local area masks according to the initial local area reference feature map, the number of the local area masks being the same as the number of local areas;

[0023] The initial distorted depth feature map or the initial reference depth feature map is multiplied by each of the local area masks to obtain a selected local area distortion feature or a selected local area reference feature;

[0024] The selected local area distortion feature or the selected local area reference feature is globally averaged pooled to obtain the local area distortion depth feature or the local area reference depth feature.

[0025] Optionally, the step of acquiring the weighted fusion local area reference depth feature or the weighted fusion local area distortion depth feature is:

[0026] Generate the weight matrix using a multi-layer perceptron and a Softmax function according to the enhanced local area reference depth feature;

[0027] The enhanced local area reference depth features or the enhanced local area distorted depth features at different stages are weighted averaged according to the weight matrix to generate the weighted fused local area reference depth features or the weighted fused local area distorted depth features.

[0028] Optionally, the quality score acquisition step is:

[0029] Splicing the initial local area distortion feature map and the weighted fusion local area distortion depth feature or the initial local area reference feature map and the weighted fusion local area reference depth feature in the channel dimension to obtain a distorted image splicing feature or a reference image splicing feature;

[0030] Splicing the distorted image stitching features and the reference image stitching features in the channel dimension to obtain an overall stitching image feature;

[0031] A quality score is calculated based on the overall stitched image features.

[0032] A full-reference image quality assessment system based on local area mask, comprising:

[0033] An image acquisition module, used for acquiring a distorted image and a reference image;

[0034] A local area division module, used to extract local area information from the distorted image and the reference image respectively, to obtain an initial local area distortion feature map and an initial local area reference feature map;

[0035] A depth feature extraction module, used to perform multi-stage feature extraction on the distorted image and the reference image respectively and generate an initial distorted depth feature map and an initial reference depth feature map;

[0036] A depth feature processing module, configured to generate a plurality of local area masks based on the initial local area reference feature map, wherein the plurality of local area masks are respectively combined with the initial distorted depth feature map and the initial reference depth feature map to generate a plurality of local area distorted depth features and a plurality of local area reference depth features;

[0037] A feature enhancement module is used to generate an enhanced local area reference depth feature by fusing multiple local area reference depth features based on a self-attention mechanism; and to generate an enhanced local area distortion depth feature based on multiple local area distortion depth features using the enhanced local area reference depth feature as a guide feature based on a cross-attention mechanism;

[0038] A weighted fusion module, used to generate a weight matrix according to the enhanced local area reference depth feature, and use the weight matrix to weightedly aggregate the enhanced local area reference depth feature and the enhanced local area distortion depth feature to generate a weighted fused local area reference depth feature and a weighted fused local area distortion depth feature;

[0039] The quality evaluation module is used to splice the weighted fused local area reference depth feature, the weighted fused local area distortion depth feature, the initial local area distortion feature map and the initial local area reference feature map, and calculate the quality score.

[0040] It can be seen from the above technical solutions that the present invention discloses a full-reference image quality evaluation method and system based on local area mask, which has the following beneficial effects compared with the prior art:

[0041] 1. Improve the accuracy of image quality evaluation: The present invention can accurately extract the local area features and depth features of the reference image and the distorted image through the local area masking technology. Combined with the self-attention mechanism and the cross-attention mechanism, the model can better focus on the key local areas in the image, and use the global information to guide the learning of each local area. It not only introduces the information of other local areas, but also enables the model to extract more fine-grained features, thereby improving the accuracy and reliability of image quality evaluation;

[0042] 2. Enhance the model's sensitivity to local details: The deep features extracted by the local area mask, combined with the pooling operation, can fuse the features of each pixel in the local area, providing a good foundation for subsequent feature extraction; it makes the model more sensitive to the local details of the image, can capture richer image information, and help improve the precision and recognition of quality evaluation;

[0043] 3. Improve the stability of quality score prediction: The present invention generates a weight matrix using the local area depth features of the reference image, performs weighted averaging on the reference features and distortion features, and obtains the final weighted feature map; it adaptively generates the weight coefficients of the local area features at different scales in the image, thereby being able to more accurately reflect the quality status of different areas in the image; it improves the stability of quality score prediction, making the prediction result more reliable;

[0044] 4. Optimize computational efficiency and generalization ability: This invention combines the advantages of local area features and global information, and realizes effective fusion and extraction of features through the attention mechanism, which not only improves computational efficiency but also makes the model more generalizable; it can be applied to different types of distorted images and reference images, providing a more universal and effective solution for image quality evaluation;

[0045] 5. Quality score prediction that is closer to human eye perception: The present invention concatenates the extracted weighted features and the initial local area features, performs supervised learning through a multi-layer perceptron, and regresses to the quality score between the reference image and the distorted image. This method makes the predicted quality score closer to the human eye's perception result, and provides an evaluation standard for image quality evaluation that is more in line with human visual characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0047] Figure 1 A schematic flow chart of a full-reference image quality assessment method based on a local area mask provided by the present invention;

[0048] Figure 2 A schematic diagram of the architecture of the full-reference image quality assessment method based on local area mask provided by the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] This embodiment discloses a full reference image quality evaluation method based on local area mask, such as Figure 1 As shown, the specific steps are:

[0051] S1, obtaining a distorted image and a reference image;

[0052] S2, extracting local area information from the distorted image and the reference image respectively to obtain an initial local area distortion feature map and an initial local area reference feature map;

[0053] S3, performing multi-stage feature extraction on the distorted image and the reference image respectively and generating an initial distorted depth feature map and an initial reference depth feature map;

[0054] S4, generating multiple local area masks based on the initial local area reference feature map, and respectively generating multiple local area distorted depth features and multiple local area reference depth features with the initial distorted depth feature map and the initial reference depth feature map;

[0055] S5. Based on the self-attention mechanism, multiple local area reference depth features are fused to generate enhanced local area reference depth features; based on the cross-attention mechanism, the enhanced local area reference depth features are used as the guiding features, and the enhanced local area distortion depth features are generated according to multiple local area distortion depth features;

[0056] S6. Generate a weight matrix according to the enhanced local area reference depth feature, and use the weight matrix to weightedly aggregate the enhanced local area reference depth feature and the enhanced local area distortion depth feature to generate a weighted fused local area reference depth feature and a weighted fused local area distortion depth feature;

[0057] S7, concatenating the weighted fused local area reference depth feature, the weighted fused local area distortion depth feature, the initial local area distortion feature map and the initial local area reference feature map, and calculating the quality score.

[0058] In one embodiment, in S1, after the distorted image and the reference image are acquired, the distorted image and the reference image need to be preprocessed.

[0059] Furthermore, in S1, the distorted image and the reference image are first obtained, and then the images are uniformly resized to a size of 224x224 pixels using image processing software or a library, thereby ensuring consistency of subsequent processing and effectiveness of comparison.

[0060] like Figure 2 As shown, in one embodiment, in S2, the steps of obtaining the initial local area distortion feature map or the initial local area reference feature map are:

[0061] The distorted image or the reference image is input into the local area sampling network, and an initial local area distortion feature map or an initial local area reference feature map is generated through multiple iterations.

[0062] Furthermore, in S2, the reference image and the distorted image unified into the same size are input into the local area sampling network to perform local area segmentation on the reference image and the distorted image. Through multiple iterations (in this embodiment, the number of iterations is set to 10 times), a local area image corresponding to the image is generated, and the extracted initial local area feature map (including the initial local area distortion feature map or the initial local area reference feature map) is output. The initial local area feature map contains information such as texture, color and shape of the image.

[0063] In S3, the steps of obtaining the initial distorted depth feature map or the initial reference depth feature map are:

[0064] S31, extracting depth features of the distorted image or the reference image according to the depth extraction model, and selecting depth features of a plurality of pre-set stages as a pre-depth feature map;

[0065] S32. Upsample the pre-depth feature map to the size of the distorted image or the reference image to generate an initial distorted depth feature map or an initial reference depth feature map.

[0066] Furthermore, a deep extraction model, i.e., a pre-trained VGG-16 neural network model, is used to extract the deep features of the reference image and the distorted image, and the output feature maps of the 2nd, 3rd, and 4th stages in the VGG-16 network are taken out as pre-depth feature maps, which represent the visual features of the image at different levels. Then, the number of channels of the output feature maps of the three stages is unified to D through a 1x1 convolutional layer, and the feature maps are upsampled to the original image size (224*224) using bilinear interpolation to generate an initial depth feature map, which includes an initial distorted depth feature map and an initial reference depth feature map.

[0067] In one embodiment, in S4, the step of acquiring the local area distortion depth feature or the local area reference depth feature is:

[0068] S41, generating a local region mask according to the initial local region reference feature map, the number of the local region masks being the same as the number of the local regions;

[0069] S42, multiplying the initial distorted depth feature map or the initial reference depth feature map by each local area mask respectively to obtain a selected local area distortion feature or a selected local area reference feature;

[0070] S43, performing global average pooling on the selected local area distortion feature or the selected local area reference feature to obtain the local area distortion depth feature or the local area reference depth feature.

[0071] Further, in S4, a local region mask is generated according to the initial local region reference feature map. For each local region, a local region mask is generated, which allows the current local region to pass through and masks all other local regions. Then, the initial depth feature map is multiplied by the mask to extract the depth feature of each local region, and its expression is:

[0072] F ij =M i ⊙D j ;

[0073] in,

[0074] Where M i is the mask of the i-th local area; D j is the initial depth feature map of the jth stage; ⊙ represents element-by-element multiplication (Hadamard multiplication); F ij is the feature of the initial depth feature map of the jth stage in the i-th local area.

[0075] Furthermore, the initial distorted depth feature map and the initial reference depth feature map are multiplied by each mask to obtain a selected local region feature, which includes the selected local region distorted feature or the selected local region reference feature;

[0076] The selected local area features are globally averaged pooled to obtain a feature vector with a length and width of 1. This feature is the feature vector corresponding to the local area, and finally all local area depth feature vectors are obtained.

[0077] In one embodiment, in S5, for the local region reference depth feature, a self-attention mechanism is used to use the global local region feature to guide each local region feature to learn richer features with reference to the global information, and then fuse other local regions to obtain the globally guided enhanced local region reference depth feature;

[0078] For distorted images, the globally guided enhanced local region reference depth features are used as guiding features, and the cross-attention technique is adopted to guide the local region distorted depth features to learn the feature map of the comprehensive attention mechanism, that is, the enhanced local region distorted depth features.

[0079] Furthermore, for the reference image, the self-attention mechanism is used to fuse the depth features between local regions through global average pooling and multi-head attention mechanism to obtain the enhanced local region reference depth features. For the distorted image, the cross attention mechanism is used to extract the features of the distorted image with the enhanced local region reference depth features as a guide. The expression is:

[0080] Q R =F R W Q

[0081] K R =F R W K ;

[0082] V R =F R W V

[0083]

[0084] Where W Q , W K , W V , are all learnable weight matrices; F R is the local area reference depth feature; F D is the local area distortion depth feature; Q R and Q D denote the query of the reference image and the distorted image respectively; K R and K D and V R and V D Respectively represent the key and value of the corresponding image; and They represent the enhanced local area distortion depth feature and the enhanced local area reference depth feature respectively.

[0085] In one embodiment, in S6, the step of obtaining the weighted fusion local area reference depth feature or the weighted fusion local area distortion depth feature is:

[0086] S61, generating a weight matrix using a multi-layer perceptron and a Softmax function according to the enhanced local region reference depth feature;

[0087] S62. Perform weighted averaging of enhanced local area reference depth features or enhanced local area distorted depth features at different stages according to the weight matrix to generate a weighted fused local area reference depth feature or a weighted fused local area distorted depth feature.

[0088] Furthermore, the number of channels of the enhanced local area depth features (including enhanced local area reference depth features and enhanced local area distortion depth features) is adjusted to 1 through global average pooling, and then a weight matrix is ​​generated using a multi-layer perceptron and a Softmax function. The weight matrix is ​​used to perform weighted averaging of the depth features at different stages to generate weighted fused local area depth features (including weighted fused local area reference depth features and weighted fused local area distortion depth features). It can be expressed as:

[0089]

[0090] W = Softmax(MLP(G));

[0091]

[0092] Where D is the number of channels, MLP is a learnable multi-layer perceptron, and W is the corresponding adaptive weight matrix. and are the weighted features corresponding to the reference image and the distorted image respectively; i, j represent the features of the i-th local region and the j-th stage respectively.

[0093] In one embodiment, in S7, the quality score acquisition step is:

[0094] S71, splicing the initial local area distortion feature map and the weighted fusion local area distortion depth feature or the initial local area reference feature map and the weighted fusion local area reference depth feature in the channel dimension to obtain a distorted image splicing feature or a reference image splicing feature;

[0095] S72, stitching the distorted image stitching features and the reference image stitching features in the channel dimension to obtain an overall stitching image feature;

[0096] S73. Calculate a quality score based on overall stitched image features.

[0097] Furthermore, the initial local region feature map and the weighted fusion local region depth feature are spliced ​​in the channel dimension to obtain the spliced ​​features corresponding to the reference image and the distorted image; then, the spliced ​​features of the reference image and the distorted image are flattened, and the flattened features are spliced ​​in the channel dimension to obtain a one-dimensional vector, which is input into a multi-layer perceptron, which includes two hidden layers, each with 128 neurons, and uses the ReLU activation function. Through supervised learning, the quality score between the reference image and the distorted image is regressed.

[0098] This embodiment also includes a full-reference image quality assessment system based on a local area mask, including:

[0099] An image acquisition module, used for acquiring a distorted image and a reference image;

[0100] A local area division module is used to extract local area information from the distorted image and the reference image respectively to obtain an initial local area distortion feature map and an initial local area reference feature map;

[0101] A depth feature extraction module, used to perform multi-stage feature extraction on the distorted image and the reference image respectively and generate an initial distorted depth feature map and an initial reference depth feature map;

[0102] A depth feature processing module, used to generate multiple local area masks based on the initial local area reference feature map, and the multiple local area masks are respectively combined with the initial distorted depth feature map and the initial reference depth feature map to generate multiple local area distorted depth features and multiple local area reference depth features;

[0103] A feature enhancement module is used to generate an enhanced local area reference depth feature by fusing multiple local area reference depth features based on a self-attention mechanism; based on a cross-attention mechanism, the enhanced local area reference depth feature is used as a guiding feature to generate an enhanced local area distortion depth feature according to multiple local area distortion depth features;

[0104] A weighted fusion module is used to generate a weight matrix according to the enhanced local area reference depth feature, and use the weight matrix to weightedly aggregate the enhanced local area reference depth feature and the enhanced local area distortion depth feature to generate a weighted fused local area reference depth feature and a weighted fused local area distortion depth feature;

[0105] The quality evaluation module is used to splice the weighted fused local area reference depth features, the weighted fused local area distortion depth features, the initial local area distortion feature map and the initial local area reference feature map, and calculate the quality score.

[0106] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0107] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A full-reference image quality assessment method based on local area mask, characterized in that: The specific steps are: Obtaining a distorted image and a reference image; Extracting local area information from the distorted image and the reference image respectively to obtain an initial local area distortion feature map and an initial local area reference feature map; Performing multi-stage feature extraction on the distorted image and the reference image respectively and generating an initial distorted depth feature map and an initial reference depth feature map; Generate a plurality of local area masks based on the initial local area reference feature map, and generate a plurality of local area distorted depth features and a plurality of local area reference depth features respectively with the initial distorted depth feature map and the initial reference depth feature map; Based on the self-attention mechanism, multiple local area reference depth features are fused to generate an enhanced local area reference depth feature; based on the cross-attention mechanism, the enhanced local area reference depth feature is used as a guide feature to generate an enhanced local area distortion depth feature according to multiple local area distortion depth features; A weight matrix is ​​generated according to the enhanced local area reference depth feature, and the enhanced local area reference depth feature and the enhanced local area distortion depth feature are weightedly aggregated by using the weight matrix to generate a weighted fused local area reference depth feature and a weighted fused local area distortion depth feature; The weighted fused local area reference depth feature, the weighted fused local area distortion depth feature, the initial local area distortion feature map and the initial local area reference feature map are spliced, and a quality score is calculated.

2. The method for evaluating the quality of a full-reference image based on a local area mask according to claim 1, characterized in that: After acquiring the distorted image and the reference image, the distorted image and the reference image need to be preprocessed.

3. The method for evaluating the quality of a full-reference image based on a local area mask according to claim 1, characterized in that: The steps of obtaining the initial local area distortion feature map or the initial local area reference feature map are: The distorted image or the reference image is input into a local area sampling network, and the initial local area distortion feature map or the initial local area reference feature map is generated through multiple iterations.

4. The method for evaluating the quality of a full-reference image based on a local area mask according to claim 1, characterized in that: The steps of acquiring the initial distorted depth feature map or the initial reference depth feature map are as follows: Extracting depth features of the distorted image or the reference image according to a depth extraction model, and selecting the depth features of a plurality of pre-set stages as a pre-depth feature map; The pre-depth feature map is up-sampled to a size of the distorted image or the reference image to generate the initial distorted depth feature map or the initial reference depth feature map.

5. The method for evaluating the quality of a full-reference image based on a local area mask according to claim 1, characterized in that: The steps of acquiring the local area distortion depth feature or the local area reference depth feature are: Generating the local area masks according to the initial local area reference feature map, the number of the local area masks being the same as the number of local areas; The initial distorted depth feature map or the initial reference depth feature map is multiplied by each of the local area masks to obtain a selected local area distortion feature or a selected local area reference feature; The selected local area distortion feature or the selected local area reference feature is globally averaged pooled to obtain the local area distortion depth feature or the local area reference depth feature.

6. The method for evaluating the quality of a full-reference image based on a local area mask according to claim 4, characterized in that: The steps of acquiring the weighted fusion local area reference depth feature or the weighted fusion local area distortion depth feature are: Generate the weight matrix using a multi-layer perceptron and a Softmax function according to the enhanced local area reference depth feature; The enhanced local area reference depth features or the enhanced local area distorted depth features at different stages are weighted averaged according to the weight matrix to generate the weighted fused local area reference depth features or the weighted fused local area distorted depth features.

7. The method for evaluating the quality of a full-reference image based on a local area mask according to claim 1, characterized in that: The quality score acquisition steps are: Splicing the initial local area distortion feature map and the weighted fusion local area distortion depth feature or the initial local area reference feature map and the weighted fusion local area reference depth feature in the channel dimension to obtain a distorted image splicing feature or a reference image splicing feature; Splicing the distorted image stitching features and the reference image stitching features in the channel dimension to obtain an overall stitching image feature; A quality score is calculated based on the overall stitched image features.

8. A full-reference image quality assessment system based on local area mask, characterized in that: include: An image acquisition module, used for acquiring a distorted image and a reference image; A local area division module, used to extract local area information from the distorted image and the reference image respectively, to obtain an initial local area distortion feature map and an initial local area reference feature map; A depth feature extraction module, used to perform multi-stage feature extraction on the distorted image and the reference image respectively and generate an initial distorted depth feature map and an initial reference depth feature map; A depth feature processing module, configured to generate a plurality of local area masks based on the initial local area reference feature map, wherein the plurality of local area masks are respectively combined with the initial distorted depth feature map and the initial reference depth feature map to generate a plurality of local area distorted depth features and a plurality of local area reference depth features; A feature enhancement module is used to generate enhanced local area reference depth features by fusing multiple local area reference depth features based on a self-attention mechanism; Based on the cross attention mechanism, the enhanced local area reference depth feature is used as a guide feature, and an enhanced local area distorted depth feature is generated according to a plurality of local area distorted depth features; A weighted fusion module, used to generate a weight matrix according to the enhanced local area reference depth feature, and use the weight matrix to weightedly aggregate the enhanced local area reference depth feature and the enhanced local area distortion depth feature to generate a weighted fused local area reference depth feature and a weighted fused local area distortion depth feature; The quality evaluation module is used to splice the weighted fused local area reference depth feature, the weighted fused local area distortion depth feature, the initial local area distortion feature map and the initial local area reference feature map, and calculate the quality score.

Images