Reference-free image quality evaluation method based on inverted pyramid structure multi-scale attention mechanism

Abstract

The invention discloses a no-reference image quality evaluation method based on an inverted pyramid structure multi-scale attention mechanism, which is suitable for the field of image processing and comprises the following steps of: firstly, preprocessing a distorted image data set; inputting the processed data into a feature extraction network, and extracting image features of four different scales by using a decomposition large kernel convolution operation; and adjusting the extracted four scale features to the same space size through adaptive average pooling, and adding a learnable position code for each space position. Then, gradually fusing high-level semantic information into low-level features by using a cross-scale attention mechanism (CSA block); the distortion features with semantic perception are aggregated through a multi-layer perceptron (MLP), and finally feature representation output by the MLP is mapped into a quality score by using a full connection layer. According to the method, large kernel convolution is introduced, an inverted pyramid structure is used in the multi-scale feature network, the extracted features better conform to human visual perception features, and image quality evaluation is more accurate.

Description

Technical Field

[0001] The invention relates to a reference-free image quality evaluation method based on an inverted pyramid structure multi-scale attention mechanism, and belongs to the field of image processing. Background Art

[0002] At present, image quality evaluation can be divided into two categories from the perspective of the evaluation subject: subjective evaluation and objective evaluation. Objective quality evaluation models can be divided into full reference (FR), reduced reference (RR) and no reference (NR) image quality evaluation models according to the degree of dependence on the original image information. The no reference image quality evaluation method means that in the evaluation model, there is no need to refer to the original image, only the information of the distorted image itself is used to directly output the image quality score. It is not difficult to see from the description that this method is applicable and practical, so it has become a research hotspot supported by scholars in recent years.

[0003] The research on no-reference image quality assessment can be traced back to the early days of image processing and computer vision. Initial research focused on developing some simple statistical features and heuristic algorithms to estimate the degree of image distortion. For example, the image’s brightness histogram, edge information, or texture features are analyzed to determine the type of distortion, such as blur or noise. These early methods usually have low computational complexity, but their performance is often limited when faced with complex distortion types and diverse image content.

[0004] With the vigorous development of machine learning and deep learning technologies, especially the introduction of convolutional neural networks, NR-IQA research has entered a new stage. Through large-scale data training, deep learning methods can automatically extract high-level features of images and can more effectively capture complex distortion patterns. For example, researchers have developed many NR-IQA models based on deep learning, which are trained with a large amount of labeled data and significantly improve the accuracy and robustness of image quality evaluation. Summary of the invention

[0005] The purpose of the present invention is to disclose a reference-free image quality evaluation method based on a multi-scale attention mechanism of an inverted pyramid structure, which uses the multi-scale attention mechanism to evaluate the quality of an image and accurately predict the objective image quality value of a distorted image.

[0006] In an embodiment of the present disclosure, the method for no-reference image quality assessment based on the inverted pyramid structure multi-scale attention mechanism comprises the following steps:

[0007] Step 1: First, preprocess the distorted image dataset;

[0008] Step 2: Input the processed data into the feature extraction network and use the decomposed large kernel convolution operation to extract image features of four different scales;

[0009] Step 3: The four scale features extracted in step 2 are adjusted to the same spatial size through adaptive average pooling, and a learnable position encoding is added to each spatial position. Subsequently, the cross-scale attention mechanism (CSA block) is used to gradually fuse high-level semantic information into low-level features:

[0010] Step 4: Aggregate these semantically aware distortion features through a multi-layer perceptron (MLP), and finally use a fully connected layer to map the feature representation of the MLP output into a quality score.

[0011] A further technical solution is to pair the images in the distorted image data set with their corresponding labels in step 1, and then divide the paired data into a training set and a test set in a certain ratio, perform operations such as cropping and random flipping on the images in the training set for training set data enhancement, and finally normalize the images.

[0012] In a further technical solution, step 2 specifically comprises the following steps:

[0013] Step 2.1: First, the input image is split into multiple overlapping small blocks through the convolution layer, and these small blocks are embedded into the high-dimensional feature space. Then, the convolved tensor is batch normalized to ensure that the features of each small block are more stable during training. Finally, the normalized feature map F1 and the corresponding height and width are returned for subsequent processing.

[0014] Step 2.2: First, batch normalize the input feature F1. This operation helps reduce the distribution deviation of the feature value and makes the subsequent convolution operation more stable. Then use 1x1 convolution to process the normalized features, and obtain feature F2 through the GELU activation function. The 1x1 convolution is mainly used to adjust the number of channels and fuse the information between channels, while the GELU activation function provides a smooth nonlinear transformation, which helps to enhance the expressiveness of the features.

[0015] Step 2.3: First, perform a 5x5 deep convolution DW_Conv operation on feature F2 to extract preliminary spatial information. Deep convolution only operates on each channel separately and does not change the relationship between channels, thereby maintaining the channel structure of the input. Then apply a 7x7 deep convolution DW_D_Conv with a dilation rate to capture spatial dependencies in a larger range by expanding the receptive field. Dilated convolution can expand the receptive field without increasing the number of parameters, allowing the model to obtain a wider range of spatial features. Finally, use 1x1 convolution to fuse the spatial information extracted in the first two steps to generate the final attention map. 1x1 convolution is used here to synthesize information from different channels and combine the captured spatial features with channel information.

[0016] Step 2.4: Perform element-by-element multiplication (element convolution) on the generated attention map and the original input features. This process fine-tunes the original features through the attention mechanism so that important features are strengthened. The adjusted features are projected to obtain the output feature F3.

[0017] Step 2.5: F3 is batch normalized again to ensure that the features input to the MLP remain within an appropriate numerical range. The normalized features are input to the MLP, and after multiple layers of linear transformation and nonlinear activation, the output features F4 are obtained. MLP further improves the feature representation capability through complex nonlinear transformations, allowing the network to capture richer patterns and information in the input data.

[0018] This step is performed by sequentially stacking L groups of operations to extract features. The operation of each group is described as follows:

[0019]

[0020] Among them, f GELU indicates GELU activation, f BN represents batch normalization, f 1×1 represents 1×1 channel convolution, f DW_Conv (·) and f DW_D_Conv (·) represent depthwise convolution and depthwise dilated convolution respectively. f FFN represents a convolutional feed-forward network, Represents element-wise convolution.

[0021] In a further technical solution, step 3 specifically comprises the following steps:

[0022] Step 3.1: Process the features F4 obtained in step 2 in order from high to low. First, use adaptive average pooling to adjust the feature maps to the same spatial size. In order to enable the model to better capture spatial position information, we add a learnable position encoding to each spatial position, and adjust its size by bicubic interpolation to adjust the position encoding to a size that matches the current feature map. This unified position encoding enables subsequent CSA blocks to better match features from different scales. Then perform dimensionality reduction on the feature map, flatten and transpose the feature map so that it can be added to the adjusted position encoding.

[0023] Step 3.2: In order to better fuse feature maps of different scales, the cross-scale attention mechanism (CSA) is used. The query (Q), key (K), and value (V) generated by features of different scales are used as inputs, and high-level semantic information is recursively aggregated into low-level features layer by layer. For the input feature map, first start from the highest level feature map G n At the beginning, it is processed as the initial query, and the query is recursively calculated with the lower-level feature map across scales to generate a new query:

[0024] Q=G i+1 W q , K=G i W k , V=G i W v ,

[0025] The attention calculation at each step is based on the softmax function to calculate the attention weight A:

[0026]

[0027] These weights A are then used to perform a weighted sum on the feature maps and the result is superimposed on the original feature maps:

[0028]

[0029] in Nq, Nv represent the number of feature vectors, dk, dv are the dimensions of the feature vectors, and are used to scale the attention scores. i∈{1,...,n-1}.

[0030] G i represents the feature map of the i-th layer, Q, K, V represent the feature vectors of query, key and value respectively, which are usually generated from the feature map through linear transformation. A is the attention matrix, representing G i and G i+1The recursive cross-scale processing of feature maps is carried out from high-level to low-level, gradually aggregating high-level semantic information into low-level features.

[0031] In a further technical solution, step 4 specifically comprises the following steps:

[0032] Step 4.1: After the cross-scale attention processing is completed, the processed query feature map is finally globally integrated through the attention pooling layer to obtain the final global feature representation:

[0033]

[0034] Among them, H n ×W n represents the size of the feature map after dimensionality reduction, and D is the feature dimension after dimensionality reduction. Specifically, the input features are first normalized, the normalized features are passed through the first linear layer, then through the activation function, and then through the second linear layer, and finally the output is added to the original input to form a residual structure. This process ensures the gradual fusion from high-level to low-level layers, which can effectively aggregate information of different scales and finally generate feature maps rich in multi-scale semantics.

[0035] Step 4.2: Use the data obtained in step 4.1 These semantically aware distortion features are aggregated through a multi-layer perceptron (MLP), and finally a fully connected layer is used to map the feature representation of the MLP output into the final score p^, as follows:

[0036]

[0037] Here, the role of MLP is to process the final feature representation through multiple fully connected layers, further refine the feature representation, and then convert the output feature representation into a probability distribution through the softmax function to obtain the final score p^.

[0038] p^=softmax(y^)

[0039] The present invention provides a no-reference image quality assessment method based on an inverted pyramid structure multi-scale attention mechanism, which has the following beneficial effects:

[0040] 1) The present invention can more accurately evaluate image quality through the inverted pyramid structure multi-scale attention mechanism. It first preprocesses and data enhances the distorted image to ensure the diversity and representativeness of the training set. Then, by decomposing the large kernel convolution operation, image features of different scales are extracted, which helps to capture the distortion information of the image at different levels. Adaptive average pooling and learnable position encoding further enhance the model's perception of image details. The introduction of the cross-scale attention mechanism (CSA block) enables the model to gradually fuse high-level semantic information into low-level features, thereby comprehensively considering the global and local features of the image in the evaluation process, significantly improving the accuracy of the evaluation.

[0041] 2) The present invention uses multi-level feature extraction and fusion to effectively capture complex features in images. Decomposing large kernel convolution and deep convolution operations enables the model to identify subtle changes in images, and the use of MLP further enhances the nonlinear transformation capabilities of these features. This multi-level and multi-scale processing method enables the model to accurately evaluate the quality of images with complex textures and structures.

[0042] 3) The present invention increases the adaptability of the model to different image transformations by performing data augmentation in the training set, such as image cropping and random flipping. This augmentation operation helps the model learn more robust feature representations, so that it can still give accurate quality evaluations when faced with new, unseen images. In addition, by introducing adaptive average pooling and position encoding, the model can better generalize to different image sizes and contents.

[0043] 4) The present invention takes computational efficiency into consideration when designing. Through operations such as element convolution and batch normalization, the computational burden of the model is reduced while maintaining the depth and breadth of feature extraction. This design enables the model to have a low computational cost while maintaining high accuracy, making it suitable for use in resource-constrained environments, such as mobile devices or embedded systems.

[0044] 5) The present invention can perform quality evaluation without reference images, which is particularly important for scenarios where high-quality reference images are often lacking in practical applications. Through adaptive average pooling and cross-scale attention mechanisms, the model can extract and analyze features directly from the distorted image itself, independent of the reference image, to achieve reference-free quality evaluation. This greatly expands the application scope of image quality evaluation, making it more flexible and practical. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of a no-reference image quality assessment method based on an inverted pyramid structure multi-scale attention mechanism according to the present invention;

[0046] Figure 2This is a schematic diagram of the network structure of a reference-free image quality evaluation method based on an inverted pyramid structure multi-scale attention mechanism involved in the present invention. DETAILED DESCRIPTION

[0047] The present invention proposes a reference-free image quality assessment method, which uses a multi-scale attention mechanism with an inverted pyramid structure to realize automatic image quality assessment. In order to more clearly explain the purpose, technical solution and advantages of the present invention, the technical solution of the present invention will be described in detail below with reference to the accompanying drawings. It should be pointed out that the specific embodiments described below are only used to explain the principles of the present invention and are not used to limit the scope of implementation of the present invention.

[0048] The present invention first provides a reference-free image quality evaluation method based on a multi-scale attention mechanism of an inverted pyramid structure. Figure 1 , including the following steps:

[0049] Step 1: First, preprocess the distorted image dataset;

[0050] Step 2: Input the processed data into the feature extraction network and use the decomposed large kernel convolution operation to extract image features of four different scales;

[0051] Step 3: The four scale features extracted in step 2 are adjusted to the same spatial size through adaptive average pooling, and a learnable position code is added to each spatial position. Subsequently, the cross-scale attention mechanism (CSA block) is used to gradually fuse high-level semantic information into low-level features;

[0052] Step 4: Aggregate these semantically aware distortion features through a multi-layer perceptron (MLP), and finally use a fully connected layer to map the feature representation of the MLP output into a quality score.

[0053] As a preferred embodiment of the present invention, in step 1, the images in the distorted image data set are paired with their corresponding labels, and then the paired data are divided into a training set and a test set in a certain ratio, and the images in the training set are cropped, randomly flipped, and other operations are performed for training set data enhancement, and finally the images are normalized.

[0054] As a preferred embodiment of the present invention, in step 2, the processed data is decomposed into multiple overlapping small blocks through multiple convolutional layers, and these small blocks are embedded in a high-dimensional feature space. These small blocks are processed by large-kernel convolution operations to extract feature information of different scales. Ultimately, these feature maps of different scales lay the foundation for subsequent multi-scale fusion and attention mechanisms, allowing the network to learn and fuse information from features of multiple scales to improve the accuracy and robustness of image quality assessment.

[0055] Wherein, step 2 specifically includes the following steps:

[0056] Step 2.1: First, the input image is split into multiple overlapping small blocks through the convolution layer, and these small blocks are embedded into the high-dimensional feature space. Then, the convolved tensor is batch normalized to ensure that the features of each small block are more stable during training. Finally, the normalized feature map F1 and the corresponding height and width are returned for subsequent processing.

[0057] Step 2.2: First, batch normalize the input feature F1. This operation helps reduce the distribution deviation of the feature value and makes the subsequent convolution operation more stable. Then use lx1 convolution to process the normalized features, and obtain feature F2 through GELU activation function. 1x1 convolution is mainly used to adjust the number of channels and fuse the information between channels, while GELU activation function provides a smooth nonlinear transformation, which helps to enhance the expressiveness of features.

[0058] Step 2.3: First, perform a 5x5 deep convolution DW_Conv operation on feature F2 to extract preliminary spatial information. Deep convolution only operates on each channel separately and does not change the relationship between channels, thereby maintaining the channel structure of the input. Then apply a 7x7 deep convolution DW_D_Conv with a dilation rate to capture spatial dependencies in a larger range by expanding the receptive field. Dilated convolution can expand the receptive field without increasing the number of parameters, allowing the model to obtain a wider range of spatial features. Finally, use 1x1 convolution to fuse the spatial information extracted in the first two steps to generate the final attention map. 1x1 convolution is used here to synthesize information from different channels and combine the captured spatial features with channel information.

[0059] Step 2.4: Perform element-by-element multiplication (element convolution) on the generated attention map and the original input features. This process fine-tunes the original features through the attention mechanism so that important features are strengthened. The adjusted features are projected to obtain the output feature F3.

[0060] Step 2.5: F3 is batch normalized again to ensure that the features input to the MLP remain within an appropriate numerical range. The normalized features are input to the MLP, and after multiple layers of linear transformation and nonlinear activation, the output features F4 are obtained. MLP further improves the feature representation capability through complex nonlinear transformations, allowing the network to capture richer patterns and information in the input data.

[0061] This step is performed by sequentially stacking L groups of operations to extract features. The operation of each group is described as follows:

[0062]

[0063] Among them, f GELU indicates GELU activation, f BN represents batch normalization, f 1×1 represents 1×1 channel convolution, f DW_Conv (·) and f DW_D_Conv (·) represent depthwise convolution and depthwise dilated convolution respectively. f FFN represents a convolutional feed-forward network, Represents element-wise convolution.

[0064] As a preferred embodiment of the present invention, in step 3, the feature map is first processed by adaptive average pooling, adjusted to a uniform spatial size, and a learnable position encoding is added to each spatial position, which helps to align features of different scales. Subsequently, a cross-scale attention mechanism (CSA block) is used to fuse high-level semantic information into low-level features. Specifically, the feature map is converted into query, key, and value vectors, and the attention weights are calculated. Then, the feature map is weighted and summed, and the high-level semantic information is gradually integrated into the low-level features layer by layer to enhance the comprehensive ability of multi-scale features. This processing method can effectively combine high-level global information with low-level detail information, thereby enhancing the accuracy of image quality assessment.

[0065] Wherein, step 3 specifically includes the following steps:

[0066] Step 3.1: Process the features F4 obtained in step 2 in order from high to low. First, use adaptive average pooling to adjust the feature maps to the same spatial size. In order to enable the model to better capture spatial position information, we add a learnable position encoding to each spatial position, and adjust its size by bicubic interpolation to adjust the position encoding to a size that matches the current feature map. This unified position encoding enables subsequent CSA blocks to better match features from different scales. Then perform dimensionality reduction on the feature map, flatten and transpose the feature map so that it can be added to the adjusted position encoding.

[0067] Step 3.2: In order to better fuse feature maps of different scales, the cross-scale attention mechanism (CSA) is used. The query (Q), key (K), and value (V) generated by features of different scales are used as inputs, and high-level semantic information is recursively aggregated into low-level features layer by layer. For the input feature map, first start from the highest level feature map G n At the beginning, it is processed as the initial query, and the query is recursively calculated with the lower-level feature map across scales to generate a new query:

[0068] Q=G i+1 Wq , K=G i W k , V=G i W v ,

[0069] The attention calculation at each step is based on the softmax function to calculate the attention weight A:

[0070]

[0071] These weights A are then used to perform a weighted sum on the feature maps and the result is superimposed on the original feature maps:

[0072]

[0073] in Nq, Nv represent the number of feature vectors, dk, dv are the dimensions of the feature vectors, and are used to scale the attention scores. i∈{1,...,n-1}.

[0074] G i represents the feature map of the i-th layer, Q, K, V represent the feature vectors of query, key and value respectively, which are usually generated from the feature map through linear transformation. A is the attention matrix, representing G i and G i+1 The recursive cross-scale processing of feature maps is carried out from high-level to low-level, gradually aggregating high-level semantic information into low-level features.

[0075] As a preferred embodiment of the present invention, in step 4, the features processed by cross-scale attention are first globally integrated through the attention pooling layer to generate the final global feature representation. This process ensures the effective fusion of high-level and low-level features. Next, the aggregated features are further processed using a multi-layer perceptron (MLP), the features are refined through multiple fully connected layers, and finally the processed features are mapped to a quality score using a softmax function to provide accurate image quality assessment.

[0076] Wherein, step 4 specifically includes the following steps:

[0077] Step 4.1: After the cross-scale attention processing is completed, the processed query feature map is finally globally integrated through the attention pooling layer to obtain the final global feature representation:

[0078]

[0079] Among them, H n ×W nrepresents the size of the feature map after dimensionality reduction, and D is the feature dimension after dimensionality reduction. Specifically, the input features are first normalized, the normalized features are passed through the first linear layer, then through the activation function, and then through the second linear layer, and finally the output is added to the original input to form a residual structure. This process ensures the gradual fusion from high-level to low-level layers, which can effectively aggregate information of different scales and finally generate feature maps rich in multi-scale semantics.

[0080] Step 4.2: Use the data obtained in step 4.1 These semantically aware distortion features are aggregated through a multi-layer perceptron (MLP), and finally a fully connected layer is used to map the feature representation of the MLP output into the final score p^, as follows:

[0081]

[0082] Here, the role of MLP is to process the final feature representation through multiple fully connected layers, further refine the feature representation, and then convert the output feature representation into a probability distribution through the softmax function to obtain the final score p^.

[0083] p^=softmax(y^)

[0084] In the above description, various combinations of technical features are mentioned, but not all possible combinations are listed in detail. However, it should be emphasized that as long as the combination of these technical features is reasonable in implementation and does not conflict with each other, it should be considered to be within the scope of this specification.

Claims

1. A no-reference image quality assessment method based on an inverted pyramid structure multi-scale attention mechanism, characterized in that: The method comprises the following steps: Step 1: First, preprocess the distorted image dataset; Step 2: Input the processed data into the feature extraction network and use the decomposed large kernel convolution operation to extract image features of four different scales; Step 3: The four scale features extracted in step 2 are adjusted to the same spatial size through adaptive average pooling, and a learnable position code is added to each spatial position. Subsequently, the cross-scale attention mechanism (CSA block) is used to gradually fuse high-level semantic information into low-level features; Step 4: Aggregate these semantically aware distortion features through a multi-layer perceptron (MLP), and finally use a fully connected layer to map the feature representation of the MLP output into a quality score.

2. The no-reference image quality assessment method of the inverted pyramid structure multi-scale attention mechanism according to claim 1, characterized in that: In the step 1, the images in the distorted image data set are paired with their corresponding labels, and then the paired data are divided into a training set and a test set in a certain ratio. The images in the training set are cropped, randomly flipped, and other operations are performed for training set data enhancement, and finally the images are normalized.

3. The no-reference image quality assessment method of the inverted pyramid structure multi-scale attention mechanism according to claim 1, characterized in that: In step 2, the processed data is input into the feature extraction network, and the image features of four different scales are extracted by decomposing the large kernel convolution operation, as follows: Step 2.1: First, the input image is split into multiple overlapping small blocks through the convolution layer, and these small blocks are embedded into the high-dimensional feature space. Then, the convolved tensor is batch normalized to ensure that the features of each small block are more stable during training. Finally, the normalized feature map F1 and the corresponding height and width are returned for subsequent processing. Step 2.2: First, batch normalize the input feature F1. This operation helps reduce the distribution deviation of the feature value and makes the subsequent convolution operation more stable. Then use 1x1 convolution to process the normalized features, and obtain feature F2 through the GELU activation function. The 1x1 convolution is mainly used to adjust the number of channels and fuse the information between channels, while the GELU activation function provides a smooth nonlinear transformation, which helps to enhance the expressiveness of the features. Step 2.3: First, perform a 5x5 deep convolution DW_Conv operation on feature F2 to extract preliminary spatial information. Deep convolution only operates on each channel separately and does not change the relationship between channels, thereby maintaining the channel structure of the input. Then apply a 7x7 deep convolution DW_D_Conv with a dilation rate to capture spatial dependencies in a larger range by expanding the receptive field. Dilated convolution can expand the receptive field without increasing the number of parameters, allowing the model to obtain a wider range of spatial features. Finally, use 1x1 convolution to fuse the spatial information extracted in the first two steps to generate the final attention map. 1x1 convolution is used here to synthesize information from different channels and combine the captured spatial features with channel information. Step 2.4: Perform element-by-element multiplication (element convolution) on the generated attention map and the original input features. This process fine-tunes the original features through the attention mechanism so that important features are strengthened. The adjusted features are projected to obtain the output feature F3. Step 2.5: F3 is batch normalized again to ensure that the features input to the MLP remain within an appropriate numerical range. The normalized features are input to the MLP, and after multiple layers of linear transformation and nonlinear activation, the output features F4 are obtained. MLP further improves the feature representation capability through complex nonlinear transformations, allowing the network to capture richer patterns and information in the input data. This step is performed by sequentially stacking L groups of operations to extract features. The operation of each group is described as follows: Among them, f GELU indicates GELU activation, f BN represents batch normalization, f 1×1 represents 1×1 channel convolution, f DW_Conv (·) and f DW_D_Conv (·) represent depthwise convolution and depthwise dilated convolution respectively. f FFN represents a convolutional feed-forward network, Represents element-wise convolution.

4. The method for no-reference image quality assessment based on the inverted pyramid structure multi-scale attention mechanism according to claim 1, characterized in that: The step 3 specifically comprises the following steps: Step 3.1: Process the feature F4 obtained in step 2 in order from high to low. First, use adaptive average pooling to adjust the feature map to the same spatial size. In order to enable the model to better capture spatial position information, we add a learnable position encoding to each spatial position, and adjust its size by bicubic interpolation to adjust the position encoding to a size that matches the current feature map. This unified position encoding enables subsequent CSA blocks to better match features from different scales. Then perform dimensionality reduction on the feature map, flatten and transpose the feature map so that it can be added to the adjusted position encoding. Step 3.2: In order to better fuse feature maps of different scales, the cross-scale attention mechanism (CSA) is used. The query (Q), key (K), and value (V) generated by features of different scales are used as inputs, and high-level semantic information is recursively aggregated into low-level features layer by layer. For the input feature map, first start from the highest level feature map G n At the beginning, it is processed as the initial query, and the query is recursively calculated with the lower-level feature map across scales to generate a new query: Q=G i+1 W q ,K=G i W k ,V=G i W v , The attention calculation at each step is based on the softmax function to calculate the attention weight A: These weights A are then used to perform a weighted sum on the feature maps and the result is superimposed on the original feature maps: in Nq, Nv represent the number of feature vectors, dk, dv are the dimensions of the feature vectors, and are used to scale the attention scores. i∈{1,...,n-1}. G i represents the feature map of the i-th layer, Q, K, V represent the feature vectors of query, key and value respectively, which are usually generated from the feature map through linear transformation. A is the attention matrix, representing G i and G i+1 The recursive cross-scale processing of feature maps is carried out from high-level to low-level, gradually aggregating high-level semantic information into low-level features.

5. The method for no-reference image quality assessment based on the inverted pyramid structure multi-scale attention mechanism according to claim 1, characterized in that: The step 4 specifically comprises the following steps: Step 4.1: After the cross-scale attention processing is completed, the processed query feature map is finally globally integrated through the attention pooling layer to obtain the final global feature representation: Among them, H n ×W n represents the size of the feature map after dimensionality reduction, and D is the feature dimension after dimensionality reduction. Specifically, the input features are first normalized, the normalized features are passed through the first linear layer, then through the activation function, and then through the second linear layer, and finally the output is added to the original input to form a residual structure. This process ensures the gradual fusion from high-level to low-level layers, which can effectively aggregate information of different scales and finally generate feature maps rich in multi-scale semantics. Step 4.2: Use the data obtained in step 4.1 These semantically aware distortion features are aggregated through a multi-layer perceptron (MLP), and finally a fully connected layer is used to map the feature representation of the MLP output into the final score p^, as follows: Here, the role of MLP is to process the final feature representation through multiple fully connected layers, further refine the feature representation, and then convert the output feature representation into a probability distribution through the softmax function to obtain the final score p^. p^=softmax(y^).

Images