No-reference image quality assessment method based on inverted pyramid structure multi-scale attention mechanism

Abstract

The application discloses a kind of based on inverted pyramid structure multi-scale attention mechanism's no reference image quality evaluation method, it is applicable to image processing field, including the following steps: first, the distorted image dataset is preprocessed;The data input to feature extraction network after processing is completed, four different scale image features are extracted using decomposition large kernel convolution operation;Four scale features extracted are adjusted to the same spatial size by adaptive average pooling, and learnable position coding is added for each spatial position.Subsequently, cross-scale attention mechanism (CSA block) is used to gradually fuse high-level semantic information into low-level features;These distorted features with semantic perception are aggregated by multilayer perceptron MLP, and finally the feature representation output by MLP is mapped into a quality score using a fully connected layer.The method introduces large kernel convolution, uses inverted pyramid structure in multi-scale feature network, and the extracted features are more consistent with human visual perception characteristics, making the image quality evaluation more accurate.

Description

TECHNICAL FIELD

[0001] The present application relates to a no-reference image quality evaluation method based on a multi-scale attention mechanism of an inverted pyramid structure, and belongs to the field of image processing. BACKGROUND

[0002] At present, image quality evaluation can be divided into subjective evaluation and objective evaluation from the perspective of evaluation subjects. According to the degree of dependence on original image information, objective quality evaluation models can be divided into full reference (FR), reduced reference (RR) and no reference (NR) image quality evaluation models. The no-reference image quality evaluation method refers to the method that does not need to refer to the original image in the evaluation model, and only uses the information of the distorted image itself to directly output the quality score of the image. As can be seen from the description, this method has applicability and practicality, and has become a research hotspot supported by scholars in recent years.

[0003] The research of no-reference image quality evaluation can be traced back to the early field of image processing and computer vision. The initial research mainly focused on developing some simple statistical features and heuristic algorithms to estimate the distortion degree of images. For example, by analyzing the brightness histogram, edge information or texture features of the image to judge the blur, noise and other distortion types of the image. These early methods usually have low computational complexity, but when faced with complex distortion types and diversified image contents, their performance is often limited.

[0004] With the vigorous development of machine learning and deep learning technologies, especially the introduction of convolutional neural networks, the research of NR-IQA has entered a new stage. Deep learning methods can automatically extract high-level features of images through large-scale data training, and can more effectively capture complex distortion patterns. For example, researchers have developed many deep learning-based NR-IQA models, which use a large amount of labeled data for training, significantly improving the accuracy and robustness of image quality evaluation. SUMMARY

[0005] The purpose of the present application is to disclose a no-reference image quality evaluation method based on a multi-scale attention mechanism of an inverted pyramid structure, which uses a multi-scale attention mechanism to evaluate the quality of an image and accurately predicts the image objective quality value of a distorted image.

[0006] In embodiments according to the present disclosure, the no-reference image quality evaluation method based on a multi-scale attention mechanism of an inverted pyramid structure includes the following steps:

[0007] Step 1: First, pre-process the distorted image dataset;

[0008] Step 2: The processed data is input into a feature extraction network, and four different scale image features are extracted using a decomposition large kernel convolution operation;

[0009] Step 3: The four scale features extracted in Step 2 are adjusted to the same spatial size through adaptive average pooling, and a learnable position encoding is added for each spatial position. Then, a cross-scale attention mechanism (CSA block) is used to gradually fuse high-level semantic information into low-level features:

[0010] Step 4: These distortion features with semantic perception are aggregated through a multi-layer perceptron (MLP), and finally the feature representation output by the MLP is mapped to a quality score using a fully connected layer.

[0011] In a further technical solution, in Step 1, the images in the distortion image dataset are paired with their corresponding labels, and then the paired data are divided into a training set and a test set according to a certain proportion. The images in the training set are cropped, randomly flipped, and the like for data augmentation, and finally the images are normalized.

[0012] In a further technical solution, Step 2 specifically includes the following steps:

[0013] Step 2.1: First, the input image is divided into multiple overlapping small blocks through a convolutional layer, and these small blocks are embedded into a high-dimensional feature space. Then, batch normalization is performed on the convolutional tensor to ensure that the features of each small block are more stable during training. Finally, the normalized feature map F1 and the corresponding height and width are returned for subsequent processing.

[0014] Step 2.2: First, the input feature F1 is batch normalized, which helps to reduce the distribution deviation of feature values and makes the subsequent convolution operation more stable. Then, a 1x1 convolution is used to process the normalized feature, and a GELU activation function is used to obtain the feature F2. The 1x1 convolution is mainly used to adjust the channel number and fuse the information between channels, while the GELU activation function provides a smooth nonlinear transformation, which helps to enhance the expression ability of the feature.

[0015] Step 2.3: First, a 5x5 deep convolution DW_Conv operation is performed on feature F2 to extract preliminary spatial information. Deep convolution only operates on each channel individually and does not change the relationship between channels, thus preserving the input channel structure. Then a 7x7 deep convolution DW_D_Conv with dilation rate is applied to capture spatial dependencies over a larger range by expanding the receptive field. Dilation convolution can expand the receptive field without increasing the number of parameters, enabling the model to capture more extensive spatial features. Finally, a 1x1 convolution is used to fuse the spatial information extracted in the previous two steps to generate the final attention map. The 1x1 convolution here is used to integrate information from different channels, combining the captured spatial features with channel information.

[0016] Step 2.4: Element-wise multiplication (element convolution) is performed between the generated attention map and the original input feature. This process fine-tunes the original feature through the attention mechanism, strengthening important features. The adjusted feature is projected to obtain the output feature F3.

[0017] Step 2.5: Batch normalization is performed again on F3 to ensure that the features input into the MLP remain within an appropriate numerical range. The normalized feature is input into the MLP, and after multiple linear transformations and nonlinear activations, the output feature F4 is obtained. The MLP further enhances the feature representation capability through complex nonlinear transformations, enabling the network to capture more rich patterns and information in the input data.

[0018] This step is sequentially stacked with L groups of operations to extract features. The operations of each group are described as follows:

[0019]

[0020] where f GELU represents GELU activation, f BN represents batch normalization, f 1×1 represents 1x1 channel convolution, f DW_Conv (·) and f DW_D_Conv (·) represent deep convolution and deep dilation convolution respectively. f FFN represents a convolution feedforward network, represents element-wise convolution.

[0021] Further technical solutions, step 3 specifically includes the following steps:

[0022] Step 3.1: The features F4 obtained in step 2 are processed in descending order. First, the feature maps are adjusted to the same spatial size using adaptive average pooling. To enable the model to better capture spatial position information, we add a learnable position encoding for each spatial position, which is adjusted to match the size of the current feature map by bicubic interpolation. This unified position encoding enables the subsequent CSA block to better match features from different scales. Then the feature map is reduced in dimension and flattened and transposed to be able to be added to the adjusted position encoding.

[0023] Step 3.2: To better fuse feature maps of different scales, a cross-scale attention mechanism (CSA) is adopted, which takes three groups of feature vectors generated by query (Q), key (K), and value (V) of different scale features as input, and recursively aggregates high-level semantic information into low-level features. For the input feature map, first, the highest level feature map G n is taken as the initial query for processing, and the query is recursively calculated with the cross-scale attention of the lower level feature map, generating a new query:

[0024] Q = G i+1 W q , K = G i W k , V = G i W v ,

[0025] The attention calculation at each step is based on the softmax function to calculate the attention weight A:

[0026]

[0027] Then use these weights A to weight the sum of the feature maps, and add the result to the original feature map:

[0028]

[0029] where Nq, Nv represent the number of feature vectors, dk, dv are the dimensions of the feature vectors, which are used to scale the attention score. i ∈ {1,..., n-1}.

[0030] G i represents the i-th layer feature map ， Q, K, V represent the query, key and value feature vectors, respectively, which are usually generated from the feature map by linear transformation. A is the attention matrix, which represents G i and G i+1The similarity between the two is that the feature map is processed recursively across scales, from high to low, gradually aggregating high-level semantic information into low-level features.

[0031] Further technical solutions, step 4 specifically includes the following steps:

[0032] Step 4.1: After the cross-scale attention processing is completed, the processed query feature map is finally integrated globally through an attention pooling layer to obtain the final global feature representation:

[0033]

[0034] Where H n ×W n represents the size of the feature map after dimension reduction processing, and D is the feature dimension after dimension reduction. Specifically, first, the input features are normalized, then the normalized features are passed through the first linear layer, then the activation function, then the second linear layer, and finally the output is added to the original input to form a residual structure. This process ensures that the gradual fusion from high to low can effectively aggregate information of different scales, and finally generate a feature map rich in multi-scale semantics.

[0035] Step 4.2: Use the These semantic-aware distortion features are aggregated through a multi-layer perceptron (MLP), and finally a fully connected layer is used to map the feature representation output by the MLP into the final score p^, as follows:

[0036]

[0037] Here, the role of MLP is to process the final feature representation through multiple fully connected layers to further refine the feature representation, and then convert the output feature representation into a probability distribution through a softmax function, thereby obtaining the final score p^.

[0038] p^=softmax(y^)

[0039] The application provides a no-reference image quality evaluation method based on the inverted pyramid structure multi-scale attention mechanism, which has the following beneficial effects:

[0040] 1) The present invention can more accurately evaluate image quality through the inverted pyramid structure multi-scale attention mechanism. It first preprocesses and data augments the distorted image to ensure the diversity and representativeness of the training set. Then, by decomposing the large kernel convolution operation, image features of different scales are extracted, which helps to capture the distortion information of the image at different levels. Adaptive average pooling and learnable position encoding further enhance the model's ability to perceive image details. The introduction of the cross-scale attention mechanism (CSA block) enables the model to gradually fuse high-level semantic information into low-level features, thereby considering both global and local features of the image during evaluation, significantly improving the accuracy of the evaluation.

[0041] 2) The present invention can effectively capture complex features in images through multi-level feature extraction and fusion. Decomposing large kernel convolution and deep convolution operations enable the model to identify subtle changes in images, while the use of MLP further enhances the ability to perform nonlinear transformations on these features. This multi-level, multi-scale processing approach enables the model to accurately evaluate the quality of images with complex textures and structures.

[0042] 3) The present invention increases the model's adaptability to different image transformations by performing data augmentation such as image cropping and random flipping in the training set. This augmentation operation helps the model learn more robust feature representations, enabling it to provide accurate quality evaluations even when faced with new, unseen images. In addition, through the introduction of adaptive average pooling and position encoding, the model can better generalize to different image sizes and contents.

[0043] 4) The present invention considers computational efficiency in its design, reducing the computational burden of the model through element convolution and batch normalization operations, while maintaining the depth and breadth of feature extraction. This design enables the model to maintain high precision while having lower computational cost, making it suitable for use in resource-constrained environments such as mobile devices or embedded systems.

[0044] 5) The present invention does not require reference images for quality evaluation, which is particularly important for scenarios where high-quality reference images are often lacking in practical applications. Through adaptive average pooling and cross-scale attention mechanisms, the model can independently extract and analyze features from the distorted image itself, achieving no-reference quality evaluation. This greatly expands the application range of image quality evaluation, making it more flexible and practical. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 The flowchart of the no-reference image quality evaluation method based on the inverted pyramid structure multi-scale attention mechanism of the present invention;

[0046] Figure 2A network structure schematic diagram of a no-reference image quality evaluation method based on a multi-scale attention mechanism of an inverted pyramid structure. DETAILED DESCRIPTION

[0047] The present application provides a no-reference image quality evaluation method, which uses a multi-scale attention mechanism of an inverted pyramid structure to realize automatic evaluation of image quality. In order to more clearly set forth the objects, technical solutions and advantages of the present application, the technical solutions of the present application will be described in detail below in conjunction with the drawings. It should be pointed out that the specific embodiments described below are only used to explain the principles of the present application, and do not limit the scope of the present application.

[0048] The present application first provides a no-reference image quality evaluation method based on a multi-scale attention mechanism of an inverted pyramid structure, with reference to Figure 1 , comprising the following steps:

[0049] Step 1: first pre-process the distorted image dataset;

[0050] Step 2: input the processed data into a feature extraction network, and use a decomposition large kernel convolution operation to extract four different scale image features;

[0051] Step 3: adjust the four scale features extracted in step 2 to the same spatial size through adaptive average pooling, and add learnable position encoding to each spatial position. Then, use a cross-scale attention mechanism (CSA block) to gradually fuse high-level semantic information into low-level features;

[0052] Step 4: aggregate these distortion features with semantic perception through a multi-layer perceptron (MLP), and finally use a fully connected layer to map the feature representation output by the MLP to a quality score.

[0053] As a preferred embodiment of the present application, in step 1, the images in the distorted image dataset are paired with their corresponding labels, and then the paired data are divided into a training set and a test set in a certain proportion. The images in the training set are cropped, randomly flipped, etc. for training set data enhancement, and finally the images are normalized.

[0054] As a preferred embodiment of the present application, in step 2, the processed data are decomposed into multiple overlapping small blocks through multiple convolution layers, and these small blocks are embedded into a high-dimensional feature space. These small blocks are processed through a large kernel convolution operation to extract feature information of different scales. Finally, these feature maps of different scales lay the foundation for subsequent multi-scale fusion and attention mechanism, allowing the network to learn and fuse information from multiple scale features to improve the accuracy and robustness of image quality evaluation.

[0055] wherein, step 2 specifically comprises the following steps:

[0056] Step 2.1: First, the input image is divided into multiple overlapping small blocks by the convolutional layer, and these small blocks are embedded into a high-dimensional feature space. Then, batch normalization is performed on the convolved tensor to ensure that the features of each small block are more stable during training. Finally, the normalized feature map F1 and the corresponding height and width are returned for subsequent processing.

[0057] Step 2.2: First, batch normalization is performed on the input feature F1, which helps to reduce the distribution deviation of feature values and makes the subsequent convolution operation more stable. Then, lx1 convolution is used to process the normalized features, and the GELU activation function is used to obtain the feature F2. The lx1 convolution is mainly used to adjust the number of channels and fuse the information between channels, while the GELU activation function provides a smooth nonlinear transformation, which helps to enhance the expression ability of the features.

[0058] Step 2.3: First, a 5x5 deep convolution DW_Conv operation is performed on the feature F2 to extract preliminary spatial information. Deep convolution only operates on each channel individually and does not change the relationship between channels, thus maintaining the input channel structure. Then, a 7x7 deep convolution DW_D_Conv with dilation rate is applied to expand the receptive field to capture spatial dependencies in a larger range. Dilation convolution can expand the receptive field without increasing the number of parameters, allowing the model to obtain more extensive spatial features. Finally, lx1 convolution is used to fuse the spatial information extracted in the previous two steps to generate the final attention map. The lx1 convolution here is used to integrate information from different channels, combining the captured spatial features with channel information.

[0059] Step 2.4: Element-wise multiplication (element convolution) is performed between the generated attention map and the original input feature. This process fine-tunes the original features through the attention mechanism, allowing important features to be strengthened. The adjusted features are projected to obtain the output feature F3.

[0060] Step 2.5: F3 is again batch normalized to ensure that the features input to the MLP remain within an appropriate numerical range. The normalized features are input into the MLP, and after multiple linear transformations and nonlinear activations, the output feature F4 is obtained. The MLP further enhances the feature representation ability through complex nonlinear transformations, allowing the network to capture more rich patterns and information in the input data.

[0061] This step is sequentially stacked with L groups of operations to extract features, and the operation description of each group is as follows:

[0062]

[0063] where f GELU represents GELU activation, f BN represents batch normalization, f 1×1 represents 1x1 channel convolution, f DW_Conv and f DW_D_Conv represent depthwise convolution and depthwise separable convolution, respectively. FFN represents convolutional feed-forward network, represents element-wise convolution.

[0064] As a preferred embodiment of the present application, in step 3, the feature map is first subjected to adaptive average pooling processing, adjusted to a unified spatial size, and a learnable position encoding is added for each spatial position, which helps to align features of different scales. Subsequently, the cross-scale attention mechanism (CSA block) is used to fuse high-level semantic information into low-level features. Specifically, the feature map is converted into query, key and value vectors, and the attention weight is calculated. Then, the feature map is weighted and summed, and the high-level semantic information is gradually integrated into the low-level features layer by layer, improving the comprehensive ability of multi-scale features. This processing method can effectively combine the global information of the high layer and the detailed information of the low layer, thereby enhancing the accuracy of image quality assessment.

[0065] wherein step 3 specifically comprises the following steps:

[0066] Step 3.1: The features F4 obtained in step 2 are processed in descending order. First, adaptive average pooling is used to adjust the feature map to the same spatial size. In order to enable the model to better capture spatial position information, we add a learnable position encoding to each spatial position, and adjust the size of the position encoding to match the size of the current feature map by bicubic interpolation. This unified position encoding can enable the subsequent CSA block to better match features from different scales. Then, the feature map is subjected to dimensionality reduction operation, and the feature map is flattened and transposed so as to be able to be added to the adjusted position encoding.

[0067] Step 3.2: In order to better fuse feature maps of different scales, a cross-scale attention mechanism (CSA) is adopted, which takes three groups of feature vectors generated by different scale features as input, i.e., query Q, key K and value V, and recursively aggregates high-level semantic information into low-level features. For the input feature map, first take the highest level feature map G n as the initial query for processing, and recursively perform cross-scale attention calculation between the query and the lower level feature map to generate a new query:

[0068] Q=G i+1 Wq , K = G i W k , V = G i W v ,

[0069] The attention calculation of each step is based on the soft-max function softmax to calculate the attention weight A:

[0070]

[0071] Then use these weights A to weight the sum of the feature map, and superimpose the result on the original feature map:

[0072]

[0073] Where Nq, Nv represents the number of feature vectors, dk, dv is the dimension of the feature vector, which is used to scale the attention score. i ∈ {1,..., n-1}.

[0074] G i represents the i-th layer feature map, Q, K, V represent the query, key and value feature vectors, which are usually generated by linear transformation from the feature map. A is the attention matrix, which represents the similarity between G i and G i+1 . The recursive cross-scale processing of the feature map is carried out from high to low, and the semantic information of the high level is gradually aggregated into the low-level features.

[0075] As a preferred embodiment of the present application, in step 4, first, the cross-scale attention processed features are globally integrated by the attention pooling layer to generate the final global feature representation. This process ensures the effective fusion of high-level and low-level features. Next, the aggregated features are further processed using a multi-layer perceptron (MLP), which extracts features through multiple fully connected layers, and finally maps the processed features to a quality score using a softmax function, providing accurate image quality assessment.

[0076] Wherein, step 4 specifically comprises the following steps:

[0077] Step 4.1: After the cross-scale attention processing is completed, the processed query feature map is finally globally integrated by the attention pooling layer to obtain the final global feature representation:

[0078]

[0079] Wherein, H n ×W nD is the dimension of the feature after dimension reduction. Specifically, first, the input feature is normalized, then the normalized feature is passed through the first linear layer, then through the activation function, and then through the second linear layer. Finally, the output is added to the original input to form a residual structure. This process ensures gradual fusion from high to low layers, effectively aggregating information at different scales, and ultimately generating feature maps rich in multi-scale semantics.

[0080] Step 4.2: Use the obtained in step 4.1 These distortion features with semantic perception are aggregated by a multi-layer perceptron (MLP), and finally the feature representation output by the MLP is mapped to the final score p^ using a fully connected layer, as follows:

[0081]

[0082] Here, the role of the MLP is to process the final feature representation through multiple fully connected layers, further refining the feature representation, and then convert the output feature representation into a probability distribution through the softmax function, thus obtaining the final score p^.

[0083] p^ = softmax(y^)

[0084] In the above description, various combinations of technical features are mentioned, but not all possible combinations are exhaustively listed. However, it should be emphasized that as long as the combination of these technical features is reasonable in implementation and does not contradict each other, it should be considered within the scope contained in this specification.

Claims

1. A no-reference image quality assessment method based on a multi-scale attention mechanism using an inverted pyramid structure, characterized in that, The method includes the following steps: Step 1: First, preprocess the distorted image dataset; Step 2: Input the processed data into the feature extraction network and use the decomposed large kernel convolution operation to extract image features at four different scales. Specifically, in Step 2, the processed data is input into the feature extraction network, and the decomposed large kernel convolution operation is used to extract image features at four different scales, as follows: Step 2.1: First, the input image is segmented into multiple overlapping small blocks through a convolutional layer, and the multiple overlapping small blocks are embedded into a high-dimensional feature space. Then, the tensors after convolution are batch normalized. Finally, the normalized feature F1 and the corresponding height and width are returned for subsequent processing. Step 2.2: First, use 1x1 convolution to process the normalized features, and obtain feature F2 through the GELU activation function. 1x1 convolution is mainly used to adjust the number of channels and fuse information between channels, while the GELU activation function provides a smooth non-linear transformation. Step 2.3: First, a 5x5 depthwise convolution (DW_Conv) is performed on feature F2 to extract preliminary spatial information. The depthwise convolution operates only on each channel individually, without changing the relationships between channels, thus preserving the channel structure of the input. Then, a 7x7 depthwise convolution (DW_D_Conv) with dilation is applied to expand the receptive field and capture spatial dependencies over a wider range. Dilated convolutions can expand the receptive field without increasing the number of parameters, enabling the model to acquire a wider range of spatial features. Finally, a 1x1 convolution is used to fuse the spatial information extracted in the first two steps to generate the final attention map. The 1x1 convolution is used here to integrate information from different channels, combining the captured spatial features with channel information. Step 2.4: Perform element-wise convolution between the generated attention map and the original input features. This process fine-tunes the original features through the attention mechanism, thereby enhancing important features. The adjusted features are then projected to obtain the output feature F3. Step 2.5: Perform batch normalization on F3 again, input the normalized features into MLP, and after multiple linear transformations and nonlinear activation, obtain the output feature F4. MLP further improves the feature representation capability through complex nonlinear transformations, enabling the network to capture richer patterns and information in the input data. Step 2 involves sequentially stacking L groups of operations to extract features. The operation description for each group is as follows: Among them, f GELU Indicates GELU activation, f BN Indicates batch normalization, f 1×1 f represents a 1×1 channel convolution. DW_Conv (·) and f DW_D_Conv (·) denote depthwise convolution and depthwise dilated convolution, respectively, f FFN This represents a convolutional feedforward network. Represents element-wise convolution; Step 3: The four scale features extracted in Step 2 are adjusted to the same spatial size through adaptive average pooling, and a learnable positional encoding is added to each spatial location; then, the high-level semantic information is gradually fused into the low-level features using the cross-scale attention mechanism CSA block. Step 4: Aggregate semantically aware distortion features through a multilayer perceptron (MLP), and finally use a fully connected layer to map the feature representation output by the MLP into a quality score.

2. The no-reference image quality assessment method based on the inverted pyramid structure multi-scale attention mechanism according to claim 1, characterized in that, In step 1, the images in the distorted image dataset are paired with their corresponding labels. Then, the paired data is divided into a training set and a test set according to a certain ratio. The images in the training set are cropped and randomly flipped for training set data augmentation. Finally, the images are normalized.

3. The no-reference image quality assessment method based on a multi-scale attention mechanism with an inverted pyramid structure according to claim 1, characterized in that, Step 3 specifically includes the following steps: Step 3.1: Process the features F4 obtained in Step 2 in descending order. First, use adaptive average pooling to adjust the features to the same spatial size, add a learnable positional encoding for each spatial location, then perform dimensionality reduction on the features, and flatten and transpose the features so that they can be added to the adjusted positional encoding. Step 3.2: Employing a cross-scale attention mechanism (CSA), the query, key, and values ​​feature vectors generated from features at different scales are used as input. This recursively aggregates high-level semantic information into low-level features layer by layer. Specifically, for the input features, the highest-level feature G is first... n Initially, this is processed as the initial query. Then, recursively, cross-scale attention calculations are performed between the query and lower-level features to generate new queries. Q=G i+1 W q ,K=G i W k ,V=G i W v , Attention calculation at each step is based on the softmax function to calculate attention weight A: Then, these weights A are used to perform a weighted summation of the features, and the result is superimposed on the original features to obtain the features fused by the cross-scale attention mechanism (CSA). in Nq represents the number of feature vectors for the query, Nv represents the number of feature vectors for the value, dk is the dimension of the feature vectors for the key, and dv is the dimension of the feature vectors for the value, used to scale the attention score, i∈{1,...,n-1}. H n ×W n G represents the size of the feature after dimensionality reduction, where D is the feature dimension after dimensionality reduction, and G is the feature size after dimensionality reduction. i Let G represent the features of the i-th layer, where Q, K, and V represent the feature vectors of the query, key, and value, respectively, and A is the attention matrix. i and G i+1 The similarity between features is determined by the recursive cross-scale processing of features, which proceeds from high to low levels, gradually aggregating high-level semantic information into low-level features.

4. The no-reference image quality assessment method based on the inverted pyramid structure multi-scale attention mechanism according to claim 1, Its features are, Step 4 specifically includes the following steps: Step 4.1: After cross-scale attention processing, the processed query features are globally integrated through an attention pooling layer to obtain the final global feature representation: Specifically, the input features are first normalized, then passed through the first linear layer, then through the activation function, then through the second linear layer, and finally the output is added to the original input to form the residual structure. Step 4.2: Use the solution obtained in Step 4.1 The MLP aggregates semantically aware distorted features, and finally uses a fully connected layer to map the feature representation output by the MLP into the final score p^, as shown in the following formula: The role of MLP is to process the final global feature representation through multiple fully connected layers to further refine the global feature representation, and then use the softmax function to transform the output global feature representation into a probability distribution, thereby deriving the final score. p^ = softmax(y^).

Images