An attention mechanism guided content-aware no-reference image quality assessment method

Abstract

The application provides an attention mechanism guided content-aware non-reference image quality evaluation method, which combines a meta-learning framework and image content-aware prediction, obtains a meta model by optimizing and training a Resnet50 network through double-layer gradient descent optimization during meta training, improves sensitivity to various distortion information, inserts a polarized self-attention module in the meta model during meta testing, introduces an attention mechanism in the spatial and channel aspects, effectively utilizes global information and gives different attention, simultaneously fuses multi-level features of the meta model, realizes complete description of local distortion and global distortion of an image, and finally obtains a perceptual quality score of the image through feature dimension reduction and fusion of the weight of the multi-level features. The method uses refined deep semantic features, performs quality evaluation on the basis of understanding image content, is more in line with the human perceptual process, and has good generalization.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically to an attention mechanism-guided method for evaluating the quality of content-aware, referenceless images. Background Technology

[0002] Introducing different types and degrees of distortion into images can negatively impact user experience and subsequent image processing. Therefore, image quality assessment (IQA) is becoming increasingly important in the image imaging process. Since effective reference images or image features are often unavailable in practical applications, referenceless image quality assessment (NR-IQA) methods have become a mainstream research direction, widely used in image compression, image denoising, image fusion, and surveillance systems, among other fields.

[0003] Recently, deep learning networks have gained widespread attention due to their ability to not only map image features to quality scores but also automatically capture deep features to represent distortion, enabling end-to-end evaluation. Consequently, deep learning-based IQA methods have attracted considerable interest, leading to various evaluation models based on deep convolutional neural networks. Since image quality perception is highly subjective, key mechanisms within the human visual system should be considered when developing IQA models. In many cases, image quality is also related to the consistency of image content, as distortion significantly impacts image readability. Overall, several challenges remain in IQA, such as the diversity of distorted image content, the complexity of distortion, and the uneven distribution of human visual attention to distorted images. Summary of the Invention

[0004] The main problems addressed by this invention are: the diversity of content in distorted images, the complexity of distortion in distorted images, and the uneven distribution of human visual attention to distorted images.

[0005] To achieve the above objectives, this invention proposes a novel attention-guided content-aware no-reference image quality assessment model. It employs a meta-learning method to learn distortion information in images, enhancing the overall network's ability to extract distortion-sensitive features. Simultaneously, it introduces a polarization self-attention module from the no-reference image quality assessment model to explore spatial and channel attention, simulating the selective spatial attention and contrast sensitivity of the human visual system. Furthermore, it extracts multi-level features from images to more comprehensively describe image details and high-level semantic information. After learning content features, it adaptively adjusts the learned perceptual quality rules based on the image content. This invention utilizes refined deep semantic features to perform quality assessment based on an understanding of image content, which is more consistent with the human perceptual process and exhibits good generalization ability.

[0006] This invention provides an attention-based, content-aware, no-reference image quality assessment method, comprising the following steps:

[0007] A no-reference image quality assessment model is constructed, which includes two phases: meta-training and meta-testing.

[0008] Obtain no-reference images and construct a training set;

[0009] In the meta-training phase, the training set is input into the model, and a meta-learning method is adopted. ResNet50 is used as the backbone network, and the model is trained through a two-layer gradient optimization strategy. After training, the meta-model is obtained.

[0010] In the meta-testing phase, a polarization self-attention module is added to the meta-model to obtain a new model, which learns a weighted combination of features along the spatial and channel dimensions.

[0011] The features of the fusion attention mechanism output from the last layer of the new model are fused with the local features of the lower layers of the new model to obtain multi-level features;

[0012] Image content information is learned by extracting high-level semantic features from the last layer of the new model. The weights and biases of the new model are adaptively determined by the weight generation network, resulting in a quality prediction network.

[0013] Multi-level features are input into the quality prediction network to obtain the image quality prediction score.

[0014] Furthermore, a new model is obtained by adding a polarization self-attention module to the meta-model. The specific expression for the weighted combination of features learned along the spatial and channel dimensions is as follows:

[0015] Z = A C (X)⊙ C X+A S (X)⊙ S X

[0016] Among them, A C and A S These are the learned channel and spatial weight matrices, respectively. C and ⊙ S These represent the corresponding multiplication of the channel and spatial dimensions, respectively, where X is the original input feature map.

[0017] Furthermore, the features of the fusion attention mechanism output from the last layer of the new model are fused with the local features from the lower layers of the new model to obtain the specific expression of the multi-level features:

[0018]

[0019] Among them, v z This represents the multi-level features after fusion. L represents the concatenation operation, Z represents the feature of the fusion attention mechanism in the output of the last layer of the new model, and L represents the concatenation operation. i This represents the attention-weighted local features extracted from convolutional layers Layer1, Layer2, and Convolutional Layer3 in ResNet, where i = 1, 2, 3.

[0020] Furthermore, the dimensions of Layer1, Layer2, and Layer3 are 256×56×56, 512×28×28, and 1024×14×14, respectively.

[0021] Furthermore, the high-level semantic information is reduced in dimensionality through convolution, and then the weights Fci_w of the corresponding fully connected layer are generated through two-dimensional convolution and shape reshaping, and the biases Fci_b of the corresponding fully connected layer are generated through global average pooling and shape reshaping.

[0022] Furthermore, the specific process of feeding the multi-level features into the quality prediction network composed of fully connected layers to obtain the quality prediction score is as follows:

[0023] f θ (v z ,g(Z))=q

[0024] Where g(·) represents the weight generation network, and θ represents the quality prediction network f θ The parameter q represents the predicted quality score.

[0025] Furthermore, the backbone network is a ResNet50 feature extraction network.

[0026] Furthermore, the quality prediction network is composed of multiple fully connected layers connected in sequence.

[0027] Furthermore, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the attention mechanism-guided content-aware no-reference image quality assessment method described above.

[0028] Furthermore, the present invention provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the attention mechanism-guided content-aware no-reference image quality assessment method described above.

[0029] This invention provides an attention-guided content-aware, no-reference image quality assessment method. This method combines a meta-learning framework with image content-aware prediction. During training, a ResNet50 network is optimized using double-layer gradient descent to obtain a meta-model, improving sensitivity to various distortions. During testing, a polarization attention module is inserted into the meta-model, introducing spatial and channel-level attention mechanisms to effectively utilize global information and assign different attention levels. Simultaneously, multi-level semantic features of the meta-model are fused to achieve a complete description of both local and global image distortions. Finally, feature dimensionality reduction and fusion are used to obtain the weights of the multi-level semantic features, resulting in a perceived image quality score. This method utilizes refined deep semantic features to perform quality assessment based on an understanding of image content, which is more consistent with human perception and exhibits good generalization ability. Attached Figure Description

[0030] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0031] Figure 1 The following is an execution flowchart of an attention mechanism-guided content-aware no-reference image quality assessment method provided in an embodiment of the present invention;

[0032] Figure 2 This is a schematic diagram of the structure of an attention mechanism-guided content-aware no-reference image quality assessment model provided in an embodiment of the present invention.

[0033] Figure 3 This is a schematic diagram of the polarization self-attention module structure provided in an embodiment of the present invention;

[0034] Figure 4 This is a schematic diagram of the local feature extraction module structure provided in an embodiment of the present invention;

[0035] Figure 5 A comparison chart of image quality evaluation scores between the model of the present invention and other models is provided for embodiments of the present invention.

[0036] Figure 6 This is a comparison of the visual heatmaps of the model provided in this embodiment of the invention and the baseline model without any added modules;

[0037] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0038] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0039] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0040] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0041] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0042] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0043] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0044] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0045] See Figure 1 and Figure 2 This invention provides an attention-based content-aware no-reference image quality assessment method, comprising the following steps:

[0046] S1. Construct a no-reference image quality assessment model, which includes two stages: meta-training and meta-testing.

[0047] S2: Obtain no-reference images and construct a training set;

[0048] S3: In the meta-training phase, the training set is input into the model, and a meta-learning method is adopted. ResNet50 is used as the backbone network, and the model is trained through a two-layer gradient optimization strategy. That is: ① Through the training of the optimizer, the meta-model can optimize the update direction and step size of each parameter under the guidance of the meta-learner, and adapt to the current distortion task more quickly. ② The meta-model can effectively obtain shared prior knowledge between different distortion types, so that only a small number of samples are needed to obtain good generalization performance on the target task. After training, the meta-model is obtained.

[0049] S4: In the meta-testing phase, a polarization self-attention module is added to the meta-model to obtain a new model, which learns a weighted combination of features along the spatial and channel dimensions.

[0050] Specifically, see Figure 3In the channel attention branch, the feature map is transformed into a V of size C / 2×H×W and a Q of size 1×H×W through 1×1 convolution, and then obtained through shape reshaping. and Enhanced with Softmax The information is processed through a 1×1 convolution, LayerNorm, and Sigmoid activation function to obtain the weight value for each channel, specifically:

[0051]

[0052]

[0053] Among them, F sm (·) is the Softmax activation function, F sig (·) is the Sigmoid activation function.

[0054] In the spatial attention branch, the feature maps are transformed into V and Q of size C / 2×H×W using 1×1 convolutions. Global pooling is used to compress the Q-space dimension, transforming it into a 1×1 size, which is then reshaped to obtain... and And through Softmax The information is enhanced and then obtained through shape reshaping. The spatial weights obtained using the Sigmoid function are represented as follows:

[0055]

[0056]

[0057] The specific expression for the weighted combination of features learned along the spatial and channel dimensions is:

[0058] Z = A C (X)⊙ C X+A S (X)⊙ S X

[0059] Among them, A C and A S These are the learned channel and spatial weight matrices, respectively. C and ⊙ S These represent the corresponding multiplication of the channel and spatial dimensions, respectively, where X is the original input feature map.

[0060] S5: Fuse the features of the fusion attention mechanism output from the last layer of the new model with the local features of the lower layers of the new model to obtain multi-level features.

[0061] Specifically, the high-level features extracted by the last convolutional layer of a convolutional neural network only represent global information, while the shallower features of the network often contain richer detailed information, such as local details and texture information. Therefore, combining high-level semantic information with low-level detailed information is crucial. (See [link to documentation]). Figure 4 Multi-level features are extracted from the convolutional layers Layer1 (256×56×56), Layer2 (512×28×28), and convolutional Layer3 (1024×14×14) in ResNet50.

[0062] To reduce network parameters and facilitate faster computation, 1×1 convolutions are applied for dimensionality reduction in the local feature extraction branch, followed by global average pooling. This reduces dimensionality while preserving more image information, facilitating feature fusion. Finally, fully connected layers map the refined multi-level features into vectors. With the introduction of multi-level features, the network can now be defined as follows:

[0063]

[0064] Among them, v z This represents the multi-level features after fusion. L represents the concatenation operation, Z represents the feature of the fusion attention mechanism in the output of the last layer of the new model, and L represents the concatenation operation. i This represents the attention-weighted local features extracted from convolutional layers Layer1, Layer2, and Convolutional Layer3 in ResNet, where i = 1, 2, 3.

[0065] S6. Learn image content information through the high-level semantic features extracted from the last layer of the new model, input the weight generation network to adaptively determine the weights and biases of the new model, and obtain the quality prediction network.

[0066] The high-level semantic information is reduced in dimensionality through convolution, and then the weights Fci_w of the corresponding fully connected layers are generated through two-dimensional convolution and shape reshaping. The biases Fci_b of the corresponding fully connected layers are generated through global average pooling and shape reshaping.

[0067] Specifically, the process of feeding the multi-level features into a quality prediction network composed of fully connected (FC) layers to obtain quality scores is as follows:

[0068] f θ (v z ,g(Z))=q

[0069] Where g(·) represents the weight generation network, and θ represents the quality prediction network f θ The parameter q represents the predicted quality score.

[0070] To verify the beneficial effects of the method of this invention, experimental verification was conducted. The experimental setup, implementation details, and dataset selection are described in the specific experimental scheme. Then, overall dataset performance evaluation and single-distortion dataset performance evaluation were carried out compared with other methods. Finally, ablation experiments were conducted to verify the effectiveness of each module.

[0071] The implementation of the method in this invention includes two steps: (1) during meta-training, the base network Restnet50 is trained using a specific distortion NR-IQA task constructed from a synthetic distortion dataset; (2) during meta-testing, a content-aware network (new model) based on an attention mechanism is trained on each dataset. During network training, the network parameters are uniformly trained 16 times using an Adam optimizer with a weight decay rate of 5e-4 and a learning rate of 2e-5. The evaluation metrics are SROCC and PLCC to measure the prediction results.

[0072] During meta-testing, four public datasets were selected for experimental validation: LIVE, CSIQ, LIVEC, and KonIQ-10k. First, training and testing sets were created. During training, input images were randomly sampled and horizontally flipped into 25 224×224 pixel image patches. During testing, the scores of the sampled image patches from each image were averaged to obtain the final prediction score. Since all datasets used had small sample sizes, choosing an appropriate batch size (bs) when adjusting model parameters improved the model's training performance and accuracy. Therefore, bs was adjusted for different datasets: 72 for the KonIQ-10k dataset, 96 for the LIVEC dataset, and 72 for both the LIVE and CSIQ datasets. Finally, the relationship between the predicted quality score and the labeled score was trained by minimizing the L1 loss function, calculated as follows:

[0073]

[0074] In the formula, f(·) represents the overall network, and q i The label score represents the image's annotation score, and N represents the total number of training samples.

[0075] The performance of current mainstream NR-IQA methods was compared, including methods based on manual feature extraction, synthetic IQA methods based on deep learning, and real IQA methods based on deep learning. To avoid bias, the training process was repeated 10 times, and the median values ​​of PLCC and SROCC were reported. The results are shown in Tables 1 and 2.

[0076] As shown in Tables 1 and 2, the model of this invention achieved SROCC results of 0.884, 0.917, 0.965, and 0.975, and PLCC results of 0.895, 0.930, 0.967, and 0.974, respectively, on the LIVEC, Koniq-10k, CSIQ, and LIVE datasets. Furthermore, except for the CSIQ dataset, the method of this invention achieved the best prediction accuracy in all datasets, and also achieved the second-highest prediction accuracy on CSIQ, second only to DBCNN. For the average prediction results across the four datasets, the method of this invention improved the SROCC results by 2.41% and the PLCC results by 2.39% compared to the DBCNN method.

[0077] Table 1. SROCC results of different NR-IQA methods

[0078]

[0079] Table 2 PLCC Results for Different NR-IQA Methods

[0080]

[0081] This invention analyzes the test results from three aspects:

[0082] (1) First, for most datasets, the prediction accuracy of the model proposed in this invention is significantly better than the three models based on manually extracted features. Experimental results show that the multi-level features with attention mechanism proposed in this invention learn richer information than the methods based on manually extracted features.

[0083] (2) Next, compared with deep learning models designed for synthetic distortion, the model of this invention achieved the best SROCC values ​​on all datasets. On real datasets, the accuracy of the method of this invention is significantly better than that of synthetic IQA methods, indicating that models specifically trained for synthetic distortion have significant limitations in predicting real distortion. On synthetic datasets, the method of this invention also demonstrates great potential and excellent prediction accuracy, with PLCC results on CSIQ second only to the WaDIQaM method. Compared with WaDIQaM's SROCC results, the method of this invention outperforms it by approximately 1.05% on CSIQ and approximately 2.20% on LIVE. For PLCC results, the method of this invention outperforms WaDIQaM-NR by approximately 1.14% on LIVE.

[0084] (3) Finally, compared with real IQA methods based on deep learning, the method proposed in this invention achieves the best prediction results for both synthetic and real distortions. This shows that learning image content helps improve the quality of predicted perceptual images, because humans have different sensory perceptions of a wide range of image types. Compared with SFA, a method that uses convolutional networks to extract deep semantic features, the method of this invention significantly outperforms SFA on KonIQ-10k and LIVEC. Compared with the similarly content-aware HyperIQA method, it is demonstrated that the attention mechanism of this invention can further refine features, conforming to the perceptual characteristics of the human eye. The method of this invention outperforms it by approximately 2.91% SROCC and 1.47% PLCC on LIVEC, approximately 1.21% SROCC and 1.41% PLCC on KonIQ-10k, approximately 4.55% SROCC and 2.65% PLCC on CSIQ, and approximately 1.35% SROCC and 0.83% PLCC on LIVE.

[0085] In summary, the main reasons why this invention achieves the best prediction accuracy are: (1) the meta-model pre-trained on the synthetic dataset can learn more distortion-related knowledge and can also adapt well to distortions that have not been trained; (2) the proposed local feature extraction module integrates the attention mechanism into multi-level features, which is more in line with the characteristics of real distorted images; (3) the deep semantic features refined by the attention mechanism realize content-aware distorted image quality prediction, which is more in line with the process of human image prediction.

[0086] To evaluate the prediction performance of the content-aware model of this invention for different distortion types, experiments were conducted on a synthetic distortion dataset with single distortion types. In the experiments, all distortion types in LIVE and CSIQ were used to train the model, and then the evaluation effect for specific distortion types was tested. The SROCC results of the method of this invention and other methods are shown in Tables 3 and 4.

[0087] As shown in Table 3, the method of this invention achieved the highest prediction accuracy on the four distortion types JP2K, WN, GB, and FF in LIVE, with SROCC results of 0.965, 0.988, 0.978, and 0.957, respectively. For the JPEG distortion type, the prediction results were second only to DBCNN. In Table 4, the method of this invention outperformed all competing NR-IQA methods on the four distortion types JPEG, JP2K, PN, and CC in CSIQ. The method of this invention outperformed WaDIQaM by approximately 14.89% on JPEG, approximately 2.75% on JP2K, approximately 9.86% on PN, and approximately 2.93% on CC.

[0088] Table 3 shows the SROCC results for images with different distortion types on the LIVE dataset.

[0089]

[0090] Table 4 shows the SROCC results for images with different distortion types on the CSIQ dataset.

[0091]

[0092] In summary, the method of this invention outperforms other methods in 8 out of 11 distortion types. This is mainly because combining image content information with the attention mechanism is more helpful in learning perceptual quality, while meta-learning further improves the model's generalization performance for various distortion types.

[0093] To explore the effectiveness of each module in the model of this invention, ablation experiments were further conducted in this embodiment. First, the model was trained and tested on two datasets, LIVE and LIVEC, using a pre-trained ResNet50 as the baseline model. Second, a two-layer optimization training was performed on ResNet50, denoted as ResNet50+Meta. Next, multi-level features and image content awareness processes were extracted, denoted as ResNet50+MT_Content. Then, an attention mechanism PSA was added to simulate the attention distribution when the human eye observes an image. Finally, the final SROCC and PLCC indices were obtained by combining all modules through a meta-learning framework. The experimental results are shown in Table 5.

[0094] Table 5 Ablation study results from the LIVE and LIVEC datasets.

[0095]

[0096] It can be observed that the performance after using each module is significantly improved compared to the baseline model, which verifies that the strategy proposed in this invention is very important for the IQA process.

[0097] Using images from the LIVEC and KonIQ datasets as examples, this invention provides subjective quality scores for 20 distorted images and the corresponding quality scores predicted by the model of this invention. Figure 5As shown, the first row is the subjective quality score (GS), the second row is the predicted score with the MT_Content module added, and the third row is the predicted score of the model of this invention. It can be seen that the added MT_Content module, with its multi-level features, mimics the information generated by the human eye when evaluating an image. Combined with high-level semantic information, it can, to some extent, eliminate the impact of image content changes and local distortions on quality prediction, preserving more detailed features that are visually sensitive to the human eye, thus achieving reliable prediction results. Furthermore, after incorporating the meta-learning process and attention mechanism, the predicted values ​​of the model of this invention for most distorted images are basically consistent with the subjective quality score, which further verifies the robustness of the model to different distortions and the accuracy of the predicted image quality.

[0098] like Figure 6 As shown, Figure 6 The diagram shows a comparison of the model of this invention and a baseline model without any modules. It demonstrates that the method of this invention learns image information from both channel and spatial dimensions, enhancing effective features and suppressing ineffective features in distorted images, thus accurately focusing on regions of interest to the human eye. For example, in the left image, the model focuses more on the entire area of ​​ice cream and fingers, extracting texture features of fingers more accurately; in the right image, the model effectively delineates the outline details of feathers while ignoring the influence of a blurred background. The model of this invention can combine information from different dimensions to obtain richer image features, achieving considerable results on four datasets. This model emphasizes regions of interest to the human eye, exploring high-quality perceptual features by combining salient regions and local details based on global image analysis.

[0099] like Figure 7As shown, a schematic diagram of the physical structure of an electronic device is illustrated. The electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute the steps of the aforementioned attention mechanism-guided content-aware no-reference image quality assessment method, specifically including: constructing a no-reference image quality assessment model, which includes two stages: meta-training and meta-testing; acquiring no-reference images and constructing a training set; in the meta-training stage, inputting the training set into the model, using a meta-learning method with ResNet as the backbone network, and training through a two-layer gradient optimization strategy, resulting in a meta-model; in the meta-testing stage, adding a polarization self-attention module to the meta-model to obtain a new model, and learning a weighted combination of features along the spatial and channel dimensions; fusing the features of the fusion attention mechanism output from the last layer of the new model with the local features of the lower layers of the new model to obtain multi-level features; learning image content information through the high-level semantic features extracted from the last layer of the new model, inputting them into a weight generation network to adaptively determine the weights and biases of the new model, resulting in a quality prediction network; and inputting the multi-level features into the quality prediction network to obtain the image quality prediction score.

[0100] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] In another aspect, embodiments of the present invention also provide a storage medium storing a computer program, which, when executed by a processor, implements the steps of the aforementioned attention mechanism-guided content-aware no-reference image quality assessment method. Specifically, the method includes: constructing a no-reference image quality assessment model, which comprises two phases: meta-training and meta-testing; acquiring no-reference images and constructing a training set; in the meta-training phase, inputting the training set into the model, employing a meta-learning method with ResNet as the backbone network, and training through a two-layer gradient optimization strategy, resulting in a meta-model; in the meta-testing phase, adding a polarization self-attention module to the meta-model to obtain a new model, and learning a weighted combination of features along spatial and channel dimensions; fusing the features from the last layer of the new model that incorporate the attention mechanism with the local features from the lower layers of the new model to obtain multi-level features; learning image content information through the high-level semantic features extracted from the last layer of the new model, inputting them into a weight generation network to adaptively determine the weights and biases of the new model, resulting in a quality prediction network; and inputting the multi-level features into the quality prediction network to obtain the image quality prediction score.

[0102] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0103] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

Claims

1. An attention mechanism guided content-aware no-reference image quality assessment method, characterized in that, The method comprises the following steps: A no-reference image quality evaluation model is constructed, which comprises two stages of meta-training and meta-testing; A no-reference image is acquired to construct a training set; In the meta-training stage, the training set is input into the model, a meta-learning method is adopted, Resnet is used as a backbone network, a double-layer gradient optimization strategy is used for training, and after the training is completed, a meta-model is obtained; In the meta-testing stage, a polarization self-attention module is added to the meta-model to obtain a new model, and the weighted combination of features along the spatial and channel dimensions is learned; The fused attention mechanism features output by the last layer of the new model and the local features of the low layer of the new model are fused to obtain multi-level features; The high-level semantic feature extracted by the last layer of the new model is used to learn the image content information, the weight and bias of the new model are adaptively determined by inputting the weight generation network, and a quality prediction network is obtained; The multi-level features are input into the quality prediction network to obtain the quality prediction score of the image; The specific expression of the weighted combination of features along the spatial and channel dimensions learned by the new model obtained by adding the polarization self-attention module to the meta-model is: wherein, and are learned channel and spatial weight matrices, respectively, and denote channel and spatial dimension-wise multiplication, respectively, is the original input feature map; The specific expression of the multi-level features obtained by fusing the fused attention mechanism features output by the last layer of the new model and the local features of the low layer of the new model is: wherein, represents the fused multi-level features, represents the concatenation operation, represents the feature of the fusion attention mechanism of the last layer output of the new model, represents the attention mechanism weighted local features extracted from the convolutional layers Layer1, Layer2 and convolutional Layer3 in Resnet, .

2. The method of claim 1, wherein the method is attention mechanism guided content-aware no-reference image quality assessment. The sizes of Layer 1, Layer 2, and Layer 3 are , and .

3. The method of claim 1, wherein the method is attention mechanism guided content-aware no-reference image quality assessment. The high-level semantic information, after being reduced in dimension by convolution, generates the weights of the corresponding fully connected layer through two-dimensional convolution and shape remodeling , and generates the bias of the corresponding fully connected layer through global average pooling and shape remodeling .

4. The method of claim 1, wherein the method is attention mechanism guided content-aware no-reference image quality assessment. The specific process of inputting the multi-level features into the quality prediction network composed of fully connected layers to obtain the quality prediction score is: wherein, denotes a weight generation network, denotes a quality prediction network parameters of, denotes a predicted quality score.

5. The method of claim 1, wherein the method is attention mechanism guided content-aware no-reference image quality assessment. The backbone network is a Resnet50 feature extraction network.

6. The method of claim 1, wherein the method is attention mechanism guided content-aware no-reference image quality assessment. The quality prediction network is composed of a plurality of fully connected layers connected in sequence.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the content-aware no-reference image quality evaluation method guided by the attention mechanism according to any one of claims 1-6.

8. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the content-aware no-reference image quality evaluation method guided by the attention mechanism according to any one of claims 1-6.

Images