Image quality evaluation method based on triple attention double scale residual network

Abstract

The application discloses a super-resolution image quality evaluation method based on a triple attention double-scale residual network, first, a super-resolution image is given, and the super-resolution image is cropped into super-resolution image blocks without overlapping; then, the features of the super-resolution image blocks are extracted through a convolution layer, a triple attention module and a double-scale convolution module in sequence; finally, the output features of the feature extraction stage are compressed through a max-pooling operation, the compressed features are sent into a full connection layer for nonlinear quality mapping, and the predicted quality score of the super-resolution image block is output. The triple attention module is used to explore the significant features in the image, and a double-scale convolution module DSCM is designed to capture more comprehensive image information, so that the super-resolution image can be more effectively evaluated.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and relates to a super-resolution image quality evaluation method based on a triple attention double-scale residual network. BACKGROUND

[0002] Existing image quality evaluation methods mainly include subjective quality evaluation methods and objective quality evaluation methods. It is undoubtedly the most direct and effective method to score the image directly by the human eye. However, this process is not only time-consuming, labor-intensive, costly and labor-intensive, but also inconvenient to use, which is not conducive to the integration into the super-resolution application system in the real scene. In contrast, the objective quality evaluation method is to calculate the quality index of the image by the computer according to a certain algorithm, and automatically evaluate. According to whether a reference image is used, the evaluation can be divided into three categories: full reference (FR), reduced reference (RR) and no reference (NR). At present, the peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) are widely used in FR-IQA. However, these two indicators are not designed specifically for super-resolution images, resulting in poor consistency with human perception evaluation. In addition, in practical applications, it is difficult to find the original high-resolution image as a reference, thereby hindering the further development of FRIQA and RRIQA, and therefore, NRIQA has attracted widespread attention from researchers.

[0003] At present, the no-reference type quality evaluation method mostly uses the attention mechanism to simulate the human visual system (HSV) to provide more identifiable features. However, the existing NRIQA method often considers the significant areas in the image from a single branch, ignoring the mutual dependence between them, which limits the effectiveness of the prediction. SUMMARY

[0004] The purpose of the present application is to provide a super-resolution image quality evaluation method based on a triple attention double-scale residual network, which uses a triple attention module to explore the significant features in the image, and designs a double-scale convolution module DSCM to capture more comprehensive image information, so as to achieve the purpose of more effective evaluation of super-resolution images.

[0005] The technical scheme adopted by the present application is a super-resolution image quality evaluation method based on a triple attention double scale residual network, first, a super-resolution image is given, and the super-resolution image is cropped into super-resolution image blocks without overlapping; then, the features of the super-resolution image blocks are extracted through a convolution layer, a triple attention module and a double scale convolution module in turn; finally, the output features of the feature extraction stage are compressed through a max-pooling operation, and the compressed features are sent to a fully connected layer for nonlinear quality mapping, and the predicted quality score of the super-resolution image block is output.

[0006] The present application is also characterized in that:

[0007] Specifically includes the following steps:

[0008] Step 1, all super-resolution images in the super-resolution image dataset are cropped without overlapping to obtain 32x32 size super-resolution image blocks, and local contrast normalization is used to remove areas with the same intensity in the super-resolution image blocks and highlight edge areas;

[0009] Step 2, divide the super-resolution image dataset into training set, test set and validation set in the ratio of 6:2:2, wherein each super-resolution image label in the training set is assigned to the corresponding super-resolution image block, and the label value is denoted as y m ;

[0010] Step 3, a triple attention double scale residual model is constructed to extract super-resolution image features, which mainly includes a triple attention module and a double scale convolution module;

[0011] Step 4, obtain the quality score of the super-resolution image block based on regression mapping;

[0012] Step 5, train the triple attention double scale residual model constructed in step 3 to obtain the optimal model after training;

[0013] Step 6, use the test set to predict the optimal model after training in step 5 to output the predicted quality score of the super-resolution image block.

[0014] The specific process of step 3 is:

[0015] Step 3.1, input the super-resolution image blocks and the corresponding label values y m in the training set of step 2 into a convolution layer Conv with a convolution kernel size of 3x3 to extract the shallow features of the super-resolution image blocks, wherein the output of the kth convolution layer is represented as:

[0016] f k =wx+b (1);

[0017] Wherein, x represents the input feature of the convolution layer, w and b are the learnable parameters of the convolution layer, and fk represent the output features of the convolutional layer;

[0018] Step 3.2, input the output features f k into three branches of the triple attention module to extract saliency features in the super-resolution image block;

[0019] Step 3.3, use the DSCM module to extract multi-scale features in the super-resolution image block.

[0020] The specific process of step 3.2 is as follows:

[0021] Step 3.2.1, the first branch of the triple attention module, establishes interaction between the H dimension and the C dimension: first input the feature tensor f k ∈R C×H×W Rotate 90° counterclockwise along the H axis, and the rotated feature tensor is represented as Then, send to the Z-Pool layer to perform average pooling and max pooling operations, and the obtained features are represented as Next, send through a standard convolutional layer with a kernel size of 7x7 and a batch normalization layer, and use the Sigmoid activation function to generate attention weights, and finally obtain the output feature tensor f 11 , the process is represented as:

[0022]

[0023] wherein, represents the feature tensor rotated along the H axis, represents the feature tensor obtained after the Z-Pool layer, Conv represents the standard convolutional layer, BN represents the batch normalization layer, σ represents the Sigmoid activation function, and f 11 represents the output feature tensor obtained from the first branch;

[0024] Step 3.2.2, the second branch of the triple attention module, establishes interaction between the C dimension and the W dimension, first input the feature tensor f k ∈R C×H×W Rotate 90° counterclockwise along the W axis, and the rotated feature tensor is represented as Then, send to the Z-Pool layer to perform average pooling and max pooling operations, and the obtained features are represented as Next, send through a standard convolutional layer with a kernel size of 7x7 and a batch normalization layer, and use the Sigmoid activation function to generate attention weights, and finally obtain the output feature tensor f 12 , the process is represented as:

[0025]

[0026] Step 3.2.3, the third branch of the triple attention module, establishes an interaction between the H dimension and the W dimension, first inputting the feature tensor f k ∈R C×H×W into the Z-Pool layer to perform average pooling and max pooling operations, and obtaining the feature Next, the is passed through a standard convolution layer with a kernel size of 7x7 and a batch normalization layer, and an attention weight is generated using a Sigmoid activation function (σ), which is added to the input tensor f k ∈R C×H×W , the process is represented as:

[0027]

[0028] wherein, represents the feature tensor obtained after the Z-Pool layer, Conv represents the standard convolution layer, BN represents the batch normalization layer, σ represents the Sigmoid activation function, f k represents the input feature tensor, f 13 represents the output feature tensor obtained from the third branch;

[0029] Step 3.2.4, the saliency feature obtained by the triple attention module is represented as:

[0030]

[0031] wherein, represents the output tensor f 11 in the first branch, and the feature tensor is rotated 90° clockwise in the H dimension; represents the output tensor f 12 in the second branch, and the feature tensor is rotated 90° clockwise in the W dimension; f 13 represents the feature obtained from the third branch, f all represents the saliency feature extracted by the triple attention module.

[0032] The specific process of step 3.3 is as follows:

[0033] Step 3.3.1, the DSCM module has a total of 3, wherein the output of the first DSCM module is denoted as M1;

[0034] Step 3.3.2, the output feature f k from the convolution layer in step 3.1 is concatenated with the second DSCM module to obtain the output M2;

[0035] Step 3.3.3, the output feature f of the convolutional layer in step 3.1 is taken as the input of the third DSCM module, and a residual connection operation is performed with the third DSCM module to obtain an output M3. k Step 3.3.3, the output feature f of the convolutional layer in step 3.1 is taken as the input of the third DSCM module, and a residual connection operation is performed with the third DSCM module to obtain an output M3.

[0036] The specific process of step 4 is as follows:

[0037] Step 4.1, the output feature M3 obtained in step 3.3.3 is subjected to a maximum pooling operation Maxpool for feature compression;

[0038] Step 4.2, the compressed feature obtained in step 4.1 is sent to the fully connected layer FC for nonlinear quality mapping, and finally the quality score q of the super-resolution image block prediction is output, and the above process is represented as:

[0039] q=FC((MaxPool(M3))) (6)。

[0040] The specific process of step 5 is as follows:

[0041] Step 5.1, during the training process, an L1 loss function is used to calculate the loss between the predicted quality score and the label value of the super-resolution image block obtained in the image representation stage, and the loss function L is represented as:

[0042]

[0043] Wherein, q represents the predicted quality score of the super-resolution image block, y m represents the label value of the super-resolution image block, m represents the mth super-resolution image block, and M represents the number of super-resolution image blocks;

[0044] Step 5.2, after the loss is calculated by step 5.1, the model parameters are updated by the Adam optimizer until the loss no longer changes, at which time the training is completed, and the model with the best SROCC index value on the validation set is saved.

[0045] The beneficial effects of the present application are as follows:

[0046] 1. A triple attention module is used to explore the salient regions of the super-resolution image, which overcomes the problem of neglecting the dependent relationship between the channel and the spatial dimension of the single attention mechanism. This is conducive to extracting more accurate quality perception features, thereby quantifying the distortion of the super-resolution image.

[0047] 2. In order to better reveal the distortion of the super-resolution image at different scales, a double-scale convolution module (DSCM) is designed to enhance the representation ability. And the residual connection is used in the network to make up for the loss of visual information, so as to obtain more comprehensive perception features.

[0048] 3. Experimental results on two commonly used super-resolution image quality evaluation databases show that the method can obtain more consistent evaluation results with human visual perception than other existing methods. BRIEF DESCRIPTION OF DRAWINGS

[0049] Figure 1 is the network structure diagram of the super-resolution image quality evaluation method based on the triple attention double-scale residual network of the application;

[0050] Figure 2 is the triple attention module diagram in the application;

[0051] Fig. 3 (a) to (f) are result comparison diagrams of embodiment 2 in the super-resolution image quality evaluation method based on the triple attention double-scale residual network of the application;

[0052] Fig. 4 (a) to (d) are result comparison diagrams of embodiment 3 in the super-resolution image quality evaluation method based on the triple attention double-scale residual network of the application. DETAILED DESCRIPTION

[0053] The application will be described in detail below in combination with the drawings and specific embodiments.

[0054] The super-resolution image quality evaluation method based on the triple attention double-scale residual network of the application mainly consists of image representation, feature extraction and regression mapping three parts on the network model structure. In the image block representation part, given a super-resolution (SR) image, it is cropped into non-overlapping image blocks of 32x32 size, and the quality score of each image block is the same as that of the corresponding original image. In the feature extraction part, the network framework proposed by the application first passes through a 3x3 size convolution, simultaneously simulates the perception process of the human visual system HSV, and introduces a novel triple attention mechanism. Then the application uses different size convolution kernels to obtain image features at different scales, and adds residual blocks to supplement the shallow features extracted and the deep feature information extracted by the double-scale convolution module, and the output thereof is obtained through a maximum pooling layer to obtain deep perception features. In the regression mapping part, the deep features after pooling are sent to a multi-layer perception (MLP) for nonlinear quality mapping.

[0055] In general, the method mainly includes two stages: training stage and test stage. In the training stage, the following steps are implemented:

[0056] The network structure diagram of the super-resolution image quality evaluation method based on the triple attention double-scale residual network is as follows Figure 1Firstly, given a super-resolution image, it is cropped into non-overlapping super-resolution image patches of size 32x32. Then, the features of the super-resolution image patches are extracted by convolutional layers, triple attention module (TripleAttention) and double scale convolution module (DSCM). Finally, the output features of the feature extraction stage are compressed by the max-pooling operation, and the compressed features are sent to the fully connected layer for nonlinear quality mapping to achieve the purpose of effectively evaluating the quality of super-resolution images.

[0057] Specifically comprising the following steps:

[0058] I. Training phase

[0059] Step 1, all super-resolution (SR) images in the super-resolution image dataset are non-overlapping cropped to obtain super-resolution image patches of size 32x32. Local contrast normalization (LCN) is used to remove areas of the same intensity in the super-resolution image patches and highlight edge areas. Step 1 corresponds to Figure 1 the image representation stage.

[0060] Step 2, the super-resolution image dataset is divided in the ratio of 6:2:2, wherein each super-resolution image label in the training set is assigned to the corresponding super-resolution image patch, and the label value is denoted as y m .

[0061] Step 3, a triple attention double scale residual model is constructed to extract super-resolution image features. The model mainly includes a triple attention module and a double scale convolution module. Steps 3.1 to 3.3 correspond to Figure 1 the feature extraction process.

[0062] Step 3.1, the super-resolution image patches and the corresponding label values y m obtained in step 2 are input into a convolutional layer Conv with a convolutional kernel size of 3x3 to extract the shallow features of the super-resolution image patches. The output of the kth convolutional layer can be expressed as

[0063] f k = w x +b (1);

[0064] where x represents the input features of the convolutional layer, w and b are the learnable parameters of the convolutional layer, and f k represents the output features of the convolutional layer.

[0065] Step 3.2, a triple attention module is used for the extraction of significant features in the super-resolution image patches. The triple attention module is as Figure 2 shown. The output features f kThe input is fed into three branches of the triple attention module, and the specific operations in the module are implemented in the following steps:

[0066] Step 3.2.1, the first branch, establishes interaction between the H dimension and the C dimension.

[0067] First, the input feature tensor f k ∈R C×H×W is rotated 90° counterclockwise along the H axis, and the rotated feature tensor is represented as Then, f is fed into the Z-Pool layer to perform average pooling and max pooling operations, and the obtained feature is represented as Next, f is passed through a standard convolution layer with a kernel size of 7x7 and a batch normalization layer, and an attention weight is generated using a Sigmoid activation function. Finally, the output feature tensor f 11 is obtained. This process can be represented as:

[0068]

[0069] wherein, represents the feature tensor rotated along the H axis, represents the feature tensor obtained after the Z-Pool layer, Conv represents the standard convolution layer, BN represents the batch normalization layer, σ represents the Sigmoid activation function, and f 11 represents the output feature tensor obtained from the first branch.

[0070] Step 3.2.2, the second branch, establishes interaction between the C dimension and the W dimension.

[0071] First, the input feature tensor f k ∈R C×H×W is rotated 90° counterclockwise along the W axis, and the rotated feature tensor is represented as Then, f is fed into the Z-Pool layer to perform average pooling and max pooling operations, and the obtained feature is represented as Next, f is passed through a standard convolution layer with a kernel size of 7x7 and a batch normalization layer, and an attention weight is generated using a Sigmoid activation function. Finally, the output feature tensor f 12 is obtained. This process can be represented as:

[0072]

[0073] wherein, represents the feature tensor rotated along the W axis, and f 12is the feature tensor obtained after Z-Pool layer, Conv denotes the standard convolution layer, BN denotes the batch normalization layer, and f 12 is the output feature tensor obtained from the second branch.

[0074] Step 3.2.3, the third branch, which establishes interaction between the H dimension and the W dimension.

[0075] First, the input feature tensor f k ∈R C×H×W is sent to the Z-Pool layer to perform average pooling and max pooling operations, and the obtained feature is represented as Then, f is passed through a standard convolution layer with a kernel size of 7x7 and a batch normalization layer, and a Sigmoid activation function (σ) is used to generate attention weights, which are added to the input tensor f k ∈R C×H×W The process can be represented as:

[0076]

[0077] wherein, is the feature tensor obtained after Z-Pool layer, Conv denotes the standard convolution layer, BN denotes the batch normalization layer, and f k is the input feature tensor, and f 13 is the output feature tensor obtained from the third branch.

[0078] Step 3.2.4, finally, the saliency feature obtained by the triple attention module can be represented as:

[0079]

[0080] wherein, is the output tensor f 11 in the first branch, and f is the feature tensor rotated 90° clockwise in the H dimension. 12 is the output tensor f 13 in the second branch, and f 13 is the feature obtained from the third branch, and f all is the saliency feature extracted by the triple attention module.

[0081] Step 3.3, the DSCM module is used for the extraction of multi-scale features in the super-resolution image block. The saliency feature f allis denoted as M0, and is input into the DSCM module. The DSCM module is composed of a double-branch structure, the first branch is composed of a 3x3 size convolution layer (Conv), and the second branch is composed of a 5x5 size convolution layer and a 3x3 size deconvolution layer (Deconv).

[0082] Step 3.3.1, there are 3 DSCM modules in total. Among them, the output of the first DSCM module is denoted as M1.

[0083] Step 3.3.2, the output feature f k is concatenated (Concat) with the second DSCM module to obtain the output M2.

[0084] Step 3.3.3, the output feature f k is concatenated with the third DSCM module to obtain the output M3.

[0085] Step 4, obtain the quality score of the super-resolution image block.

[0086] Step 4.1 corresponds to Step 4.2 Figure 1 in the regression mapping process.

[0087] Step 4.1, the output feature M3 obtained in Step 3.3.3 is subjected to a max-pooling operation (Maxpool) for feature compression.

[0088] Step 4.2, the compressed feature obtained in Step 4.1 is input into a fully connected layer (FC) for nonlinear quality mapping, and finally outputs the predicted quality score q (Quality Score) of the super-resolution image block. The above process can be represented as:

[0089] q = FC((MaxPool(M3))) (6);

[0090] Step 5, train the model constructed in Step 3, and use the L1 loss function in the training process to calculate the loss between the predicted quality score and the label value of the super-resolution image block obtained in the image representation stage. The loss function L can be represented as:

[0091]

[0092] where q represents the predicted quality score of the super-resolution image block, y m represents the label value of the super-resolution image block, m represents the mth super-resolution image block, and M represents the number of super-resolution image blocks.

[0093] Step 6, after calculating the loss, the model parameters are updated by the Adam optimizer until the loss tends to be stable. In the experiment of the present method, after training all the training samples for 100 rounds, the value of the loss of the validation set no longer changes substantially, at which time the training can be ended, and the model with the best SROCC index value on the validation set is saved.

[0094] II. Test phase

[0095] Step 7, the triple attention double scale residual model of step 3 is called to extract the features of the super-resolution image block. The specific steps are as follows:

[0096] Step 7.1, the super-resolution image block in the test set is input into the convolutional layer in step 3.1 to obtain the shallow features f k .

[0097] Step 7.2, the shallow features f k obtained in step 7.1 are input into the triple attention module of step 3.2 to obtain the saliency features f all .

[0098] Step 7.3, the saliency features f all obtained in step 7.2 are denoted as M0 and input into step 3.3 to obtain the multi-scale features M3.

[0099] Step 8, the regression model in step 4 is called to obtain the predicted quality score of the super-resolution image block. The specific steps are as follows:

[0100] Step 8.1, the multi-scale features obtained in step 7.3 are input into step 4.1 to perform feature compression through the max-pooling layer.

[0101] Step 8.2, the compressed features obtained in step 8.1 are input into the fully connected layer (FC), and the optimal model saved in step 6 is called to predict the super-resolution image block. The average value of all super-resolution image block prediction scores is taken as the quality score of the final super-resolution image, and the test indicators KROCC / SROCC / PLCC / RMSE are calculated, and the predicted quality score of the super-resolution image block is finally output.

[0102] Example 1

[0103] Table 1 shows the effectiveness of the proposed method evaluated by five methods, namely CNN++, Two-Stream, DBCNN, HyperIQA and the latest JCSAN. The obtained performance indicator values are shown in the following table:

[0104] Table 1: Comparison of indicators of the present method and existing super-resolution image quality evaluation methods

[0105]

[0106] Compared with other several mainstream comparison methods, the method of the application is superior in objective evaluation indicators.

[0107] Example 2

[0108] Figures 3(a) to 3(f) The comparison results of the predicted quality scores and the subjective quality scores obtained by various image quality evaluation methods in the QADS dataset are shown in FIG. 3. FIG. 3(a) represents CNN++, FIG. 3(b) represents Two-Stream, FIG. 3(c) represents DBCNN, FIG. 3(d) represents HyperIQA, FIG. 3(e) represents JCSAN, and FIG. 3(f) represents the method of the application TADSRNet. The abscissa represents the subjective quality score, the ordinate represents the predicted quality score, and each point in the coordinate system represents a super-resolution image. By observing Figures 3(a) to 3(f) It can be seen that, compared with other image quality evaluation methods, the points in the scatter diagram of the method of the application TADSRNet fluctuate the least near the fitting curve, so the method of the application has good subjective-objective consistency.

[0109] Example 3

[0110] Figures 4(a) to 4(d) The comparison results of the predicted quality scores and the subjective quality scores of the method of the application in the MA super-resolution dataset under different scale factors are shown in FIG. 4. These super-resolution images are generated by four different super-resolution algorithms for six different scale factors (s∈2, 3, 4, 5, 6, 8). FIG. 4(a) represents the super-resolution algorithm Bicubic, FIG. 4(b) represents the super-resolution algorithm BP, FIG. 4(c) represents the super-resolution algorithm Glanser09, and FIG. 4(d) represents the super-resolution algorithm SRCNN. The abscissa represents the scale factor size, and the ordinate represents the predicted quality score. By observing Figures 4(a) to 4(d) It can be seen that the predicted quality scores of the method of the application TADSRNet are consistent with the corresponding subjective quality scores, which indicates that the method of the application has high consistency in evaluating the quality of super-resolution images under different scale factors.

Claims

1. An image super-resolution quality evaluation method based on a triple attention double-scale residual network, characterized in that: First, given a super-resolution image, the super-resolution image is cropped into super-resolution image blocks without overlapping; then, the features of the super-resolution image blocks are extracted through a convolution layer, a triple attention module and a double scale convolution module in turn; finally, the output features of the feature extraction stage are compressed through a max-pooling operation, and the compressed features are sent to a fully connected layer for nonlinear quality mapping, and the predicted quality score of the super-resolution image block is output; The first branch of the triple attention module establishes interaction between the H dimension and the C dimension: first input the feature tensor Rotate 90° counterclockwise along the H axis, and the rotated feature tensor is represented as ; Then, send to the layer to perform average pooling and maximum pooling operations, and the obtained features are represented as ; Next, pass through a standard convolution layer with a convolution kernel size of and a batch normalization layer, and use a Sigmoid activation function to generate attention weights, and finally obtain an output feature tensor , which is represented as: （1） wherein, denotes the feature tensor after rotation along the H-axis, denotes the feature tensor after passing through the layer, denotes a standard convolutional layer, denotes a batch normalization layer, denotes a Sigmoid activation function, denotes the output feature tensor obtained from the first branch; The second branch of the triple attention module establishes an interaction between the C dimension and the W dimension. First, the input feature tensor is rotated counterclockwise by 90° along the W axis, and the rotated feature tensor is represented as ; then, the rotated feature tensor is sent to the layer to perform average pooling and max pooling operations, and the obtained feature is represented as ; then, the feature is passed through a standard convolution layer with a kernel size of and a batch normalization layer, and an attention weight is generated using a Sigmoid activation function. Finally, the output feature tensor is obtained, and the process is represented as: （2）； The third branch of the triple attention module establishes an interaction between the H dimension and the W dimension, first inputs the input feature tensor into a layer to perform average pooling and maximum pooling operations, and the obtained feature is represented as ; then, the is input into a standard convolution layer with a convolution kernel size of 7*7 and a batch normalization layer, and a Sigmoid activation function is used to generate attention weight values, which are added to the input tensor , and the process is represented as: （3） wherein, denotes the feature tensor obtained after passing through the layers, denotes a standard convolutional layer, BN denotes a batch normalization layer, denotes a Sigmoid activation function, denotes the input feature tensor, denotes the output feature tensor obtained from the third branch; The significance features obtained by the triple attention module are represented as: （4）； wherein, represents the output tensor in the first branch represents the feature tensor after rotating 90° clockwise in the H dimension; represents the output tensor in the second branch represents the feature tensor after rotating 90° clockwise in the W dimension; represents the feature obtained from the third branch, represents the saliency feature extracted by the triple attention module; There are three double-scale convolution modules DSCM in total, wherein the output of the first DSCM module is denoted as ; the output feature of the convolution layer is concatenated with the second DSCM module to obtain the output . ​ The output features of the convolutional layer are denoted as and the residual connection operation with the third DSCM module is performed to obtain the output .

2. The super-resolution image quality assessment method based on triple attention dual-scale residual network according to claim 1, characterized in that: Specifically, the steps include the following steps: Step 1, all super-resolution images in the super-resolution image dataset are cropped without overlapping to obtain 32x32 super-resolution image blocks, and local contrast normalization is used to remove areas with the same intensity in the super-resolution image blocks and highlight edge areas; Step 2, divide the super-resolution image dataset into training set, test set and validation set in the ratio of 6:2:2, wherein each super-resolution image label in the training set is assigned to the corresponding super-resolution image block, and the label value is denoted as ; Step 3, a triple attention double scale residual model is constructed to extract super-resolution image features, which mainly includes a triple attention module and a double scale convolution module; Step 4, based on regression mapping, the quality score of the super-resolution image block is obtained; Step 5, the triple attention double scale residual model constructed in step 3 is trained to obtain the optimal trained model; Step 6, the test set is used to predict the optimal model trained in step 5, and the predicted quality score of the super-resolution image block is output.

3. The super-resolution image quality assessment method based on triple attention dual-scale residual network according to claim 2, characterized in that: The specific process of step 3 is: Step 3.1, obtaining the super-resolution image patches and the corresponding label values in the training set of step 2 The shallow features of the super-resolution image patches are extracted by a convolution layer Conv with a kernel size of 3x3, and the output of the k-th convolution layer is represented as: （1）； wherein x represents an input feature of the convolutional layer, w and b are learnable parameters of the convolutional layer, represents an output feature of the convolutional layer; Step 3.2, output features from step 3.1 extracting saliency features in the super-resolution image patch into three branches of the triple attention module; Step 3.3, use the DSCM module to extract multi-scale features in the super-resolution image block.

4. The super-resolution image quality assessment method based on triple attention dual-scale residual network according to claim 3, characterized in that: The specific process of step 4 is: Step 4.1, the output features obtained in step 3.3.3 are A max-pooling operation Maxpool is performed for feature compression; Step 4.2, the compressed features obtained in step 4.1 are sent to the fully connected layer FC for nonlinear quality mapping, and finally the predicted quality score q of the super-resolution image block is output, and the above process is represented as: （6）。 5. The super-resolution image quality assessment method based on triple attention dual-scale residual network according to claim 4, characterized in that: The specific process of step 5 is: Step 5.1, during the training process, the L1 loss function is used to calculate the loss between the predicted quality score and the label value of the super-resolution image block obtained in the image representation stage, and the loss function L is represented as: （7） where q denotes a prediction quality score of the super-resolved image patch, denotes a label value of the super-resolved image patch, m denotes the m-th super-resolved image patch, and M denotes the number of super-resolved image patches; Step 5.2, after calculating the loss in step 5.1, the model parameters are updated by the Adam optimizer until the loss no longer changes, at which time the training is completed, and the model with the best SROCC index value on the validation set is saved.

Images