A no-reference image quality assessment method based on deep learning
Through multi-module feature extraction network and feature fusion technology, the problem of difficulty in accurately judging multiple distorted types of images in the prior art is solved, and accurate evaluation of image quality and effective analysis of hierarchical features are achieved.
Patent Information
- Application Number
- CN202210834263.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-14
- Publication Date
- 2025-04-11
- Estimated Expiration
- 2042-07-14
AI Technical Summary
Existing reference-free image quality evaluation methods are difficult to accurately judge the distortion types of images of multiple distortion types, and deep learning-based methods lack clear physical significance, which affects the analysis of feature hierarchy degradation.
A multi-module feature extraction network is adopted, including the underlying texture feature extraction network, the low-level contour feature extraction network and the advanced global semantic feature extraction network. The final fusion feature map is generated through feature fusion, and multiple fully connected layers and activation layers are input to obtain the image quality evaluation prediction score.
Accurate distortion type judgment and quality evaluation prediction scores for various distortion types images are achieved, which improves the accuracy of image quality evaluation and the clearness of physical significance of hierarchical characteristics.
Smart Images
Figure CN115272203B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a reference-free image quality evaluation method based on deep learning. Background Art
[0002] Among the three popular objective image quality assessment methods, the full-reference image quality assessment method and the semi-reference image quality assessment method both need to use the information of lossless reference images. Their core is to compare the amount of information or feature similarity between the reference image and the distorted image. However, in practical applications, it is difficult to obtain lossless reference images in most cases, that is, only distorted images can be provided to the model, and the original undistorted reference images cannot be provided. With the introduction of the non-reference image quality assessment method, the method's independence from lossless reference images has, to a certain extent, made up for the shortcoming of the full-reference and semi-reference image quality assessment methods' strong dependence on lossless reference images.
[0003] Traditional no-reference image quality assessment methods attempt to manually design feature descriptors in the hope of extracting image features associated with quality degradation. Among them, the method based on natural scene statistics is one of the most popular methods. However, the types of distortion in natural scenes vary greatly, and manually designed feature descriptors can only represent a single type of distortion.
[0004] In the no-reference image quality assessment method based on deep learning, if only a single neural network is used to extract hierarchical information, both the underlying local information and the high-level global information will lack clear physical meaning, which will seriously affect the network's analysis of feature hierarchy degradation.
[0005] Therefore, based on the existing image quality evaluation methods, how to accurately judge the distortion type of an image when faced with images of various distortion types has become a problem that technical personnel in this field urgently need to solve. Summary of the invention
[0006] In view of the above problems, the present invention proposes a reference-free image quality assessment method based on deep learning that at least solves some of the above technical problems. This method can accurately judge the distortion type of an image when faced with images of multiple distortion types, and give a predicted score for the distorted image.
[0007] The embodiment of the present invention provides a method for evaluating image quality without reference based on deep learning, comprising the following steps:
[0008] S1. Obtaining a dataset image of an image quality to be predicted; and uniformly adjusting the dataset image to a first preset size;
[0009] S2, inputting the dataset image adjusted to a first preset size into a feature extraction network, and extracting three different types of feature maps; the feature extraction network includes: an underlying texture feature extraction network, a low-level contour feature extraction network, and a high-level global semantic feature extraction network; the three different types of feature maps include: a texture feature matrix map, a low-level contour feature matrix map, and a global semantic feature map;
[0010] S3, respectively broadcasting and adding the intermediate features in the texture feature matrix map and the intermediate features in the low-level contour feature matrix map to perform feature fusion and generate a first fused feature map;
[0011] S4, broadcasting and adding the intermediate features in the first fused feature map and the intermediate features in the global semantic feature map to perform feature fusion and generate a final fused feature map;
[0012] S5. Input the final fused feature map into multiple fully connected layers and activation layers to obtain a quality evaluation prediction score of the image in the data set whose image quality is to be predicted.
[0013] Furthermore, the step S2 comprises:
[0014] Inputting the dataset image adjusted to a first preset size into the underlying texture feature extraction network to obtain underlying texture features; the underlying texture feature extraction network is a VGG19 network;
[0015] Passing the underlying texture features through the neural network convolution layer Conv2d(256,512,1) and the neural network convolution layer Conv2d(512,512,9,7) in sequence, adjusting the underlying texture features to a texture feature matrix image of a second preset size;
[0016] Further,
[0017] Inputting the dataset image adjusted to a first preset size into the low-level contour feature extraction network to obtain low-level contour features;
[0018] Passing the low-level contour features through a neural network convolution layer Conv2d (640, 512, 1) to adjust the low-level contour features into a low-level contour feature matrix image of a third preset size;
[0019] Further,
[0020] Inputting the dataset image adjusted to a first preset size into the advanced global semantic feature extraction network to obtain a global semantic feature map;
[0021] Further,
[0022] The advanced global semantic feature extraction network removes the flattening layer and the fully connected layer in the original resNet50 network, and adds a neural network convolution layer Conv2d (2048, 512, 1) before the average pooling layer.
[0023] Furthermore, the feature fusion in step S3 and step S4 includes:
[0024] S11, respectively broadcast and add the intermediate features in the two feature maps for feature fusion;
[0025] S12, extracting the global feature context and the local feature context of the intermediate feature after broadcast addition; calculating the channel weight according to the global feature context and the local feature context; recalculating the intermediate feature according to the channel weight; replacing the intermediate feature in step S11 with the recalculated intermediate feature;
[0026] S13, repeatedly executing steps S11 to S12 for a preset number of times; using the broadcast addition result of the global feature context and the local feature context extracted by the last execution of step S12 as the feature fusion result of the two feature maps for feature fusion.
[0027] Further, extracting the global feature context of the intermediate features after broadcast addition includes: performing global average pooling on the intermediate features after broadcast addition;
[0028] The intermediate features after broadcast addition are globally averaged pooled using the following formula:
[0029]
[0030] In the above formula, g(x) represents the global average pooling operation; H*W represents the size of the feature map; x[ :,i,j ] represents the pixel information of all coordinate points (i, j) in the feature map; x represents the intermediate features in the feature map.
[0031] Furthermore, the global feature context of the intermediate features after broadcast addition is extracted by the following formula:
[0032] G(x)=b(D i r(b(D r (g(x))))
[0033] In the above formula, G(x) represents the global feature context; g(x) represents the global average pooling operation; D r represents a channel reduction layer; b represents a batch normalization operation; r represents a linear rectification activation function; D i represents a channel enlargement layer; x represents the intermediate feature in the feature map.
[0034] Furthermore, the local feature context of the intermediate features after broadcast addition is extracted by the following formula:
[0035] L(x)=b(PWconv(r(b(PWConv(x)))),
[0036] In the above formula, L(x) represents the local feature context; b represents the batch normalization operation; r represents the linear rectification activation function; PWConv represents the point-level convolution; x represents the intermediate feature in the feature map.
[0037] Furthermore, the channel weight is calculated by the following formula:
[0038]
[0039] In the above formula, w(x) represents the channel weight; represents a broadcast weighted operation; L(x) represents the local feature context; G(x) represents the global feature context; x represents the intermediate feature in the feature map; sS(.) represents the sigmoid function.
[0040] Furthermore, it also includes:
[0041] Input the three different types of feature maps into multiple fully connected layers and activation layers respectively to obtain three intermediate prediction scores to form an intermediate score set;
[0042] Generate an overall loss function by calculating the loss between the intermediate score set and the true label score, and the loss between the quality assessment prediction score and the true label score;
[0043] The feature extraction network is auxiliary trained according to the overall loss function.
[0044] By reducing the loss between the quality assessment prediction score and the true label score, an optimal mapping relationship between the dataset image of the image quality to be predicted and the quality assessment prediction score is obtained.
[0045] Furthermore, the overall loss function is:
[0046]
[0047] In the above formula, L s represents the sum of the losses between the intermediate score set and the true label score; L e Represents the loss between the quality evaluation prediction score and the true label score; s i represents the intermediate prediction score; s e represents the quality evaluation prediction score; b i Represents the relative weight of the intermediate prediction score in the overall loss function.
[0048] Furthermore, L is calculated by the following formula: s and L e :
[0049]
[0050] In the above formula, a represents a preset hyperparameter; s represents the quality evaluation prediction score or the intermediate prediction score; and m represents the true label score.
[0051] Furthermore, b is calculated by the following formula i :
[0052]
[0053] in,
[0054] In the above formula, si represents the intermediate prediction score; m represents the true label score.
[0055] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:
[0056] The embodiment of the present invention provides a method for evaluating image quality without reference based on deep learning, comprising: obtaining a dataset image of an image quality to be predicted; uniformly adjusting the dataset image to a first preset size; inputting the adjusted dataset image into a feature extraction network to extract three different feature maps; wherein the feature extraction network comprises: an underlying texture feature extraction network, a low-level contour feature extraction network and a high-level global semantic feature extraction network; the three different feature maps comprise: a texture feature matrix map, a low-level contour feature matrix map and a global semantic feature map; respectively broadcasting and adding the intermediate features in the texture feature matrix map and the intermediate features in the low-level contour feature matrix map, performing feature fusion, and generating a first fused feature map; broadcasting and adding the intermediate features in the first fused feature map and the intermediate features in the global semantic feature map, performing feature fusion, and generating a final fused feature map; inputting the final fused feature map into multiple fully connected layers and activation layers to obtain a quality evaluation prediction score of the dataset image of the image quality to be predicted. The method can accurately judge the distortion type of the image when facing multiple types of distortion images, and give a quality evaluation prediction score of the distorted image.
[0057] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.
[0058] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0060] Figure 1 A flow chart of a no-reference image quality assessment method based on deep learning provided by an embodiment of the present invention;
[0061] Figure 2 A schematic diagram of a no-reference image quality assessment method based on deep learning provided by an embodiment of the present invention;
[0062] Figure 3 A schematic diagram of feature fusion based on the attention mechanism of AFF provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0063] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.
[0064] The embodiment of the present invention provides a method for evaluating image quality without reference based on deep learning. Figure 1 As shown, the following steps are included:
[0065] S1, obtaining a dataset image of image quality to be predicted; uniformly adjusting the dataset images to a first preset size;
[0066] S2, inputting the dataset image adjusted to the first preset size into the feature extraction network, extracting three different feature maps; the feature extraction network includes: an underlying texture feature extraction network, a low-level contour feature extraction network and a high-level global semantic feature extraction network; the three different feature maps include: a texture feature matrix map, a low-level contour feature matrix map and a global semantic feature map;
[0067] S3, respectively broadcasting and adding the intermediate features in the texture feature matrix map and the intermediate features in the low-level contour feature matrix map, performing feature fusion, and generating a first fused feature map;
[0068] S4, broadcasting and adding the intermediate features in the first fused feature map and the intermediate features in the global semantic feature map, performing feature fusion, and generating a final fused feature map;
[0069] S5. Input the final fused feature map into multiple fully connected layers and activation layers to obtain the quality evaluation prediction score of the image in the data set whose image quality is to be predicted.
[0070] The deep learning-based no-reference image quality assessment method provided in this embodiment can accurately determine the distortion type of an image when faced with images of various distortion types, and provide a quality assessment prediction score for the distorted image.
[0071] The following is a detailed description of the no-reference image quality evaluation method:
[0072] After batch loading the dataset images of distorted images, adjust the image size to 224*224*3. Figure 2 As shown in the figure, three different feature maps (texture feature matrix map, low-level contour feature matrix map and global semantic feature map) are extracted from the original 224*224*3 image through the underlying texture feature extraction module, the low-level contour feature extraction module and the high-level global semantic extraction module. After each module performs convolution, pooling and other operations, the size of the three feature maps is uniformly adjusted to 512*7*7.
[0073] Specifically: after batch loading the dataset images, the image size is uniformly adjusted to 224*224*3. Based on the modified VGG19 network, the size of the texture feature matrix map is 256*56*56; the underlying texture feature extraction module passes the matrix in the texture feature matrix map through Conv2d(256,512,1) (256 refers to the number of channels of the input convolution layer, 512 refers to the number of channels of the output convolution layer, and 1 refers to the size of the convolution kernel) and Conv2d(512,512,9,7) (512 refers to the number of channels of the input convolution layer, 512 refers to the number of channels of the output convolution layer, 9 refers to the size of the convolution kernel, and 7 refers to the step size; the convolution kernel and step size are set to change the length and width of the feature map), and then the size of the underlying texture feature map is adjusted to 512*7*7. Among them, Conv2d is a layer of convolution operation in the neural network. From the low-level contour feature extraction module, the size of the low-level contour feature is 1*224*224. After modifying the weight parameters, the size of the low-level contour feature matrix becomes 640*7*7. After passing this result through Conv2d(640,512,1) (640 refers to the number of channels of the input convolution layer, 512 refers to the number of channels of the output convolution layer, and 1 refers to the size of the convolution kernel), the size of the contour matrix is adjusted to 512*7*7. By removing the flattening layer and the fully connected layer in the original resNet50 network, and adding Conv2d(2048,512,1) before the average pooling layer (2048 refers to the number of channels of the input convolution layer, 512 refers to the number of channels of the output convolution layer, and 1 refers to the size of the convolution kernel), a global semantic feature map with a channel number of 512 is finally obtained.
[0074] Since there are complex correlations between the hierarchical features of the three types of feature maps extracted, if a simple connection operation is used, the ability of multi-level features to express image content will be reduced. In order to make more reasonable use of multi-level features and more effectively express image content, a feature fusion method based on the attention mechanism (i.e., AFF) is used to perform multi-level feature fusion.
[0075] Given the intermediate features X, Y∈R in the three feature maps C*H*W , where c represents the number of channels and H*W (height*width) represents the size of the feature map.
[0076] Reference Figure 2 As shown, the intermediate features in the texture feature matrix map and the intermediate features in the low-level contour feature matrix map are broadcasted and added, and feature fusion (AFF) is performed to generate a first fused feature map; the intermediate features in the first fused feature map and the intermediate features in the global semantic feature map are broadcasted and added, and feature fusion (AFF) is performed to generate a final fused feature map; Figure 2 AFF stands for the feature fusion method based on the attention mechanism. AFF refers to fusing feature X and feature Y through three layers of the same attention mechanism to obtain the final fused feature.
[0077] Reference Figure 3 As shown in the figure, AFF is as follows: feature X and feature Y are broadcast weighted, and then global average pooling is performed; then point-by-point convolution is performed respectively; a linear rectification activation function is input through batch normalization operation; after point-by-point convolution, broadcast weighting is performed through batch normalization operation; after broadcast weighting, a sigmoid activation function is input; and the sigmoid activation function is used to perform element-wise intelligent product with feature X and feature Y for auxiliary training. After executing three layers in this way, a fused feature map is generated, and AFF is completed. Specifically, in the first layer, feature X and feature Y are broadcasted and added to obtain a feature matrix, and the global feature context of the feature matrix is calculated (the calculation process includes: global average pooling-->channel reduction-->batch normalization-->linear rectification activation-->channel enlargement) and local feature context (the calculation process includes: point-level convolution-->batch normalization-->linear rectification activation-->point-level convolution-->batch normalization); the global feature context and the local feature context are broadcasted and added, and then sigmoid activation is performed to obtain a weight matrix; the weight matrix is multiplied by feature X, and each element of the 1-weight matrix is multiplied by feature Y, and they are used as the two ends of the next layer. The intermediate process of the second and third layers is the same as the first layer, except that the third layer does not have the final sigmoid activation, and the broadcast addition result is directly used as the final AFF fusion result.
[0078] The feature fusion calculation process based on the attention mechanism is as follows:
[0079] Global average pooling is one of the steps in extracting the global feature context. Both the global feature context and the local feature context are based on the matrix after broadcast addition. The intermediate features in the matrix after broadcast addition are globally average pooled using the following formula:
[0080]
[0081] In the above formula, g(x) represents the global average pooling operation; H*W represents the size of the feature map; x[ :,i,j ] represents the pixel information of all coordinate points (i, j) in the feature map; x represents the intermediate features in the feature map.
[0082] The global feature context of the intermediate features in the matrix after broadcast addition is extracted by the following formula:
[0083] G(x)=b(D i r(b(D r (g(x))))
[0084] In the above formula, G(x) represents the global feature context; g(x) represents the global average pooling operation; D r represents a channel reduction layer; b represents a batch normalization operation; r represents a linear rectification activation function (ReLU); D i represents a channel enlargement layer; x represents the intermediate feature in the feature map.
[0085] The local feature context of the intermediate features in the matrix after broadcast addition is extracted by the following formula:
[0086] L(x)=b(PWconv(r(b(PWConv(x)))),
[0087] In the above formula, L(x) represents the local feature context; b represents the batch normalization operation; r represents the linear rectification activation function; PWConv represents the point-level convolution; x represents the intermediate feature in the feature map.
[0088] The channel weight is calculated by the following formula:
[0089]
[0090] In the above formula, w(x) represents the channel weight; represents a broadcast weighted operation; L(x) represents the local feature context; G(x) represents the global feature context; x represents the intermediate feature in the feature map; sS(.) represents the sigmoid function, which controls the fusion result of the local feature context and the global feature context within the range of [0,1].
[0091] The feature fusion is performed through the following formula:
[0092]
[0093]
[0094]
[0095] In the above formula, Z1 represents the preliminary fusion result of the input intermediate feature X and the input intermediate feature Y; Z2 represents the intermediate processing process of the preliminary fusion result Z1; Z3 represents the final feature fusion result based on the attention mechanism; represents a broadcast weighted operation; represents the smart product between elements; wS(.) represents the channel weight.
[0096] After performing multiple fully connected layer and activation layer operations (specifically, it can be set to 4 fully connected operations and 3 activation operations) on the feature map fused based on the attention mechanism, the quality evaluation prediction score of the current distorted image is finally obtained.
[0097] The extracted three-level feature matrices (bottom-level texture features, low-level contour features, and high-level global semantic features), i.e., texture feature matrix graph, low-level contour feature matrix graph, and global semantic feature graph, are passed through multiple fully connected layers and activation layers to obtain three intermediate prediction scores s1, s2, and s3, which constitute an intermediate score set (s1, s2, s3). This embodiment uses the loss between the intermediate score set and the true label score (MOS), as well as the loss between the quality evaluation prediction score and the true label score for auxiliary training. The overall loss function can be expressed as:
[0098]
[0099] In the above formula, L s represents the sum of the losses between the intermediate score set and the true label score; L e Represents the loss between the quality evaluation prediction score and the true label score; s i represents the intermediate prediction score; s e represents the quality evaluation prediction score; b i Represents the relative weight of the intermediate prediction score in the overall loss function.
[0100] By continuously reducing the loss between the objective prediction score (quality evaluation prediction score) and the subjective label score (true label score), the optimal mapping relationship between the distorted image and the quality evaluation prediction score is obtained. By designing a loss function, that is, continuously reducing the difference between the quality evaluation prediction score and the true label score, the quality evaluation accuracy of this method is improved. In order to improve the robustness of this method to abnormal objective prediction scores, this embodiment selects the Huber loss function to calculate the loss between the intermediate score set and the true label score, and the loss between the quality evaluation prediction score and the true label score:
[0101]
[0102] In the above formula, a represents a preset hyperparameter; s represents the quality evaluation prediction score or the intermediate prediction score; and m represents the true label score.
[0103] b i The calculation process is as follows:
[0104]
[0105] S(.) is the sigmoid function, that is
[0106] If i The smaller the difference between and m, the smaller the weight b corresponding to the current intermediate prediction score layer i It should be smaller.
[0107] During the training process, the optimal parameters are obtained by minimizing the overall loss function, namely:
[0108] (b1,b2,b3)*=argmin(L(s1,s2,s3)),
[0109] argmin() represents the values of b1, b2, and b3 when the overall loss L is minimized.
[0110] Eye visual perception is a hierarchical processing process. The cerebral cortex processes the visual signals input by the human eye from local low-level information to global semantic information in a hierarchical manner. In the reference-free image quality evaluation method based on deep learning, if only a single neural network is used to extract hierarchical information, both the low-level local information and the high-level global information will lack clear physical meaning, which seriously affects the network's analysis of feature hierarchy degradation. By adopting a multi-module feature extraction mode in this embodiment, not only can multi-level features such as texture, edge, and global semantics be obtained, but different feature matrices (feature maps) can also have clear physical meanings.
[0111] In the face of the extracted multi-level features, if a simple connection operation is taken, the ability of the multi-dimensional hierarchical features to express distortion will be weakened, thereby affecting the ability of the no-reference image quality assessment model to evaluate image quality. This embodiment adopts a feature fusion method based on the attention mechanism, which not only reasonably utilizes the multi-level features, but also improves the expression ability of the multi-level features and multi-image content.
[0112] This method not only solves the problem that traditional methods cannot effectively evaluate image quality scores when faced with multiple distortion types, but also makes up for the situation in some deep learning-based methods where the physical meaning of hierarchical features is unclear and the hierarchical feature information is not fully utilized.
[0113] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A no-reference image quality assessment method based on deep learning, characterized in that: The steps include: S1. Obtaining a dataset image of an image quality to be predicted; and uniformly adjusting the dataset image to a first preset size; S2, inputting the dataset image adjusted to a first preset size into a feature extraction network to extract three different types of feature maps; The feature extraction network includes: a bottom-level texture feature extraction network, a low-level contour feature extraction network and a high-level global semantic feature extraction network; the three different types of feature maps include: a texture feature matrix map, a low-level contour feature matrix map and a global semantic feature map; S3, respectively broadcasting and adding the intermediate features in the texture feature matrix map and the intermediate features in the low-level contour feature matrix map to perform feature fusion and generate a first fused feature map; S4, broadcasting and adding the intermediate features in the first fused feature map and the intermediate features in the global semantic feature map to perform feature fusion and generate a final fused feature map; The feature fusion in step S3 and step S4 includes: S11, respectively broadcast and add the intermediate features in the two feature maps for feature fusion; S12, extracting the global feature context and the local feature context of the intermediate feature after broadcast addition; calculating the channel weight according to the global feature context and the local feature context; recalculating the intermediate feature according to the channel weight; replacing the intermediate feature after broadcast addition in step S12 with the recalculated intermediate feature; S13, repeatedly executing steps S11 to S12 for a preset number of times; taking the broadcast addition result of the global feature context and the local feature context extracted by the last execution of step S12 as the feature fusion result of the two feature graphs for feature fusion; S5. Input the final fused feature map into multiple fully connected layers and activation layers to obtain a quality evaluation prediction score of the image in the data set whose image quality is to be predicted.
2. The method for evaluating image quality without reference based on deep learning according to claim 1, characterized in that: Extracting the global feature context of the intermediate features after broadcast addition, including: performing global average pooling on the intermediate features after broadcast addition; The intermediate features after broadcast addition are globally averaged pooled using the following formula: In the above formula, g(x) represents the global average pooling operation; H*W represents the size of the feature map; x[ :,i,j ] represents the pixel information of all coordinate points (i, j) in the feature map; x represents the intermediate features in the feature map.
3. The method for evaluating image quality without reference based on deep learning according to claim 1, characterized in that: The global feature context of the intermediate features after broadcast addition is extracted by the following formula: G(x)=b(D i r(b(D r (g(x)))) In the above formula, G(x) represents the global feature context; g(x) represents the global average pooling operation; D r represents a channel reduction layer; b represents a batch normalization operation; r represents a linear rectification activation function; D i represents a channel enlargement layer; x represents the intermediate feature in the feature map.
4. The method for evaluating image quality without reference based on deep learning according to claim 1, characterized in that: The local feature context of the intermediate feature after broadcast addition is extracted by the following formula: L(x)=b(PWconv(r(b(PWConv(x)))), In the above formula, L(x) represents the local feature context; b represents the batch normalization operation; r represents the linear rectification activation function; PWConv represents the point-level convolution; x represents the intermediate feature in the feature map.
5. The method for evaluating image quality without reference based on deep learning according to claim 1, characterized in that: The channel weight is calculated by the following formula: In the above formula, w(x) represents the channel weight; represents a broadcast weighted operation; L(x) represents the local feature context; G(x) represents the global feature context; x represents the intermediate feature in the feature map; sS(.) represents the sigmoid function.
6. The method for evaluating image quality without reference based on deep learning according to claim 1, characterized in that: Also includes: Input the three different types of feature maps into multiple fully connected layers and activation layers respectively to obtain three intermediate prediction scores to form an intermediate score set; Generate an overall loss function by calculating the loss between the intermediate score set and the true label score, and the loss between the quality assessment prediction score and the true label score; The feature extraction network is auxiliary trained according to the overall loss function.
7. The method for evaluating image quality without reference based on deep learning according to claim 6, characterized in that: The overall loss function is: In the above formula, L s represents the sum of the losses between the intermediate score set and the true label score; L e Represents the loss between the quality evaluation prediction score and the true label score; s i represents the intermediate prediction score; s e represents the quality evaluation prediction score; b i Represents the relative weight of the intermediate prediction score in the overall loss function.
8. The method for evaluating image quality without reference based on deep learning according to claim 7, characterized in that: Calculate L by the following formula s and L e : In the above formula, a represents a preset hyperparameter; s represents the quality evaluation prediction score or the intermediate prediction score; and m represents the true label score.
9. The method for evaluating image quality without reference based on deep learning according to claim 7, characterized in that: Calculate b by the following formula i : in, In the above formula, si represents the intermediate prediction score; m represents the true label score.