A method and system for no-reference image quality assessment based on graph convolutional neural network
By using a graph convolutional neural network-based method, image features are extracted and quality label correlations are learned to predict the distribution of image quality scores. This solves the problem of ignoring the distribution of image quality scores in existing technologies and achieves more efficient image quality evaluation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI JIAOTONG UNIV
- Filing Date
- 2023-10-23
- Publication Date
- 2026-07-21
AI Technical Summary
Existing no-reference image quality assessment methods mainly focus on the average opinion score of image quality, ignoring the importance of the distribution of image quality scores, resulting in poor prediction performance.
A graph convolutional neural network-based approach is adopted. Image features are extracted through a pre-trained ResNet50 network, a graph convolutional neural network classifier is constructed, the correlation between image quality and multiple quality labels is learned, the image quality score distribution is predicted, and a Gaussian-based image quality score distribution is generated.
It improves the predictive performance of no-reference image quality assessment and can effectively evaluate the distribution of image quality scores, outperforming existing methods.
Smart Images
Figure CN117422980B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image quality assessment technology, and more specifically, to a method and system for assessing the quality of no-reference images based on graph convolutional neural networks. Background Technology
[0002] Image quality assessment is a crucial research topic in image processing. Image quality assessment methods can generally be categorized into subjective and objective methods. Subjective image quality assessment determines the quality of each image by inviting a large number of participants to rate it according to different quality labels. For most subjective image quality assessment methods, image quality is calculated as the average opinion score. Some subjective image quality assessment methods also use all subjective opinion scores, i.e., the image quality score distribution, to describe image quality. Objective image quality assessment, on the other hand, objectively predicts image quality by designing algorithms that conform to subjective opinions. Based on the amount of reference image information used, objective image quality assessment methods can be specifically divided into full-reference, semi-reference, and no-reference image quality assessment methods. Due to the difficulty in obtaining reference images in practice, no-reference image quality assessment has attracted considerable attention from researchers.
[0003] Traditional no-reference image quality assessment methods are manual, extracting low-level features from images and using regression methods to predict image quality. In recent years, due to the powerful learning capabilities of Convolutional Neural Networks (CNNs), many researchers have proposed CNN-based methods to predict image quality by extracting high-level features from images. However, these researchers only focus on the average opinion score for predicting image quality, neglecting the importance of the image quality score distribution, which can provide more subjective information about the image. Recently, researchers have proposed several methods for predicting the image quality score distribution. For example, Talebi et al. first proposed the network method NIMA to predict the image quality score distribution. Gao et al. used an alpha-stabilized model to parameterize the distribution of image quality scores. These methods only learn the mapping between image features and the image quality score distribution, which can affect the predictive performance of the methods. Summary of the Invention
[0004] To achieve the research goal of image quality score distribution, the purpose of this invention is to provide a no-reference image quality evaluation method, system, and terminal based on graph convolutional neural networks.
[0005] The first objective of this invention is to provide a no-reference image quality assessment method based on graph convolutional neural networks, comprising:
[0006] The first image feature is extracted using a pre-trained ResNet50 network. Based on the first image feature, the first-order and second-order features of the image are extracted. The first-order and second-order features of the image are then fused to obtain the second image feature.
[0007] Multiple different quality labels are set, and features of these multiple quality labels are extracted. A correlation matrix between the features is constructed. A classifier based on a graph convolutional neural network is constructed based on the correlation matrix. The classifier uses the second image features to learn the probability that the image quality belongs to each label, and obtains the predicted image quality score.
[0008] The average opinion score of the image quality is calculated based on the distribution of the image quality scores, and a Gaussian-based image quality score distribution is generated based on the average opinion score to achieve image quality evaluation.
[0009] Optionally, the first-order and second-order features of the image are extracted based on the first image features, wherein:
[0010] First-order features of the image are extracted using a global average pooling and a global max pooling;
[0011] Use a global second-order pooling to extract second-order features from the image.
[0012] Optionally, the step of setting multiple different quality labels, extracting features from these multiple quality labels, and constructing a correlation matrix between the features includes:
[0013] Set five different quality labels: "Very Poor", "Poor", "Average", "Good", and "Excellent".
[0014] The GloVe model is used to extract features of the labels “Very Poor”, “Poor”, “Average”, “Good”, and “Excellent”, and each label is represented as a real-valued vector in a new vector space model.
[0015] An initial correlation matrix among the five labels is constructed using a data-driven approach, and the final correlation matrix is then obtained.
[0016] Optionally, the step of calculating the mean opinion score of image quality based on the distribution of image quality scores, and generating a Gaussian-based image quality score distribution based on the mean opinion score, includes:
[0017] For an image quality assessment database that only provides the mean opinion score and the standard deviation of the opinion score, generate a Gaussian-based image quality score distribution for the images;
[0018] For an image quality assessment database that only provides average opinion scores, the standard deviation of the opinion scores for image quality is calculated to obtain a Gaussian-based image quality score distribution.
[0019] Optionally, a distribution-based loss function can be used to calculate the distance between the true image quality score distribution and the predicted image quality score distribution;
[0020] A loss function based on mean opinion score is used to calculate the distance between the true image quality mean opinion score and the predicted image quality mean opinion score.
[0021] A second objective of this invention is to provide a referenceless image quality assessment system based on graph convolutional neural networks, comprising:
[0022] Image feature extraction module: Uses a pre-trained ResNet50 network to extract the first image features of the image, extracts the first-order and second-order features of the image based on the first image features, and fuses the first-order and second-order features of the image to obtain the second image features;
[0023] Classification module: Set multiple different quality labels, extract features of these multiple quality labels, construct a correlation matrix between features, construct a classifier based on graph convolutional neural network based on the correlation matrix, and the classifier uses the second image features to learn the probability that the image quality belongs to each label to obtain the predicted image quality score;
[0024] Quality score distribution acquisition module: Calculates the mean opinion score of image quality based on the distribution of image quality scores, and generates a Gaussian-based image quality score distribution based on the mean opinion score to achieve image quality evaluation.
[0025] Optionally, in the image quality score acquisition module, wherein:
[0026] For an image quality assessment database that only provides the mean opinion score and the standard deviation of the opinion score, generate a Gaussian-based image quality score distribution for the images;
[0027] For an image quality assessment database that only provides average opinion scores, the standard deviation of the opinion scores for image quality is calculated to obtain a Gaussian-based image quality score distribution.
[0028] A third objective of this invention is to provide a no-reference image quality assessment terminal based on graph convolutional neural networks, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the program, it performs the aforementioned no-reference image quality assessment method based on graph convolutional neural networks.
[0029] A fourth objective of this invention is to provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned no-reference image quality assessment method based on graph convolutional neural networks.
[0030] Compared with the prior art, the embodiments of the present invention have at least one of the following beneficial effects:
[0031] This invention provides a no-reference image quality assessment method based on graph convolutional neural networks. It not only learns the mapping between image features and distributions, but also constructs a convolutional neural network to learn the mapping between distributions and quality labels. By predicting the distribution of image quality scores, it obtains the average opinion score of image quality. This method can effectively evaluate image quality in the absence of references, realizes the prediction of image quality score distributions, improves prediction performance, and outperforms the current cutting-edge no-reference image quality assessment methods. Attached Figure Description
[0032] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:
[0033] Figure 1 This is a flowchart of a no-reference image quality assessment method based on graph convolutional neural networks according to an embodiment of the present invention.
[0034] Figure 2 This is a flowchart of a no-reference image quality assessment method based on graph convolutional neural networks in a preferred embodiment of the present invention. Detailed Implementation
[0035] The embodiments of the present invention are described in detail below: These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.
[0036] Some existing methods for predicting image quality score distributions only learn the mapping between image features and image quality score distributions. However, according to the research of this invention, in addition to image features, the distribution of image quality scores is also closely related to quality labels, such as "extremely poor," "poor," "average," "good," and "excellent." Participants need to refer to quality labels when rating images, and the image quality score distribution can be viewed as the probability that an image quality belongs to each label. Furthermore, the research found that the diversity of image quality scores is due to the ambiguity of quality labels. That is, the labels are not independent but have a certain degree of correlation. Therefore, the study of image quality score distribution is very important.
[0037] To improve the predictive performance of no-reference image quality assessment and more effectively evaluate image quality, this invention proposes a no-reference image quality assessment method based on graph convolutional neural networks. The specific technical approach is as follows: an image feature extraction network is used to extract first-order and second-order features of the image. Then, a classifier is constructed using a graph convolutional network. This classifier, combined with the extracted image features, learns the probability of image quality extending to each label, i.e., the image quality score distribution. From the learned image quality score distribution, the average opinion score of the image quality can be obtained. This method, based on a convolutional neural network classifier, can learn the dependencies between quality labels, thereby improving the predictive performance of the technique. It can effectively evaluate image quality in the no-reference situation, achieving prediction of the image quality score distribution, and its performance outperforms currently proposed state-of-the-art no-reference image quality assessment methods.
[0038] This invention provides a referenceless image quality assessment method based on graph convolutional neural networks, referring to... Figure 1 This includes the following steps:
[0039] S1: Use a pre-trained ResNet50 network to extract the first image features of the image, extract the first-order and second-order features of the image based on the first image features, and fuse the first-order and second-order features of the image to obtain the second image features;
[0040] In this step, a global average pooling and a global max pooling can be used to extract the first-order features of the image, and a global second-order pooling can be used to extract the second-order features of the image. Finally, the first-order features and the second-order features of the image are fused to obtain the final image features.
[0041] S2: Set multiple different quality labels, extract the features of these multiple quality labels, construct the correlation matrix between the features, construct a classifier based on graph convolutional neural network based on the correlation matrix, and the classifier uses the second image features to learn the probability that the image quality belongs to each label, and obtain the predicted image quality score.
[0042] In this step, five different quality labels can be set, generally "Very Poor", "Poor", "Average", "Good", and "Excellent". These five labels represent different qualities and can be set according to needs. They are five criteria commonly used in subjective experiments for evaluating image quality.
[0043] In this step, GloVe can be used to extract features from the five quality labels "Very Poor", "Poor", "Average", "Good" and "Excellent", construct a correlation matrix between the features, and build a classifier based on a graph convolutional neural network to learn the probability that the image quality belongs to each label through the image features obtained from S1.
[0044] This embodiment not only learns the mapping between image features and distribution, but also learns the mapping between distribution and quality labels through a convolutional neural network, thereby improving prediction performance.
[0045] S3: Calculate the mean opinion score of the image quality based on the image quality score distribution obtained in S2, and generate a Gaussian-based image quality score distribution based on the mean opinion score to achieve image quality evaluation.
[0046] In this step, to obtain a Gaussian-based image quality score distribution for quality assessment, different methods are used for different image quality assessment databases. For example, for image quality assessment databases that only provide the mean opinion score and the standard deviation of the opinion score, a Gaussian-based image quality score distribution is generated. For image quality assessment databases that only provide the mean opinion score, the standard deviation of the opinion score is calculated to obtain the Gaussian-based image quality score distribution. By employing different processing methods, the problem that most image quality assessment databases do not provide a quality distribution, thus making it impossible to use the distribution to train the network, is solved.
[0047] In another embodiment of the invention, the method may further include: designing and integrating a distribution-based loss function and an average opinion score-based loss function. By employing this loss function to train the network using the previously predicted results, the network can update its parameters and improve its prediction performance.
[0048] Corresponding to the same concept as the no-reference image quality assessment method based on graph convolutional neural networks in the above embodiments, another embodiment of the present invention also provides a no-reference image quality assessment system based on graph convolutional neural networks, such as... Figure 2 As shown, the system includes:
[0049] Image feature extraction module: The first image feature is extracted using a pre-trained ResNet50 network. Based on the first image feature, the first-order and second-order features of the image are extracted. The first-order and second-order features of the image are fused to obtain the second image feature. Specifically, a global average pooling and a global max pooling are used to extract the first-order features of the image, and a global second-order pooling is used to extract the second-order features of the image. Finally, the first-order and second-order features of the image are fused to obtain the final image features.
[0050] The classification module based on graph convolutional neural networks (GCNNs) involves setting multiple different quality labels, extracting features from these labels, constructing a correlation matrix between the features, building a classifier based on the GCNN based on the correlation matrix, and using the second image features to learn the probability that the image quality belongs to each label, thus obtaining the predicted image quality score. Specifically, GloVe can be used to extract features from the five quality labels "Very Poor," "Poor," "Average," "Good," and "Excellent," construct a correlation matrix between the features, and build a classifier based on the GCNN to learn the probability that the image quality belongs to each label using the obtained image features.
[0051] The quality score distribution acquisition module calculates the mean opinion score of the image quality from the predicted image quality score distribution, and generates a Gaussian-based image quality score distribution based on the mean opinion score to achieve image quality evaluation.
[0052] In some possible implementations, the system further includes: a loss function: considering that most image quality assessment databases do not provide a true image quality score distribution, a method for generating an image quality score distribution is proposed. During network training of the above three modules, a distribution-based loss function and an average opinion score-based loss function are designed and integrated using the generated or true image quality score distribution and the true average opinion score of the image quality.
[0053] The referenceless image quality evaluation method and system based on graph convolutional neural networks provided in the above embodiments of the present invention can effectively evaluate the overall perceptual experience quality of images.
[0054] Specifically, such as Figure 2 As shown, the details of the methods and systems of the above embodiments of the present invention will be described in conjunction with specific examples, mainly including:
[0055] (1) Image feature extraction module
[0056] like Figure 2 As shown, in this embodiment, the image feature extraction module uses an image feature extraction network (corresponding to...). Figure 2 The image feature extraction network module (in the image processing module) is used, specifically a ResNet50 network pre-trained with ImagNet, where the last fully connected layer and pooling layer are removed. The cropped image resolution is 448×448. The first extracted image feature is denoted as F.
[0057] Then, the channel attention module is used to highlight the important feature channels of F:
[0058] F′=CAM(F)
[0059] F′ is the channel-weighted feature, i.e., the first image feature.
[0060] The first-order features of the first image feature F′ are extracted using global average pooling and global max pooling, and the second-order features of the first image feature F′ are extracted using global second-order pooling. Finally, by concatenating the first-order and second-order features of the first image feature F′ along the channels, the image features extracted by the image feature extraction network, i.e., the second image features, can be obtained.
[0061] The first-order and second-order features mentioned above are common techniques. In this embodiment, the first-order feature refers to the statistical feature, specifically the average and maximum value of the feature; the second-order feature is also a statistical feature, specifically the second-order mean square statistical feature.
[0062] (2) Classification module based on graph convolutional network
[0063] like Figure 2 As shown, in this embodiment, the construction process of the classification module based on graph convolutional networks consists of three steps:
[0064] - Label feature extraction
[0065] The GloVe model uses five quality labels: "Very Poor", "Poor", "Average", "Good", and "Excellent".
[0066] -Correlation Matrix
[0067] An initial correlation matrix among the five labels was constructed using a data-driven approach. Optionally, within a database, the number of times two labels are used simultaneously to describe the same image is first calculated: M = [M ij The conditional probability matrix can be written as P = [P ij ], where P ij =Mi j / Mi is the probability that label j appears when label i appears.
[0068] P is binarized, and to prevent the binary correlation matrix from becoming too smooth, the binarized P is reweighted to obtain the final correlation matrix.
[0069] -Graph Convolutional Neural Network
[0070] In this embodiment, the graph convolutional neural network consists of two graph convolutional layers and a Leaky ReLU layer. It can learn the correlation between nodes (labels) by propagating information between nodes, thereby turning the label features into an interdependent classifier used to extract image features and predict the image quality score distribution. N is the number of tags.
[0071] (3) Prediction of average opinion score
[0072] Average opinion score The calculation method is as follows:
[0073]
[0074] s i p represents the score assigned to the i-th label. i The image quality score distribution assigned to the i-th label.
[0075] In addition, to obtain better performance evaluation, a network training module is also included. The following two loss functions can be used when training the network:
[0076] (1) Distribution-based loss function
[0077] For an image quality assessment database that provides an image quality score distribution, the distribution-based loss function used is the bulldozer distance.
[0078] For an image quality assessment database that provides mean opinion score and standard deviation of opinion score for image quality, a Gaussian-based image quality score distribution for each image is generated using the mean opinion score and standard deviation of opinion score, and then bulldozer distance is used as the loss function.
[0079] For an image quality assessment database that only provides average opinion scores, the opinion score standard deviation for each image quality is obtained using the opinion score standard deviation calculation method, thereby obtaining a Gaussian-based image quality score distribution for each image quality, and then the bulldozer distance is used as the loss function.
[0080] (2) Loss function based on average opinion score
[0081] The loss function with mean normalization is used as the loss function based on the average opinion score in this embodiment.
[0082] The proposed graph convolutional neural network-based no-reference image quality assessment method trains the network by combining a distribution-based loss function and a loss function based on average opinion scores.
[0083] This embodiment uses the aforementioned loss function to train the network, thereby enabling the network to output more accurate results and improving the predictive performance of the method. The image quality assessment databases described above can all utilize existing technologies or be separately constructed databases.
[0084] Implementation results:
[0085] To verify the effectiveness of the no-reference image quality assessment method based on graph convolutional neural networks provided in the above embodiments of the present invention, experimental tests were conducted on the SJTU IQSD, KonIQ-10K, CSIQ, LIVE MD, LIVEChallenge, CID2013, SPAQ, and VCLFER databases.
[0086] In the experiment, the database was randomly segmented. Specifically, 80% of the images in the database were segmented into the training set, while the remaining images were divided into the test set. For databases with images exhibiting overall distortion, they were segmented into training and test sets based on reference images. To avoid bias caused by the randomness of the training and test set segmentation, the above segmentation process was repeated 10 times, and the average results were reported.
[0087] The proposed method was compared with several state-of-the-art image quality assessment methods for predicting image quality: BRISQUE, NIQE, BMPRI, Kang, (Wa)DIQaM-N, SGDNet, DBCNN, HyperIQA (Su et al. 2020), DACNN, GraphIQA, NIMA, and Gao. Three evaluation criteria were selected to measure the performance of the evaluation methods: Pearson linear correlation coefficients (PLCC), Spearman rank order correlation coefficients (SRCC), and root mean squared error (RMSE). Performance test results are shown in Tables 1-3. Table 1 shows the performance of the proposed method and competing methods for predicting the average opinion score of image quality on the SJTU IQSD and KonIQ-10K databases. The table shows that the proposed method has the best prediction performance, indicating that obtaining the average opinion score of image quality by predicting the distribution of image quality scores is effective. Table 2 shows the performance of the proposed method and competing methods for predicting the average opinion score of image quality on the CSIQ, LIVE MD, LIVE Challenge, and CID2013 databases. As can be seen from the table, the proposed method also has the best predictive performance. Table 3 shows the performance of the proposed method and some competing methods for predicting average opinion scores for image quality on the SPAQ and VCLFER databases. From the table, the proposed method outperforms these state-of-the-art image quality assessment methods in terms of predictive performance.
[0088] Table 1
[0089]
[0090] Table 2
[0091]
[0092] Table 3
[0093]
[0094] This invention provides a no-reference image quality evaluation method based on graph convolutional neural networks, which can effectively evaluate image quality.
[0095] Based on the same concept described above, in another embodiment of the present invention, a no-reference image quality assessment terminal based on graph convolutional neural networks is also provided, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the program, it is used to execute the no-reference image quality assessment method based on graph convolutional neural networks.
[0096] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs, functional modules, etc. that implement the above methods), computer instructions, etc., and the aforementioned computer programs, computer instructions, etc., can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.
[0097] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.
[0098] A processor is used to execute a computer program stored in memory to implement the various steps of the methods involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method embodiments.
[0099] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.
[0100] In this embodiment of the invention, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements a no-reference image quality assessment method based on a graph convolutional neural network in any of the above embodiments.
[0101] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0102] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0103] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0104] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0105] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.
[0106] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
Claims
1. A no-reference image quality assessment method based on graph convolutional neural networks, characterized in that, include: The first image feature is extracted using a pre-trained ResNet50 network. Based on the first image feature, the first-order and second-order features of the image are extracted. The first-order and second-order features of the image are then fused to obtain the second image feature. Multiple different quality labels are set, and features of these multiple quality labels are extracted. A correlation matrix between the features is constructed. A classifier based on a graph convolutional neural network is constructed based on the correlation matrix. The classifier uses the second image features to learn the probability that the image quality belongs to each label, and obtains the predicted image quality score. The average opinion score of the image quality is calculated based on the distribution of the image quality scores, and a Gaussian-based image quality score distribution is generated based on the average opinion score to achieve image quality evaluation.
2. The no-reference image quality assessment method based on graph convolutional neural networks according to claim 1, characterized in that, The first-order and second-order features of the image are extracted based on the first image features, wherein: First-order features of the image are extracted using a global average pooling and a global max pooling; Use a global second-order pooling to extract second-order features from the image.
3. The no-reference image quality assessment method based on graph convolutional neural networks according to claim 1, characterized in that, The process of setting multiple different quality labels, extracting features from these quality labels, and constructing a correlation matrix between the features includes: Set five different quality labels: "Very Poor", "Poor", "Average", "Good", and "Excellent"; The GloVe model is used to extract features of the labels "Very Poor", "Poor", "Average", "Good" and "Excellent", and each label is represented as a real-valued vector in a new vector space model. An initial correlation matrix among the five labels is constructed using a data-driven approach, and the final correlation matrix is then obtained.
4. The no-reference image quality assessment method based on graph convolutional neural networks according to claim 1, characterized in that, The step of calculating the mean opinion score of image quality based on the distribution of image quality scores, and generating a Gaussian-based image quality score distribution based on the mean opinion score, includes: For an image quality assessment database that only provides the mean opinion score and the standard deviation of the opinion score, generate a Gaussian-based image quality score distribution for the images; For an image quality assessment database that only provides average opinion scores, the standard deviation of the opinion scores for image quality is calculated to obtain a Gaussian-based image quality score distribution.
5. The no-reference image quality assessment method based on graph convolutional neural networks according to claim 1, characterized in that, Also includes: A distribution-based loss function and an average opinion score-based loss function were designed and integrated for training the network.
6. The no-reference image quality assessment method based on graph convolutional neural networks according to claim 5, characterized in that, A distribution-based loss function is used to calculate the distance between the true image quality score distribution and the predicted image quality score distribution; A loss function based on mean opinion score is used to calculate the distance between the true image quality mean opinion score and the predicted image quality mean opinion score.
7. A no-reference image quality assessment system based on graph convolutional neural networks, comprising: Image feature extraction module: Uses a pre-trained ResNet50 network to extract the first image features of the image, extracts the first-order and second-order features of the image based on the first image features, and fuses the first-order and second-order features of the image to obtain the second image features; Classification module: Set multiple different quality labels, extract features of these multiple quality labels, construct a correlation matrix between features, construct a classifier based on graph convolutional neural network based on the correlation matrix, and the classifier uses the second image features to learn the probability that the image quality belongs to each label to obtain the predicted image quality score; Quality score distribution acquisition module: Calculates the mean opinion score of image quality based on the distribution of image quality scores, and generates a Gaussian-based image quality score distribution based on the mean opinion score to achieve image quality evaluation.
8. The referenceless image quality assessment system based on graph convolutional neural networks according to claim 7, characterized in that, The image quality score acquisition module, wherein: For an image quality assessment database that only provides the mean opinion score and the standard deviation of the opinion score, generate a Gaussian-based image quality score distribution for the images; For an image quality assessment database that only provides average opinion scores, the standard deviation of the opinion scores for image quality is calculated to obtain a Gaussian-based image quality score distribution.
9. A referenceless image quality assessment terminal based on graph convolutional neural networks, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor executes the program, it is used to perform the method described in any one of claims 1-6.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-6.