A no-reference image quality evaluation method based on graph contrast
By constructing a dataset of combined distorted images and employing a graph contrastive learning method, the problems of weak generalization ability and model fitting in real-world scenarios of no-reference image quality assessment methods are solved, resulting in more accurate image quality assessment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHONGQING UNIV OF POSTS & TELECOMM
- Filing Date
- 2023-05-23
- Publication Date
- 2026-06-02
AI Technical Summary
Existing deep learning-based no-reference image quality assessment methods rely on labeled datasets during training, resulting in weak generalization ability and the models are prone to overfitting or underfitting, making it difficult to accurately extract distortion information in real-world scenes.
A dataset of combined distorted image pairs is constructed. Using graph contrastive learning, a graph representation generation model, a positive and negative distortion contrast sample generation model, and a contrastive learning encoder model are used to generate a distortion relationship graph structure. Contrast loss and prediction loss are calculated, and the model is optimized to improve generalization ability.
This improves the accuracy of the model in evaluating image quality in real-world scenarios, avoids overfitting or underfitting, and achieves more accurate extraction and evaluation of distortion information.
Smart Images
Figure CN116468722B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of no-reference image quality assessment, and in particular relates to a no-reference image quality assessment method based on graph comparison. Background Technology
[0002] With the increasing demand from applications in entertainment, communication, security, surveillance, and medical imaging, images are playing an increasingly important role in people's daily lives. Hundreds of billions of photos are uploaded and shared annually on social media platforms such as WeChat, QQ, and Douyin, while streaming services like Tencent Video, iQiyi, and Youku account for 60% of all downstream internet traffic. However, during the acquisition, processing, and transmission processes, images inevitably suffer distortion and degradation, reducing the accuracy of image analysis and understanding and severely impacting the viewing experience. Faced with massive amounts of image data, it is necessary to design image quality evaluation methods to accurately quantify image quality, identify the causes of image degradation, and provide guidance for improving image quality.
[0003] Full-reference image quality assessment methods evaluate the quality of distorted images by quantifying the similarity between the distorted image and its reference image. Semi-reference image quality assessment methods require partial information from the reference image and fall between full-reference and no-reference methods. No-reference image quality assessment methods do not require a reference image, only the image to be evaluated and its subjective quality score label. This allows for more flexible and real-time image quality evaluation, and since reference images for most real images are unavailable, full-reference and semi-reference methods cannot be used to assess the quality of these unreferenced real images. Furthermore, given the powerful fitting capabilities of deep neural networks, deep learning-based no-reference image quality assessment methods often exhibit better performance. In conclusion, deep learning-based no-reference image quality assessment methods have become an important technology for image quality control on social media platforms, possessing significant research and application value.
[0004] With the continuous efforts of researchers and industry practitioners, a large number of excellent deep learning-based no-reference image quality assessment methods have been proposed. However, these methods still have some problems in practical applications, which can be summarized as follows:
[0005] First, previous deep learning-based no-reference image quality assessment methods heavily relied on supervised training using labeled datasets. However, existing labeled datasets have several problems: First, some existing labeled datasets are constructed by artificially applied simulated distortions, which often fail to accurately depict the complex real-world distortions. Second, compared to the hundreds of millions or even billions of truly distorted images, existing labeled datasets of truly distorted scene images are still relatively small. Training models using these datasets will result in models with weak generalization ability when processing images in truly distorted scenes.
[0006] Secondly, previous deep learning-based no-reference image evaluation models were often pre-trained on the ImageNet dataset to obtain initial parameter settings, facilitating subsequent fine-tuning or further optimization. However, the ImageNet dataset was originally designed for classification tasks, and pre-training on it is difficult to fully adapt to image quality evaluation tasks. This can lead to the model learning features irrelevant to the image quality evaluation task, resulting in overfitting or underfitting issues.
[0007] Thirdly, existing unsupervised image quality assessment methods often only use classification tasks based on distortion type and level to obtain auxiliary information for image quality assessment. However, the feature information obtained solely from distortion type and level classification tasks is insufficient to describe complex, realistically distorted images in real-world scenes. This makes it difficult for the model to accurately extract distortion information when processing images in real-world scenarios. Summary of the Invention
[0008] To address the problems existing in the background art, this invention provides a referenceless image quality assessment method based on graph comparison, the specific technical solution of which is as follows:
[0009] S1: Obtain the synthetic distorted image dataset and the real distorted image dataset, and construct a combined distorted image pair dataset based on the synthetic distorted image dataset and the real distorted image dataset;
[0010] Preferably, the construction of the combined distorted image pair dataset includes:
[0011] Different distortion levels under the same distortion type in the synthetic distortion dataset are considered as different distortion types; images in the synthetic distortion image dataset and the real distortion image dataset are classified according to image content to obtain category labels; images with the same category labels are selected from the synthetic distortion dataset and the real distortion image dataset according to a preset ratio and quantity to form combined distortion image pairs, thereby constructing a combined distortion image pair dataset.
[0012] S2: Create a graph contrast model; the graph contrast model includes: a graph representation generation model, a positive and negative distortion contrast sample generation model, a contrastive learning encoder model, and a distortion type prediction model;
[0013] S3: Generate a second distorted relation graph structure from the input graph representation generation model by combining distorted images;
[0014] Preferably, the graph representation generation model includes: a ResNet50 network, a global average pooling layer, and a graph convolutional neural network;
[0015] The specific steps for generating the second distorted relation graph structure include:
[0016] The combined distorted image is input to the ResNet50 network to extract features, resulting in M first intermediate feature vectors.
[0017] The M first intermediate feature vectors are input into the global average pooling layer to obtain M second intermediate feature vectors and construct the intermediate feature matrix.
[0018] Multiplying the intermediate feature matrix and its transpose yields the similarity matrix between the second intermediate feature vectors;
[0019] Calculate the adjacency matrix based on the similarity between the second intermediate feature vectors;
[0020] Each second intermediate feature vector is used as a node, and the similarity between the second intermediate feature vectors is used as an edge to create a first distorted relation graph structure. The first distorted relation graph structure is then input into a graph convolutional neural network to update the nodes and generate a second distorted relation graph structure.
[0021] S4: Input the second distortion graph structure into the positive and negative distortion comparison sample generation model to generate positive and negative distortion graph structures;
[0022] Preferably, the positive and negative distortion contrast sample generation model includes: a first graph convolutional neural network, a second graph convolutional neural network, a first Gumbel-softmax and a second Gumbel-softmax;
[0023] The generation of the positive example distortion graph structure and the negative example distortion graph structure includes: inputting the second distortion graph structure into the first Gumbel-softmax and the second Gumbel-softmax respectively;
[0024] The first Gumbel-softmax and the second Gumbel-softmax respectively sample each node in the second distorted relation graph structure to obtain n first sampling probability distribution vectors and n second sampling probability distribution vectors;
[0025] Combine the n first sampling probability distribution vectors into a first sampling matrix, and combine the n second sampling probability distribution vectors into a second sampling matrix;
[0026] The first sampling matrix is used to update the nodes in the second distorted relation graph structure to obtain the amplified first intermediate graph structure.
[0027] The amplified first intermediate graph structure is input into the first graph convolutional neural network to update the nodes and obtain the positive example distortion graph structure;
[0028] The nodes in the second distorted relation graph structure are updated using the second sampling matrix to obtain the amplified second intermediate graph structure.
[0029] The amplified second intermediate graph structure is input into the second graph convolutional neural network to update the nodes, resulting in a negative example distorted relational graph structure.
[0030] S5: Input the positive example distortion graph structure and the negative example distortion graph structure into the contrastive learning encoder model to extract features and obtain positive sample feature vectors and negative sample feature vectors, respectively.
[0031] Preferably, the contrastive learning encoder model includes: a VGG network and an MLP;
[0032] The process of obtaining positive and negative sample feature vectors includes: inputting the positive and negative sample distortion graph structures into a VGG network for feature extraction, and using MLP to reduce the dimensionality of the features to obtain positive and negative sample feature vectors.
[0033] S6: Calculate the contrastive loss function based on the feature vectors of positive and negative samples;
[0034] Preferably, the contrastive loss function includes:
[0035]
[0036]
[0037]
[0038] Where B represents the batch value; 𝒯 is the temperature coefficient; exp() represents the exponential operation; and These represent the feature vector of positive samples and the feature quantity of negative samples, respectively. express 2-norm; This represents the contrast loss between a pair of positive and negative samples; This represents the total comparative loss.
[0039] S7: Separate the embedded nodes of the positive and negative example distortion graph structures to obtain the feature information of each node, input the feature information of the nodes into the distortion type prediction model to predict the category of image distortion and calculate the prediction loss function;
[0040] Preferably, the distortion type prediction model includes: a first fully connected layer, a second fully connected layer, and a softmax layer connected in sequence;
[0041] The predicted node category includes: passing the node's feature information through two fully connected layers; finally, using the Softmax function for classification prediction, with pseudo-labels used to indicate its distortion type, and custom labels used for the actual distortion image.
[0042] Preferably, the prediction loss function includes:
[0043]
[0044] Where N represents the number of samples and K represents the number of categories. It is the true label of sample i. It is the probability that sample i belongs to category j.
[0045] S8: Use the backpropagation mechanism to alternately optimize the contrastive loss function and the predictive loss function until the contrastive loss function and the predictive loss function are less than the set threshold;
[0046] S9: Input the feature information of the separated nodes into the trained quality evaluation model and output the corresponding quality evaluation score.
[0047] Preferably, the quality evaluation model consists of several fully connected layers.
[0048] The present invention has at least the following beneficial effects
[0049] This disclosure constructs a distortion relationship image pair dataset by combining real distortion datasets and synthetic distortion datasets. The constructed distortion relationship image pair dataset can help the model understand and identify distortion, especially distortion in images in real-world scenes, thereby improving the model's generalization ability.
[0050] This disclosure learns feature information, namely distortion relation feature information, suitable for image quality assessment tasks by pre-training on a constructed dataset of distorted relation image pairs. Effective distortion relation feature information enables rapid optimization or fine-tuning of model training, thereby avoiding overfitting or underfitting problems.
[0051] This disclosure employs an unsupervised approach using graph contrastive learning to design the model, and designs a core task of identifying and understanding distortion relationships to acquire feature information relevant to image quality assessment. Through this task, the model can accurately extract distortion information from images in real-world distortion scenarios, further improving the model's generalization ability and the accuracy of the evaluation results. Attached Figure Description
[0052] Figure 1 This is a flowchart of the method of the present invention;
[0053] Figure 2 This is a schematic diagram of the process for calculating the contrast loss according to the present invention;
[0054] Figure 3 This is a schematic diagram of the graph representation generation model of the present invention;
[0055] Figure 4 This is a schematic diagram of the positive and negative distortion comparison sample generation model of the present invention;
[0056] Figure 5 This is a schematic diagram of the distortion type prediction model of the present invention. Detailed Implementation
[0057] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0058] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0059] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.
[0060] Please see Figure 1 This invention provides a referenceless image quality assessment method based on graph comparison, comprising:
[0061] S1: Obtain the synthetic distorted image dataset and the real distorted image dataset, and construct a combined distorted image pair dataset based on the synthetic distorted image dataset and the real distorted image dataset;
[0062] Preferably, the construction of the combined distorted image pair dataset includes:
[0063] Different distortion levels under the same distortion type in the synthetic distortion dataset are considered as different distortion types; images in the synthetic distortion image dataset and the real distortion image dataset are classified according to image content to obtain category labels; images with the same category labels are selected from the synthetic distortion dataset and the real distortion image dataset according to a preset ratio and quantity to form combined distortion image pairs, thereby constructing a combined distortion image pair dataset.
[0064] This invention combines synthetic distortion datasets (TID2013, KADID-10K) and real distortion datasets (KonIQ-10K). Different distortion levels of the same distortion type within the synthetic distortion dataset are considered different non-distortion types, thus expanding to a new set of synthetic distortion type images. Secondly, the new set of synthetic distortion type images is categorized by image content to reduce distortion relationship errors caused by different image content. The categories are mainly architecture, transportation, seascape, plants, people, and animals. Subsequently, images with the same image content category as those from the previous step are matched in the real distortion dataset. Finally, under the condition of the same image content category, a certain number of images are randomly selected from both the expanded new synthetic distortion dataset and the real distortion dataset to form a distortion relationship pair dataset.
[0065] To achieve the understanding and recognition of distortion relationships, this disclosure describes a composite distorted image pair dataset composed of synthetic distorted datasets (TID2013, KADID-10K) and a real distorted dataset (KonIQ-10K). The composition of the synthetic distorted data (TID2013, KADID-10K) is shown in the table below:
[0066] Synthetic Distortion Dataset Number of reference figures Distortion type Distortion level TID2013 25 24 5 KADID-10K 81 25 5
[0067] The real distortion dataset (KonIQ-10K) has no reference images and consists of 10,073 real distortion images obtained from real scenes.
[0068] The combination methods are as follows:
[0069] First, based on the assumption that different distortion levels of the same distortion type in a synthetic distortion dataset are considered different non-distortion types, a new set of synthetic distortion types is created. Specifically, in the TID2013 dataset, each reference image originally had 25 distortion types; this becomes 24 * 5 = 120 distortion types. Similarly, in the KADID-10K dataset, each reference image has 25 * 5 = 125 distortion types.
[0070] Secondly, by combining the hypothesized expanded TID2013 dataset and the KADID-10K dataset, the resulting image set is as follows:
[0071] Synthetic Distortion Dataset Number of reference figures <![CDATA[Distortion type (N Syn )]]> TID2013 and KADID-10 combined image collection 25 + 81 = 106 120 / 125
[0072] Next, to reduce the error caused by the distortion due to different image content, the reference images are classified according to their content category in the combined image set. These are mainly divided into categories such as architecture, transportation, seascape, plants, people, and animals.
[0073] Finally, in the same reference figure N Syn The following is a random selection of a certain number n1 of synthetic distorted images, i.e. Meanwhile, while ensuring the same image content category, in a stack of N truly distorted images... Auth A certain number of real distortion images are randomly selected from n², i.e. The two selected image types are combined together, i.e., N. i = + N i This is a combined distorted image pair in the desired distorted image pair dataset. Following the above method, several distorted image pairs are generated, and combining these image pairs constitutes the desired combined distorted image pair dataset.
[0074] S2: Create a graph contrast model; the graph contrast model includes: a graph representation generation model, a positive and negative distortion contrast sample generation model, a contrastive learning encoder model, and a distortion type prediction model;
[0075] The graph representation generation model takes as input a pair of combined distorted images (one or more image pairs derived from a constructed dataset of combined distorted image pairs) and outputs as a graph structure representation of a pair of combined distorted images, which represents the distortion relationships between images within the pair.
[0076] Please see Figure 3 S3: Generate a second distorted relation graph structure by combining the distorted image with the input graph representation generation model;
[0077] Preferably, the graph representation generation model includes: a ResNet50 network, a global average pooling layer, and a graph convolutional neural network;
[0078] The specific steps for generating the second distorted relation graph structure include:
[0079] The combined distorted image is input to the ResNet50 network to extract features, resulting in M first intermediate feature vectors.
[0080] The M first intermediate feature vectors are input into the global average pooling layer to obtain M second intermediate feature vectors and construct the intermediate feature matrix.
[0081] Multiplying the intermediate feature matrix and its transpose yields the similarity matrix between the second intermediate feature vectors;
[0082] Calculate the adjacency matrix based on the similarity between the second intermediate feature vectors;
[0083] Each second intermediate feature vector is used as a node, and the similarity between the second intermediate feature vectors is used as an edge to create a first distorted relation graph structure. The first distorted relation graph structure is then input into a graph convolutional neural network to update the nodes and generate a second distorted relation graph structure.
[0084] First, select B combined-distortion image pairs (each combined-distortion image pair has n = n1 + n2 images) from the combined-distortion image pair dataset as a Batach as input to the graph representation generation model (the Batch dimension is ignored below for simplicity). After random cropping, the images in the combined-distortion image pairs are cropped to PatchSize * PatchSize size.
[0085] Secondly, the cropped and combined distorted image pairs are used for feature extraction via a ResNet50 network; the second intermediate feature vectors are then stacked and combined using global average pooling (GAP) to form an intermediate feature matrix X. Among them, each Treat it as a node in a graph structure; and copy the generated feature matrix and transpose it to obtain the transposed intermediate feature matrix X. T= The original feature matrix and the transposed feature matrix are multiplied to generate a similarity matrix S, which is then thresholded.
[0086] Then, the adjacency matrix is calculated and normalized, as shown in the following formula:
[0087]
[0088] in, To add self-loop I S The adjacency matrix is then used to ensure that each node in the graph structure has at least one neighbor node; Let A be the degree matrix, a diagonal matrix representing the sum of the number of edges connecting a node to another node in the graph structure; A is the normalized adjacency matrix.
[0089] Each second intermediate feature vector is used as a node, and the adjacency relationship between the second intermediate feature vectors is used as an edge to create a first distorted relation graph structure. The first distorted relation graph structure is then input into a graph convolutional neural network for graph structure representation to generate a second distorted relation graph structure.
[0090] Next, the initial feature map The data is fed into a Graph Convolutional Network (GCN) to perform graph convolution, thereby generating a graph structure representation. The formula is as follows:
[0091]
[0092] in, GCN represents the generated graph structure representation; GCN represents the graph convolution process. This represents the node embedding process, where A is the adjacency matrix, X is the feature matrix, and W is the number of nodes. θ This represents the parameters of the graph convolutional network.
[0093] Please see Figure 2 and Figure 4 S4: Input the second distortion graph structure into the positive and negative distortion comparison sample generation model to generate positive and negative distortion graph structures;
[0094] Preferably, the positive and negative distortion contrast sample generation model includes: a first graph convolutional neural network, a second graph convolutional neural network, a first Gumbel-softmax and a second Gumbel-softmax;
[0095] The generation of the positive example distortion graph structure and the negative example distortion graph structure includes: inputting the second distortion graph structure into the first Gumbel-softmax and the second Gumbel-softmax respectively;
[0096] The first Gumbel-softmax and the second Gumbel-softmax respectively sample each node in the second distorted relation graph structure to obtain n first sampling probability distribution vectors and n second sampling probability distribution vectors;
[0097] Combine the n first sampling probability distribution vectors into a first sampling matrix, and combine the n second sampling probability distribution vectors into a second sampling matrix;
[0098] The first sampling matrix is used to update the nodes in the second distorted relation graph structure to obtain the amplified first intermediate graph structure.
[0099] The amplified first intermediate graph structure is input into the first graph convolutional neural network to update the nodes and obtain the positive example distortion graph structure;
[0100] The nodes in the second distorted relation graph structure are updated using the second sampling matrix to obtain the amplified second intermediate graph structure.
[0101] The amplified second intermediate graph structure is input into the second graph convolutional neural network to update the nodes, resulting in a negative example distorted relational graph structure.
[0102] An example, a specific implementation of generating positive example distorted relational graph structures and negative example distorted relational graph structures, firstly, using Gumbel-softmax to embed each node in the graph structure representation (the eigenvector in the feature matrix X) Sampling is performed (a probability distribution is calculated) to obtain a continuous sampling probability distribution vector. Combine the sampling probability distribution vectors into a sampling matrix. With graph representation of structure node matrix By multiplying the corresponding nodes, we obtain the expanded node matrix. The implementation method is shown in the following formula:
[0103]
[0104] The nodes output from the previous step are then embedded and fed into a three-layer graph convolutional network. The process is as follows:
[0105]
[0106] in, This represents the graph structure after further graph convolution to aggregate node information; This describes the node embedding process in a graph structure; the GCN operation is the same as above.
[0107] Finally, the graph structure generated from a distorted relation image pair is processed by two distorted relation graph representation generators (Distortion Relation Generator 1 and Distortion Relation Generator 2) to generate two augmented node embeddings. .in .
[0108] S5: Input the positive example distortion graph structure and the negative example distortion graph structure into the contrastive learning encoder model to extract features and obtain positive sample feature vectors and negative sample feature vectors, respectively.
[0109] Preferably, the contrastive learning encoder model includes: a VGG network and an MLP;
[0110] The process of obtaining positive and negative sample feature vectors includes: inputting the positive and negative sample feature maps into a VGG network for feature extraction, and using MLP to reduce the dimensionality of the features to obtain the positive and negative sample feature vectors.
[0111] The feature matrix in the graph structure is implemented using a VGG network. Convolutional operations are performed, and feature dimensionality reduction is achieved through MLP to obtain the Projector Head. The process can be represented as follows:
[0112]
[0113] S6: Calculate the contrastive loss function based on the feature vectors of positive and negative samples;
[0114] Preferably, the contrastive loss function includes:
[0115]
[0116]
[0117]
[0118] Where B represents the batch value; 𝒯 is the temperature coefficient; exp() represents the exponential operation; and These represent the feature vector of positive samples and the feature quantity of negative samples, respectively. express 2-norm; This represents the contrast loss between a pair of positive and negative samples; This represents the total comparative loss.
[0119] S7: Separate the nodes of the positive and negative example distortion graph structures to obtain the graph embedding information of each node. Input the graph embedding information of the nodes into the distortion type prediction model to predict the image category and calculate the prediction loss function.
[0120] Please see Figure 5 Preferably, the distortion type prediction model includes: a first fully connected layer, a second fully connected layer, and a softmax layer connected in sequence;
[0121] The predicted node category includes: passing the node's graph embedding information through two fully connected layers; finally, using the Softmax function for classification prediction, the pseudo-label used is its distortion type label, and the distortion type of the real distortion image adopts a custom label.
[0122] Preferably, the prediction loss function includes:
[0123]
[0124] Where, N B N represents the total number of images in a batch, i.e., N = B * N; K represents the number of categories (a configurable hyperparameter). It is the true label of sample i. It is the probability that sample i belongs to category j.
[0125] S8: Use the backpropagation mechanism to alternately optimize the contrastive loss function and the prediction loss function until the contrastive loss function and the prediction loss function are less than a set threshold; the alternate optimization means optimizing the contrastive loss function and the prediction loss function once in any two adjacent iterations.
[0126] S9: Input the graph embedding information of the nodes into the trained quality assessment model and output the corresponding quality assessment score.
[0127] Preferably, the quality assessment model consists of several fully connected layers, and the quality assessment model is optimized on existing labeled datasets, including CSIQ, TID2013, LIVE, KonIQ-10K, etc.
[0128] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A no-reference image quality assessment method based on graph contrast, characterized in that, include: S1: Obtain the synthetic distorted image dataset and the real distorted image dataset, and construct a combined distorted image pair dataset based on the synthetic distorted image dataset and the real distorted image dataset; The constructed dataset of combined distorted image pairs includes: Different distortion levels under the same distortion type in the synthetic distortion dataset are regarded as different distortion types; images in the synthetic distortion image dataset and the real distortion image dataset are classified according to image content to obtain category labels; images with the same category labels are selected from the synthetic distortion dataset and the real distortion image dataset according to a preset ratio and quantity to form combined distortion image pairs, and then a combined distortion image pair dataset is constructed. S2: Create a graph contrast model; the graph contrast model includes: a graph representation generation model, a positive and negative distortion contrast sample generation model, a contrastive learning encoder model, and a distortion type prediction model; S3: Generate a second distorted relation graph structure from the input graph representation generation model by combining distorted images; S4: Input the second distortion graph structure into the positive and negative distortion comparison sample generation model to generate positive and negative distortion graph structures; S5: Input the positive example distortion graph structure and the negative example distortion graph structure into the contrastive learning encoder model to extract features and obtain positive sample feature vectors and negative sample feature vectors, respectively. S6: Calculate the contrastive loss function based on the feature vectors of positive and negative samples; S7: Separate the embedded nodes of the positive and negative example distortion graph structures to obtain the feature information of each node, input the feature information of the nodes into the distortion type prediction model to predict the category of image distortion and calculate the prediction loss function; S8: Use the backpropagation mechanism to alternately optimize the contrastive loss function and the predictive loss function until the contrastive loss function and the predictive loss function are less than the set threshold; S9: Input the feature information of the separated nodes into the trained quality evaluation model and output the corresponding quality evaluation score.
2. The referenceless image quality assessment method based on graph comparison according to claim 1, characterized in that, The graph representation generation model includes: a ResNet50 network, a global average pooling layer, and a graph convolutional neural network; The specific steps for generating the second distorted relation graph structure include: The combined distorted image is input to the ResNet50 network to extract features, resulting in M first intermediate feature vectors. The M first intermediate feature vectors are input into the global average pooling layer to obtain M second intermediate feature vectors and construct the intermediate feature matrix. Multiplying the intermediate feature matrix and its transpose yields the similarity matrix between the second intermediate feature vectors; Calculate the adjacency matrix based on the similarity between the second intermediate feature vectors; Each second intermediate feature vector is used as a node, and the similarity between the second intermediate feature vectors is used as an edge to create a first distorted relation graph structure. The first distorted relation graph structure is then input into a graph convolutional neural network to update the nodes and generate a second distorted relation graph structure.
3. The referenceless image quality assessment method based on graph comparison according to claim 2, characterized in that, The positive and negative distortion contrast sample generation model includes: a first graph convolutional neural network, a second graph convolutional neural network, a first Gumbel-softmax and a second Gumbel-softmax; The generation of the positive example distortion graph structure and the negative example distortion graph structure includes: inputting the second distortion graph structure into the first Gumbel-softmax and the second Gumbel-softmax respectively; The first Gumbel-softmax and the second Gumbel-softmax respectively sample each node in the second distorted relation graph structure to obtain n first sampling probability distribution vectors and n second sampling probability distribution vectors; Combine the n first sampling probability distribution vectors into a first sampling matrix, and combine the n second sampling probability distribution vectors into a second sampling matrix; The first sampling matrix is used to update the nodes in the second distorted relation graph structure to obtain the amplified first intermediate graph structure. The amplified first intermediate graph structure is input into the first graph convolutional neural network to update the nodes and obtain the positive example distortion graph structure; The nodes in the second distorted relation graph structure are updated using the second sampling matrix to obtain the amplified second intermediate graph structure. The amplified second intermediate graph structure is input into the second graph convolutional neural network to update the nodes, resulting in a negative example distorted relational graph structure.
4. The referenceless image quality assessment method based on graph comparison according to claim 3, characterized in that, The contrastive learning encoder model includes: VGG network and MLP; The process of obtaining positive and negative sample feature vectors includes: inputting the positive and negative sample distortion graph structures into a VGG network for feature extraction, and using MLP to reduce the dimensionality of the features to obtain positive and negative sample feature vectors.
5. The referenceless image quality assessment method based on graph comparison according to claim 4, characterized in that, The distortion type prediction model includes: a first fully connected layer, a second fully connected layer, and a softmax layer connected in sequence; The predicted node category includes: passing the node's feature information through two fully connected layers; finally, using the Softmax function for classification prediction, with pseudo-labels used to indicate its distortion type, and custom labels used for the actual distortion image.
6. The referenceless image quality assessment method based on graph comparison according to claim 5, characterized in that, The prediction loss function includes: Where N represents the number of samples and K represents the number of categories. It is the true label of sample i. It is the probability that sample i belongs to category j.
7. The referenceless image quality assessment method based on graph comparison according to claim 6, characterized in that, The contrastive loss function includes: Where B represents the batch value; 𝒯 is the temperature coefficient; exp() represents the exponential operation; and These represent the feature vector of positive samples and the feature quantity of negative samples, respectively. express 2-norm; This represents the contrast loss between a pair of positive and negative samples; This represents the total comparative loss.
8. The referenceless image quality assessment method based on graph comparison according to claim 7, characterized in that, The quality evaluation model consists of several fully connected layers.