An image quality evaluation system and method based on domain adaptation

Abstract

The present invention discloses an image quality evaluation system and method based on domain adaptation. The system includes: a dataset construction module, a network construction module, a mutual supervision module, a parameter updating module and a quality evaluation module; the dataset construction module is used to construct a source domain dataset and a target domain dataset for image quality evaluation; the network construction module is used to construct a domain adaptive twin network, and based on the domain discrimination results of the source domain dataset and the target domain dataset, guide the sub-network to focus on easy-to-transfer areas and difficult-to-transfer areas respectively; the mutual supervision module screens high-confidence pseudo-labels based on entropy sorting, so that the sub-networks in the domain adaptive twin network supervise each other's target domain prediction results through pseudo-labels; the parameter updating module updates the parameters of the domain adaptive twin network based on global loss to obtain an image quality evaluation model; the quality evaluation module uses the image quality evaluation model to perform image quality evaluation on the image to be evaluated.

Description

Technical Field

[0001] The present invention belongs to the technical field of image quality evaluation, and in particular relates to an image quality evaluation system and method based on domain adaptation. Background Art

[0002] Deep learning algorithms have demonstrated superior performance in image quality assessment. However, when there is a significant domain shift between the source domain training data and the target domain test data, the performance of deep learning-based image quality assessment methods often degrades significantly. To address this issue, domain-adaptive image quality assessment methods have emerged. These methods extract domain-invariant features through methods such as minimizing the maximum mean difference or adversarial training, effectively alleviating the performance degradation of deep learning-based image quality assessment methods in the presence of domain shift.

[0003] Current domain-adaptive image quality assessment methods face the dilemma of balancing transferability and discriminability when extracting domain-invariant features. Image quality assessment is a complex task that requires comprehensive consideration of the degree of distortion and the main content. The discriminative features of an image usually come from the hard-to-transfer regions that reflect the main content. The main content of the source and target domains is usually quite different. Forcing the alignment of features in hard-to-transfer regions can easily cause mode collapse and prevent good domain adaptation effects. The easily transferable features of an image usually come from the background region of the image. Forcing the alignment of features in the easily transferable regions of the source and target domains will lose content information, resulting in a decrease in discriminative performance. Summary of the Invention

[0004] The present invention aims to solve the deficiencies of the prior art and provides the following solutions:

[0005] An image quality assessment system based on domain adaptation, comprising: a data set construction module, a network construction module, a mutual supervision module, a parameter updating module and a quality assessment module;

[0006] The dataset construction module is used to construct a source domain dataset and a target domain dataset for image quality evaluation;

[0007] The network construction module is used to construct a domain-adaptive twin network, and guide the sub-network to focus on easy-to-transfer areas and difficult-to-transfer areas respectively based on the domain discrimination results of the source domain dataset and the target domain dataset;

[0008] The mutual supervision module selects high-confidence pseudo labels based on entropy sorting, so that the sub-networks in the domain adaptive twin network supervise each other's prediction results of the target domain through the pseudo labels;

[0009] The parameter updating module updates the parameters of the domain adaptive Siamese network based on the global loss to obtain an image quality evaluation model;

[0010] The quality assessment module uses the image quality assessment model to perform image quality assessment on the image to be assessed.

[0011] Preferably, the data set construction module includes: an initial data acquisition unit and a data processing unit;

[0012] The initial data acquisition unit is used to acquire image data, wherein the image data includes initial source domain data and initial target domain data;

[0013] The data processing unit is used to perform data enhancement on the initial source domain data and the initial target domain data to obtain the source domain data set and the target domain dataset in, represents the source domain image data, represents the target domain image data, Represents the image quality label of the source domain.

[0014] Preferably, the domain adaptive Siamese network is composed of a sub-network focusing on easy-to-transfer areas and a sub-network focusing on difficult-to-transfer areas;

[0015] The focus on the easily transferable area sub-network includes: feature extractor Local area discriminator Global Domain Discriminator and score distribution classifier

[0016] The sub-network focusing on the difficult-to-transfer area includes: a feature extractor Local area discriminator Global Domain Discriminator and score distribution classifier

[0017] Preferably, the source domain label loss of the domain adaptive Siamese network is constructed by a feature extractor and a score distribution classifier:

[0018] extracting a first global quality feature of the source domain image data by the feature extractor;

[0019] Mapping the first global quality feature into a first quality score distribution of source domain data by the score distribution classifier;

[0020] Aggregate the first quality score distribution into a mean opinion score, and calculate the root mean square error based on the mean opinion score and the source domain quality label to construct the source domain label loss, wherein the source domain label loss includes the source domain label loss of the sub-network focusing on the easy-to-transfer region and the source domain label loss of the sub-network focusing on the difficult-to-transfer region:

[0021]

[0022]

[0023] in, Indicates the source domain label loss of the sub-network focusing on the easily transferable region, n s Indicates the total amount of source domain data, represents the mean opinion score of the sub-network that focuses on the easily transferable region, Expressing concern about the source domain label loss of the sub-network in the difficult-to-transfer region, Indicates the mean opinion score of the sub-network that focuses on the hard-to-transfer region.

[0024] Preferably, the global domain discriminator motivates the domain adaptive Siamese network to align global quality features by constructing a global domain discrimination loss;

[0025] The global domain discrimination loss includes: the global domain discrimination loss of the sub-network focusing on the easy-to-transfer region and the global domain discrimination loss of the sub-network focusing on the difficult-to-transfer region:

[0026]

[0027]

[0028] in, Indicates the focus on the global domain discrimination loss of the easy-to-transfer region sub-network, Indicates the global domain discrimination loss of the sub-network focusing on the difficult-to-transfer region, n = n s +n t , n t represents the amount of data in the target domain, D = D s ∪D t , L ce represents cross entropy loss, * represents source domain / target domain, Indicates the field label.

[0029] Preferably, the local domain discriminator stimulates the domain adaptive twin network to align local quality features by constructing a local domain discrimination loss, and guides the two sub-networks in the domain adaptive twin network to focus on easy-to-transfer areas and difficult-to-transfer areas respectively through the entropy of the domain discrimination result of the local domain discriminator;

[0030] The local domain discrimination loss includes: the local domain discrimination loss of the sub-network focusing on the easy-to-transfer region and the local domain discrimination loss of the sub-network focusing on the difficult-to-transfer region:

[0031]

[0032]

[0033] in, Expressing concern about the local domain discrimination loss of the sub-network in the easily transferable area, represents the local domain discrimination loss of the sub-network focusing on the difficult-to-transfer area, R represents the number of blocks in the image division area, Indicates the label of the area described by each area, Represents different areas of the image.

[0034] Preferably, the workflow of the mutual supervision module includes:

[0035] extracting a second global quality feature of the target domain image data by the feature extractor;

[0036] Mapping the second global quality feature into a second quality score distribution of target domain data by the score distribution classifier;

[0037] Calculate the entropy of the second quality score distribution, perform entropy sorting on the second quality score distribution of the target domain prediction of the domain adaptive twin network based on the obtained entropy value, set a screening ratio that increases with the number of training steps, and screen out a high-confidence target domain score distribution of the target domain prediction with the smallest entropy based on the screening ratio;

[0038] Aggregate the high-confidence target domain score distribution to obtain a high-confidence image quality pseudo label, wherein the high-confidence image quality pseudo label includes: a sub-network image quality pseudo label focusing on the easy-to-transfer region and a sub-network image quality pseudo label focusing on the difficult-to-transfer region:

[0039]

[0040]

[0041] in, Indicates the pseudo quality label of the image in the sub-network that focuses on the easily transferred region, represents the probability that the target domain i-th confidence distribution score predicted by the sub-network focusing on the easily transferable region is j, Indicates the pseudo label of the image quality of the sub-network focusing on the difficult-to-transfer region, represents the probability that the target domain i-th confidence distribution score predicted by the sub-network focusing on the difficult-to-transfer region is j;

[0042] The sub-network focusing on the easy-to-transfer region and the sub-network focusing on the difficult-to-transfer region respectively supervise the target domain prediction results of the other network through the high-confidence image quality pseudo labels, and obtain the mutual supervision task loss of the sub-network focusing on the easy-to-transfer region and the mutual supervision task loss of the sub-network focusing on the difficult-to-transfer region:

[0043]

[0044]

[0045] in, Expressing concern about the loss of mutual supervision tasks among sub-networks in the easily transferable region, Indicates the loss of mutual supervision task of sub-network in the difficult-to-transfer region, n c represents the number of high-confidence image quality pseudo labels, Indicates the high-confidence image quality pseudo labels predicted by the sub-network that filters out and focuses on the difficult-to-transfer region The mean opinion score of the corresponding data, Indicates the high-confidence image quality pseudo labels predicted by the sub-network that filters out the easily transferable regions The mean opinion score of the corresponding data.

[0046] Preferably, the workflow of the parameter updating module includes:

[0047] Self-supervised training loss is introduced, and the network global loss is constructed based on the source domain label loss, global neighborhood discrimination loss, local neighborhood discrimination loss, self-supervised training loss and mutual supervision task loss:

[0048]

[0049]

[0050] Among them, L A represents the global loss of the sub-network focusing on the easily transferable region, L B represents the global loss of the sub-network focusing on the difficult-to-transfer region, represents the self-supervised loss of the sub-network focusing on the easily transferable region, represents the self-supervised loss of the sub-network focusing on the difficult-to-transfer region, and α, β, γ, λ, ω are hyperparameters for balancing the various losses;

[0051] The global loss is used to update the parameters of the domain adaptive Siamese network to obtain an image quality evaluation model.

[0052] The present invention also provides an image quality evaluation method based on domain adaptation, which is applied to any of the above-mentioned evaluation systems and includes the following steps:

[0053] Construct source domain datasets and target domain datasets for image quality evaluation;

[0054] Constructing a domain-adaptive twin network, and guiding the sub-network to focus on easy-to-transfer areas and difficult-to-transfer areas respectively based on the domain discrimination results of the source domain dataset and the target domain dataset;

[0055] Screening high-confidence pseudo labels based on entropy sorting, so that the sub-networks in the domain adaptive twin network supervise each other's prediction results of the target domain through the pseudo labels;

[0056] The parameters of the domain adaptive Siamese network are updated based on the global loss to obtain an image quality evaluation model;

[0057] The image quality assessment model is used to perform image quality assessment on the image to be assessed.

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] This invention solves the difficult problem of balancing transferability and discriminability in the domain adaptation process of image quality assessment. By constructing a domain-adaptive twin network that focuses on easy-to-transfer areas and hard-to-transfer areas, and obtaining high-confidence image quality pseudo-labels through a course-based pseudo-label screening and warm-up module, the target domain prediction results of the other network are supervised based on the high-confidence image quality pseudo-labels, thereby encouraging the twin network to obtain feature information that is both transferable and discriminative, thereby improving the performance of the domain-adaptive image quality assessment method. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 Schematic diagram of the system structure of an embodiment of the present invention;

[0062] Figure 2 This is a schematic diagram of the domain-adaptive twin network structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0064] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0065] Example 1

[0066] In this embodiment, if Figure 1 As shown, an image quality assessment system based on domain adaptation includes: a dataset construction module, a network construction module, a mutual supervision module, a parameter updating module and a quality assessment module.

[0067] The dataset construction module is used to construct the source domain dataset and target domain dataset for image quality evaluation.

[0068] The dataset construction module includes: an initial data acquisition unit and a data processing unit; the initial data acquisition unit is used to acquire image data, which includes initial source domain data and initial target domain data; the data processing unit is used to perform data enhancement on the initial source domain data and the initial target domain data to obtain the source domain dataset and target domain dataset in, represents the source domain image data, represents the target domain image data, Represents the image quality label of the source domain.

[0069] In this embodiment, data augmentation is performed on the image data of the source and target domains. The augmentation methods include but are not limited to random rotation, random cropping, random mirroring, scaling, and other operations. In this embodiment, the source and target domain image data are first scaled proportionally based on the short side, with the short side scaled to 384 and the long side scaled according to the original ratio of the long side to the short side. Then, image blocks of this size are randomly cropped. Finally, random horizontal mirroring is performed as data augmentation to form the final source and target domain datasets for domain adaptation training.

[0070] The network construction module is used to construct a domain-adaptive twin network, and based on the domain discrimination results of the source domain dataset and the target domain dataset, guide the sub-network to focus on easy-to-transfer areas and difficult-to-transfer areas respectively.

[0071] In this embodiment, if Figure 2 As shown in Figure 2, the domain adaptive Siamese network consists of a sub-network focusing on easy-to-transfer regions and a sub-network focusing on hard-to-transfer regions. The sub-network focusing on easy-to-transfer regions includes: feature extractor Local area discriminator Global Domain Discriminator and score distribution classifier The sub-network focusing on difficult-to-transfer areas includes: feature extractor Local area discriminator Global Domain Discriminator and score distribution classifier

[0072] The source domain label loss of the domain adaptive twin network is constructed through the feature extractor and the score distribution classifier: the first global quality feature of the source domain image data is extracted through the feature extractor; the first global quality feature is mapped to the first quality score distribution of the source domain data through the score distribution classifier; the first quality score distribution is aggregated into the mean opinion score, and the root mean square error is calculated based on the mean opinion score and the source domain quality label to construct the source domain label loss.

[0073] In this embodiment, for the sub-network focusing on the easily transferable region, the feature extractor Extract image data from the source domain Global quality characteristics Then pass the score distribution classifier The global quality features extracted from the source domain Mapped to the quality score distribution of source domain data

[0074]

[0075] in, Indicates the probability of the i-th image getting score j predicted by the sub-network focusing on the easily transferable region. Aggregate into mean opinion score

[0076]

[0077] Aggregated mean opinion score and source domain quality labels Calculate the root mean square error to construct the source domain label loss of the sub-network that focuses on the easy-to-transfer region to learn the quality discrimination knowledge implicit in the source domain label data; similarly, use the above steps to construct the source domain label loss of the sub-network that focuses on the difficult-to-transfer region:

[0078]

[0079]

[0080] in, Indicates the source domain label loss of the sub-network focusing on the easily transferable region, n s Indicates the total amount of source domain data, represents the mean opinion score of the sub-network that focuses on the easily transferable region, Expressing concern about the source domain label loss of the sub-network in the difficult-to-transfer region, Indicates the mean opinion score of the sub-network that focuses on the hard-to-transfer region.

[0081] The global domain discriminator encourages the domain-adaptive twin network to align global quality features by constructing a global domain discrimination loss. The global domain discrimination loss includes: the global domain discrimination loss of the sub-network focusing on the easy-to-transfer region and the global domain discrimination loss of the sub-network focusing on the difficult-to-transfer region:

[0082]

[0083]

[0084] in, Indicates the focus on the global domain discrimination loss of the easy-to-transfer region sub-network, Indicates the global domain discrimination loss of the sub-network focusing on the difficult-to-transfer region, n = n s +n t , n t represents the amount of data in the target domain, D = D s ∪D t , L ce represents cross entropy loss, * represents source domain / target domain, Indicates the field label.

[0085] The local domain discriminator encourages the domain adaptive twin network to align local quality features by constructing a local domain discrimination loss, and guides the two sub-networks in the domain adaptive twin network to focus on easy-to-transfer areas and difficult-to-transfer areas respectively through the entropy of the domain discrimination results of the local domain discriminator; the local domain discrimination loss includes: the local domain discrimination loss of the sub-network focusing on the easy-to-transfer area and the local domain discrimination loss of the sub-network focusing on the difficult-to-transfer area:

[0086]

[0087]

[0088] in, Expressing concern about the local domain discrimination loss of the sub-network in the easily transferable area, represents the local domain discrimination loss of the sub-network focusing on the difficult-to-transfer area, R represents the number of blocks in the image division area, Indicates the label of each area. Represents different areas of the image. It represents the local quality features of different regions of the image extracted by the easy-to-transfer region sub-network, denoted as Indicates the binary classification results of the local domain discriminator of the sub-network that focuses on the easy-to-transfer region for different regional features; the region with a classification probability of 0.5 in the binary classification result indicates the region that can confuse the local discriminator, that is, the easy-to-transfer region. Let the entropy solution function be H, and the entropy of the classification result be The local quality features extracted from different regions are weighted to guide the sub-network focusing on easy-to-transfer regions to focus on easy-to-transfer regions. It represents the local quality features of different regions of the image extracted by the sub-network focusing on the difficult-to-transfer region, denoted as Indicates the discrimination results of the local domain discriminator of the sub-network of the difficult-to-transfer region for the two-class local quality features of different regions, through The extracted local quality features are weighted to guide the sub-network that focuses on difficult-to-transfer areas to focus on difficult-to-transfer areas. The above-mentioned entropy weighting method based on the discrimination results of the local domain discriminator is used to construct a twin network that focuses on easy-to-transfer areas and difficult-to-transfer areas respectively. Before being sent to the local domain discriminator, the local quality features are also unified by the maximum and minimum game of the feature extractor and the local domain discriminator through the gradient reversal layer. In this embodiment, the entropy weighting of the discrimination results of the local domain discriminator is only applied to the attention module of the last encoder layer of the feature extractor transformer. The local quality features extracted from different patches of the image are weighted using the entropy of the discrimination results of the local domain discriminator through the Hadamard product method to guide the two sub-networks to focus on easy-to-transfer areas and difficult-to-transfer areas in the domain adaptation process respectively.

[0089] The mutual supervision module selects high-confidence pseudo labels based on entropy sorting, so that the sub-networks in the domain adaptive twin network can supervise each other's prediction results of the target domain through pseudo labels.

[0090] The workflow of the mutual supervision module includes: extracting the second global quality feature of the target domain image data through the feature extractor; mapping the second global quality feature to the second quality score distribution of the target domain data through the score distribution classifier; calculating the entropy of the second quality score distribution, and entropy sorting the second quality score distribution of the target domain prediction of the domain adaptive twin network based on the obtained entropy value, setting a screening ratio that increases with the number of training steps, and screening out the high-confidence target domain score distribution of the target domain prediction with the smallest entropy based on the screening ratio; aggregating the high-confidence target domain score distribution to obtain a high-confidence image quality pseudo label.

[0091] In this embodiment, for the sub-network focusing on the easily transferable region, the image data of the target domain Through feature extractor Obtain global quality characteristics of target domain data Then pass the score distribution classifier The global quality features extracted from the target domain Mapped to the score distribution of target domain data

[0092]

[0093] in, Indicates that the sub-network focusing on the easily transferable region predicts the probability of the i-th image obtaining score j; calculates the score distribution of the target domain data Entropy According to the low entropy assumption that the score distribution should meet, the score distribution predicted by the sub-network of the easy-to-transfer region is sorted by entropy, and the high-confidence target domain score distribution predicted by the sub-network of the easy-to-transfer region with the smallest entropy is selected with η as the ratio. Next, we will focus on the high-confidence target domain distribution predicted by the easy-to-transfer region subnetwork. Aggregate to mean opinion score to get high confidence image quality pseudo labels Similarly, according to the above steps, we can get high-confidence image quality pseudo labels

[0094]

[0095]

[0096] in, Indicates the pseudo quality label of the image in the sub-network that focuses on the easily transferred region, represents the probability that the target domain i-th confidence distribution score predicted by the sub-network focusing on the easily transferable region is j, Indicates the pseudo label of the image quality of the sub-network focusing on the difficult-to-transfer region, The probability that the target domain i-th confidence distribution score predicted by the sub-network focusing on the difficult-to-transfer region is j is expressed; by continuously increasing the screening ratio η, the process of human learning from easy to difficult is simulated. The increase in the ratio η can be continuously increased according to the performance of the sub-network on the source domain test set, or it can be continuously increased with the number of training steps. In this embodiment, the screening ratio η is continuously increased in a gradual manner by the number of training steps. Let the number of preheating training steps be T warm , the total number of training steps T,T warm <T:

[0097]

[0098] Wherein, η0 represents the baseline screening ratio, and t represents the current number of training steps. The sub-network focusing on the easy-to-transfer region and the sub-network focusing on the difficult-to-transfer region supervise the target domain prediction results of each other's network through high-confidence image quality pseudo-labels, and obtain the mutual supervision task loss of the sub-network focusing on the easy-to-transfer region and the mutual supervision task loss of the sub-network focusing on the difficult-to-transfer region; in this embodiment, the two sub-networks supervise each other's target domain prediction results based on the high-confidence image quality pseudo-labels obtained by the screening ratio by setting a screening ratio that increases with the number of training steps. The target domain score distribution predicted by the sub-network focusing on the easy-to-transfer region is calculated. Aggregate into mean opinion score Filter out high-confidence image-quality pseudo labels predicted by the sub-network that focuses on the hard-to-transfer regions The mean opinion score of the corresponding data is recorded as The mutual supervision task loss of sub-network A is constructed by the root mean square error:

[0099]

[0100] Similarly, we will focus on the target domain score distribution predicted by the sub-network in the difficult-to-transfer region Aggregate into mean opinion score Filter out high-confidence image-quality pseudo labels predicted by the sub-network that focuses on easily transferred regions The mean opinion score of the corresponding data is recorded as The mutual supervision task loss of sub-network B is constructed by the root mean square error:

[0101]

[0102] in, Expressing concern about the loss of mutual supervision tasks among sub-networks in the easily transferable region, Indicates the loss of mutual supervision task of sub-network in the difficult-to-transfer region, n c Indicates the number of high-confidence image quality pseudo labels, n c =ηn t .

[0103] The parameter update module updates the parameters of the domain adaptive Siamese network based on the global loss to obtain an image quality evaluation model.

[0104] To further improve the performance of the network, this embodiment introduces a self-supervised loss in the target domain. This loss consists of three parts: minimizing the distribution entropy of single-image predictions to encourage the network to make more confident score distribution predictions; maximizing the entropy of the average value of the probability of different score distributions within a minibatch to encourage the network to make a wider range of predictions for the scores of different images and accelerate network convergence; and minimizing the distance between the predicted score distribution and its truncated discrete normal distribution with the same expectation and variance to simulate the objective law that human scores conform to the normal distribution.

[0105] The workflow of the parameter update module includes: introducing self-supervised training loss, constructing the network global loss based on source domain label loss, global neighborhood discrimination loss, local neighborhood discrimination loss, self-supervised training loss and mutual supervision task loss:

[0106]

[0107]

[0108] Among them, LA represents the global loss of the sub-network focusing on the easily transferable region, L B represents the global loss of the sub-network focusing on the difficult-to-transfer region, represents the self-supervised loss of the sub-network focusing on the easily transferable region, represents the self-supervised loss of the sub-network focusing on the difficult-to-transfer area, and α, β, γ, λ, and ω are hyperparameters for balancing the various losses. The global loss is used to update the parameters of the domain-adaptive twin network to obtain the image quality evaluation model.

[0109] The quality assessment module uses the image quality assessment model to perform image quality assessment on the image to be evaluated.

[0110] After domain adaptation, the target domain image is passed through the domain adaptive Siamese network to output the target domain image quality assessment result. The results are aggregated into a mean opinion score (SROCC) and compared with the actual quality labels to calculate the SROCC and PLCC to verify the effectiveness of the scheme. This embodiment performs a five-parameter fitting on the aggregated mean opinion score before calculating the PLCC to ensure the accuracy of the quality assessment.

[0111]

[0112] in, Represents the average opinion score after aggregation, and β1, β2, β3, β4, and β5 are parameters to be fitted. In this embodiment, the feature extractor adopts ViT-B16 pre-trained on ImageNet-21k, the structure of the score distribution classifier is (FC-LeakyReLu-FC-LeakyReLu-FC-Softmax), and the structures of the global domain discriminator and the local domain discriminator are both (FC-ReLu-Dropout-FC-ReLu-Dropout-FC-Sigmoid). In this embodiment, the SGD optimizer is used to optimize the network weights, the weight decay weightdecay is set to 0.0001, and the momentum momentum is set to 0.9. The learning rate of the global domain discriminator and the local domain discriminator is 0.002, the learning rate of the feature extractor is 0.005, and the learning rate of the score distribution classifier is set to 0.001. The total number of training steps is 2000, and the number of preheating training steps is 500. The learning rate of each module was linearly increased from 0 to the preset learning rate for the first 1000 training steps and then decreased to 0 using a cosine decay function for the remaining training steps. The baseline screening ratio η was 0.2, and the number of warmup training steps was 500. The hyperparameters of each loss were α, β, γ, λ, and ω, respectively, which were 0.2, 0.15, 0.15, 0.2, and 0.2.

[0113] Example 2

[0114] In this embodiment, a domain-adaptive image quality assessment method includes the following steps:

[0115] Construct source domain datasets and target domain datasets for image quality evaluation; construct a domain adaptive twin network, and guide the sub-network to focus on easy-to-transfer areas and difficult-to-transfer areas based on the domain discrimination results of the source domain dataset and the target domain dataset respectively; screen high-confidence pseudo-labels based on entropy sorting, so that the sub-networks in the domain adaptive twin network can supervise each other's target domain prediction results through pseudo-labels; update the parameters of the domain adaptive twin network based on the global loss to obtain an image quality evaluation model; use the image quality evaluation model to evaluate the image quality of the image to be evaluated.

[0116] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A domain-adaptive image quality assessment system, characterized in that: include: Dataset construction module, network construction module, mutual supervision module, parameter update module and quality evaluation module; The dataset construction module is used to construct a source domain dataset and a target domain dataset for image quality evaluation; The network construction module is used to construct a domain-adaptive twin network, and guide the sub-network to focus on easy-to-transfer areas and difficult-to-transfer areas respectively based on the domain discrimination results of the source domain dataset and the target domain dataset; The mutual supervision module selects high-confidence pseudo labels based on entropy sorting, so that the sub-networks in the domain adaptive twin network supervise each other's prediction results of the target domain through the pseudo labels; The parameter updating module updates the parameters of the domain adaptive Siamese network based on the global loss to obtain an image quality evaluation model; The quality assessment module uses the image quality assessment model to perform image quality assessment on the image to be assessed; The data set construction module includes: an initial data acquisition unit and a data processing unit; The initial data acquisition unit is used to acquire image data, wherein the image data includes initial source domain data and initial target domain data; The data processing unit is used to perform data enhancement on the initial source domain data and the initial target domain data to obtain the source domain data set and the target domain dataset ,in, represents the source domain image data, represents the target domain image data, Image quality labels representing the source domain; The domain adaptive Siamese network consists of a sub-network focusing on easy-to-transfer areas and a sub-network focusing on difficult-to-transfer areas; The focus on the easily transferable area sub-network includes: feature extractor , local area discriminator , global domain discriminator and score distribution classifier ; The sub-network focusing on the difficult-to-transfer area includes: a feature extractor , local area discriminator , global domain discriminator and score distribution classifier ; The global domain discriminator encourages the domain-adaptive Siamese network to align global quality features by constructing a global domain discrimination loss; The global domain discrimination loss includes: the global domain discrimination loss of the sub-network focusing on the easy-to-transfer region and the global domain discrimination loss of the sub-network focusing on the difficult-to-transfer region: in, Indicates the focus on the global domain discrimination loss of the easy-to-transfer region sub-network, Expresses concern about the global domain discrimination loss of the sub-network in the difficult-to-transfer region, n = n s + n t , n t represents the amount of data in the target domain, , represents cross entropy loss, * represents source domain / target domain, Indicates the field label; It represents the local quality features of different regions of the image extracted by the easy-to-transfer region sub-network, denoted as , Indicates the binary classification results of the local domain discriminator of the sub-network that focuses on the easy-to-transfer region for different regional features; the region with a classification probability of 0.5 in the binary classification result indicates the region that can confuse the local discriminator, that is, the easy-to-transfer region. Let the entropy solution function be H, and the entropy of the classification result be The local quality features extracted from different regions are weighted to guide the sub-network that focuses on easy-to-transfer regions to focus on easy-to-transfer regions; It represents the local quality features of different regions of the image extracted by the sub-network focusing on the difficult-to-transfer region, denoted as , Indicates the discrimination results of the local domain discriminator of the sub-network of the difficult-to-transfer region for the two-class local quality features of different regions, through The extracted local quality features are weighted to guide the sub-network that focuses on difficult-to-transfer areas to focus on difficult-to-transfer areas. A twin network that focuses on easy-to-transfer areas and difficult-to-transfer areas is constructed based on the entropy weighting method of the local domain discriminator's discrimination results.

2. The domain-adaptive image quality assessment system according to claim 1, characterized in that: The source domain label loss of the domain adaptive Siamese network is constructed through the feature extractor and score distribution classifier: extracting a first global quality feature of the source domain image data by the feature extractor; Mapping the first global quality feature into a first quality score distribution of source domain data by the score distribution classifier; Aggregate the first quality score distribution into a mean opinion score, and calculate the root mean square error based on the mean opinion score and the source domain quality label to construct the source domain label loss, wherein the source domain label loss includes the source domain label loss of the sub-network focusing on the easy-to-transfer region and the source domain label loss of the sub-network focusing on the difficult-to-transfer region: in, Indicates the concern about the source domain label loss of the sub-network in the easily transferable region, n s Indicates the total amount of source domain data, represents the mean opinion score of the sub-network that focuses on the easily transferable region, Expressing concern about the source domain label loss of the sub-network in the difficult-to-transfer region, Indicates the mean opinion score of the sub-network that focuses on the hard-to-transfer region.

3. The domain-adaptive image quality assessment system according to claim 2, characterized in that: The local domain discriminator encourages the domain adaptive twin network to align local quality features by constructing a local domain discrimination loss, and guides the two sub-networks in the domain adaptive twin network to focus on easy-to-transfer areas and difficult-to-transfer areas respectively through the entropy of the domain discrimination results of the local domain discriminator; The local domain discrimination loss includes: the local domain discrimination loss of the sub-network focusing on the easy-to-transfer region and the local domain discrimination loss of the sub-network focusing on the difficult-to-transfer region: in, Expressing concern about the local domain discrimination loss of the sub-network in the easily transferable area, represents the local domain discrimination loss of the sub-network focusing on the difficult-to-transfer area, R represents the number of blocks in the image division area, Indicates the label of the area described by each area, Represents different areas of the image.

4. The domain-adaptive image quality assessment system according to claim 3, characterized in that: The workflow of the mutual supervision module includes: extracting a second global quality feature of the target domain image data by the feature extractor; Mapping the second global quality feature into a second quality score distribution of target domain data by the score distribution classifier; Calculate the entropy of the second quality score distribution, perform entropy sorting on the second quality score distribution of the target domain prediction of the domain adaptive twin network based on the obtained entropy value, set a screening ratio that increases with the number of training steps, and screen out a high-confidence target domain score distribution of the target domain prediction with the smallest entropy based on the screening ratio; Aggregate the high-confidence target domain score distribution to obtain a high-confidence image quality pseudo label, wherein the high-confidence image quality pseudo label includes: a sub-network image quality pseudo label focusing on the easy-to-transfer region and a sub-network image quality pseudo label focusing on the difficult-to-transfer region: in, Indicates the pseudo quality label of the image in the sub-network that focuses on the easily transferred region, Indicates that the target domain predicted by the sub-network focusing on the easily transferred region i The confidence distribution score is j The probability of Indicates the pseudo label of the image quality of the sub-network focusing on the difficult-to-transfer region, Indicates that the target domain predicted by the sub-network focusing on the difficult-to-transfer region i The confidence distribution score is j probability; The sub-network focusing on the easy-to-transfer region and the sub-network focusing on the difficult-to-transfer region respectively supervise the target domain prediction results of the other network through the high-confidence image quality pseudo labels, and obtain the mutual supervision task loss of the sub-network focusing on the easy-to-transfer region and the mutual supervision task loss of the sub-network focusing on the difficult-to-transfer region: in, Expressing concern about the loss of mutual supervision tasks among sub-networks in the easily transferable region, Expressing concern about the loss of mutual supervision tasks among sub-networks in difficult-to-transfer regions, n c represents the number of high-confidence image quality pseudo labels, Indicates the high-confidence image quality pseudo labels predicted by the sub-network that filters out and focuses on the difficult-to-transfer region The mean opinion score of the corresponding data, Indicates the high-confidence image quality pseudo labels predicted by the sub-network that filters out the easily transferable regions The mean opinion score of the corresponding data.

5. The domain-adaptive image quality assessment system according to claim 4, characterized in that: The workflow of the parameter updating module includes: Self-supervised training loss is introduced, and the network global loss is constructed based on the source domain label loss, global neighborhood discrimination loss, local neighborhood discrimination loss, self-supervised training loss and mutual supervision task loss: in, L A represents the global loss of the sub-network focusing on the easily transferable region, L B represents the global loss of the sub-network focusing on the difficult-to-transfer region, represents the self-supervised loss of the sub-network focusing on the easily transferable region, represents the self-supervised loss of the sub-network that focuses on the difficult-to-transfer region, α 、 β 、 γ 、 λ 、 ω To balance the hyperparameters of each loss; The global loss is used to update the parameters of the domain adaptive Siamese network to obtain an image quality evaluation model.

6. A domain-adaptive image quality evaluation method, the evaluation method being applied to the evaluation system according to any one of claims 1 to 5, characterized in that: The following steps are involved: Construct source domain datasets and target domain datasets for image quality evaluation; Constructing a domain-adaptive twin network, and guiding the sub-network to focus on easy-to-transfer areas and difficult-to-transfer areas respectively based on the domain discrimination results of the source domain dataset and the target domain dataset; Screening high-confidence pseudo labels based on entropy sorting, so that the sub-networks in the domain adaptive twin network supervise each other's prediction results of the target domain through the pseudo labels; The parameters of the domain adaptive Siamese network are updated based on the global loss to obtain an image quality evaluation model; The image quality assessment model is used to perform image quality assessment on the image to be assessed.

Images