Lightweight contrast learning image quality evaluation method and system based on quality decoupling

Abstract

The invention provides a lightweight contrast learning image quality evaluation method and system based on quality decoupling, and the method comprises the steps: reconstructing the pre-scale features of each scale feature based on a reconstruction network in a semantic quality decoupling module, so as to obtain the pre-reconstruction scale features of three dimensions; modeling the local conditional probability of each scale feature and the previous scale feature at the local position, obtaining the overall conditional entropy of the scale features and the previous scale feature according to the conditional probability, and obtaining a weight matrix which maximizes the overall conditional entropy through a preset modeling optimization formula; the scale features before reconstruction are multiplied by the weight matrix to carry out spatial transformation, feature splicing, pooling and other processing are carried out to obtain a quality perception vector, and the quality perception vector is mapped and predicted through a quality regression network and a quality prediction network to obtain a quality prediction score of the image to be evaluated; the whole training process does not need subjective opinion score participation, the objectivity of image quality evaluation is improved, and resource waste is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and more specifically to an image quality evaluation method and system based on lightweight contrast learning for quality decoupling. Background Art

[0002] Image Quality Assessment (IQA) aims to understand and evaluate the human perception when interpreting images. It can not only measure and optimize the quality of experience of video transmission systems, but also promote the development of visual tasks such as image restoration, image generation, and super-resolution. With the powerful representation ability of deep neural networks, no-reference image quality assessment has achieved great success. This type of method can evaluate image quality without the involvement of reference images.

[0003] Since subjective quality assessment experiments are time-consuming, the scale of images with MOS annotations is relatively limited. To address this small sample learning problem, many IQA methods use a deep visual backbone network pre-trained on other visual tasks, attached with a quality regression network, and supervised training in an end-to-end manner. This method leads to inconsistent MOS distribution due to differences in experimental conditions, equipment, and data processing procedures involved in the establishment of different datasets. When the model is transferred to a new dataset, it needs to be retrained. In order to improve the generalization and robustness of the BIQA model, some methods train the visual backbone network through self-supervised contrastive learning. When these methods are migrated to new datasets, they only need to train the quality regression network through supervised learning without retraining the deep backbone network. However, since these labels are at risk of inaccuracy when applied to real distorted data, it is not appropriate to learn features from distortion type labels.

[0004] In short, existing BIQA models are mostly trained using supervised learning, which places high demands on the quality and quantity of the Mean Opinion Score (MOS) in subjective experiments, and requires end-to-end supervised training when the model is migrated to a new dataset. In order to enhance the generalization and robustness of the BIQA model, some recent methods use contrastive learning to train the backbone network in a self-supervised manner, and only supervise the training of the regression network when testing on a new dataset. However, these contrastive learning-based methods consume a lot of computing resources during the training process and cannot be applied to resource-constrained environments.

[0005] Therefore, there is an urgent need for a lightweight contrastive learning image quality assessment method and system based on quality decoupling that can not only be efficiently trained on a single GPU but also does not require any MOS data to be involved in the entire training process, thereby reducing resource waste and improving the accuracy of quality assessment. Summary of the invention

[0006] In view of the above problems, the purpose of the present invention is to provide a lightweight contrast learning image quality evaluation method based on quality decoupling, so as to solve the problems that the prior art is inaccurate when applied to real distorted data, consumes a lot of computing resources during the training process, and cannot be applied to resource-constrained environments.

[0007] The present invention provides a lightweight contrast learning image quality assessment method based on quality decoupling, which includes:

[0008] Acquire scale features of four scales of the image to be evaluated through a preset semantic quality decoupling module, and reconstruct pre-scale features of each scale feature based on a reconstruction network in the semantic quality decoupling module to obtain pre-reconstructed scale features of three dimensions;

[0009] Modeling local conditional probabilities of each scale feature and the previous scale feature at a local position based on the pre-reconstructed scale feature, and obtaining overall conditional entropy of the scale feature and the previous scale feature according to the conditional probability;

[0010] Obtaining a weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula;

[0011] The pre-reconstruction scale features of the three dimensions are respectively multiplied by the corresponding weight matrices to perform spatial transformation to obtain a first transformation feature, a second transformation feature, and a third transformation feature, and the first transformation feature, the second transformation feature, the third transformation feature, and the scale feature of the fourth scale are concatenated and pooled to obtain a quality perception vector;

[0012] The quality perception vector is mapped and predicted by a pre-trained quality regression network and a quality prediction network to obtain a quality prediction score of the image to be evaluated.

[0013] Preferably, the step of obtaining scale features of four scales of the image to be evaluated by a preset semantic quality decoupling module, and reconstructing pre-scale features of each scale feature based on a reconstruction network in the semantic quality decoupling module to obtain pre-reconstructed scale features of three dimensions includes:

[0014] Inputting the image to be evaluated into the backbone network of the semantic quality decoupling module, so that the pooling layer in the backbone network performs feature extraction on the image to be evaluated to obtain scale features of four scales, namely, a first scale feature, a second scale feature, a third scale feature and a fourth scale feature;

[0015] The second scale feature, the third scale feature, and the fourth scale feature are input into the trained reconstruction network. to obtain the pre-reconstruction scale features for predicting reconstruction: reconstructing one-scale features, reconstructing two-scale features, and reconstructing three-scale features.

[0016] Preferably, the step of pre-training the reconstruction network comprises:

[0017] A preset number of images are selected to form a reference image set, and feature extraction is performed on the images in the reference image set to obtain scale features of four scales, which are called training scale features;

[0018] Construct a reconstruction network to be trained with parameter γ consisting of a fully connected layer The training scale feature X of the k+1th scale in the training scale feature k+1 ={x i ,i∈H x ×W x} Input to the reconstruction network to be trained In order to predict the pre-training scale feature Y of the k+1th scale k ={y i ,i∈H y ×W y}, and record the predicted reconstructed pre-training scale feature as the pre-reconstructed training scale feature Y k ′={y i ′,i∈H y ×W y}, where k = 1, 2, 3; H x ,W x and H y ,W y are the k+1th training scale features X k+1 and the kth training scale feature Y k or Y k ' height and width;

[0019] The pre-reconstruction training scale feature Y k ′ and the true pre-training scale feature Y k The Euclidean norm of is used as the loss function L; where L = ||Y k ′-Y k ||2;

[0020] Use Adam optimizer to update the reconstructed network to be trained Parameter γ, calculate the gradient of the loss function L with respect to the parameter γ Reconstruct the network to be trained by updating and correcting it The first and second moments of

[0021] Use the updated first-order moment and second-order moment to update the reconstructed network to be trained The parameter γ is adjusted until the loss function reaches a preset threshold, then the training is stopped and the last updated network to be trained is reconstructed As a reconstruction network

[0022] Preferably, the process of modeling the local conditional probability of each scale feature and the previous scale feature at a local position based on the pre-reconstructed scale feature, and obtaining the overall conditional entropy of the scale feature and the previous scale feature according to the conditional probability includes:

[0023] Based on the scale feature X k+1 With the previous scale feature Y k Perform Gaussian modeling to form a Gaussian model The pre-reconstruction scale feature y predicted by the reconstruction network is i ′ is used as the mean of the Gaussian model to reconstruct the pre-scale feature y i ′ and the previous scale feature y i The Euclidean norm of i as the standard deviation of the Gaussian model; I is the identity matrix of the Gaussian model;

[0024] Based on the Gaussian model By formula Calculate the scale feature X k+1 With the previous scale feature Y k The local conditional entropy at the local position i; where, Represents the reconstruction network;

[0025] For each local position i, the local conditional entropy h(y i |x i ) to obtain the scale feature X k+1 With the previous scale feature Y k The overall conditional entropy Among them, H y ,W y is the k-th scale feature Y k height and width.

[0026] Preferably, the step of obtaining a weight matrix that maximizes the overall conditional entropy by using a preset modeling optimization formula includes:

[0027] The previous scale feature Y k ={y i ,i∈H y ×W y} Perform spatial transformation to obtain the actual transformation feature Z after the k-th scale transformation k =ω k ×Y k ={ω i yi ,i∈H y ×W y};in, is the unknown weight matrix to be determined;

[0028] Through the preset modeling optimization formula, the scale feature X of the k+1th scale is k+1 The actual transformed feature Zk after spatial transformation with the previous scale feature Yk has an effect on the overall conditional entropy H e (Z k |X k+1 ) is maximized to calculate the unknown weight matrix to be calculated; wherein the modeling optimization formula is:

[0029]

[0030] in, It means to find the parameter ω that makes the function in the brackets reach the maximum value. k , H e (ω k Y k |X k+1 ) is the scale feature X of the k+1th scale k+1 Actual transformation feature Z k The overall conditional entropy, H y ,W y is the actual transformation feature Z k The height and width of is the scale feature X of the k+1th scale k+1 Actual transformation feature Z k The local conditional entropy at position i, 2πe is a constant, e is the base of the natural logarithm, δ i is the scale feature y before reconstruction i ′ and the previous scale feature y i The Euclidean norm of , the sum after st is the preset constraint function, and c in the formula is a preset constant.

[0031] Preferably, the process of solving the modeling optimization formula includes:

[0032] Constructing the solution function based on the Lagrange multiplier method where λ is the Lagrange multiplier, is the objective function of the modeling optimization formula, A constraint function for the modeling optimization formula;

[0033] when and When the scale feature X of the k+1th scale k+1 Actual transformation feature Z k The overall conditional entropy H e(ω k Y k |X k+1 ) to obtain the maximum value;

[0034] Will and When the optimal weight of the modeling optimization formula is As the weight matrix; where δ i is the scale feature y before reconstruction i ′ and the previous scale feature y i The Euclidean norm of , c is a constant.

[0035] Preferably, the step of mapping and predicting the quality perception vector by a pre-trained quality regression network and a quality prediction network to obtain a quality prediction score of the image to be evaluated includes:

[0036] Performing mapping processing on the quality perception vector through a preset quality regression network to obtain quality mapping features;

[0037] The quality map feature is input into a quality prediction network so that the quality prediction network performs activation mapping on the quality map feature to obtain a quality prediction score of the image to be evaluated.

[0038] Preferably, the step of training the quality regression network comprises:

[0039] Performing a cropping process of a specific specification on all images in the reference image set to form a standard image, and performing two levels of degradation processing on the standard image to obtain a medium image set and a poor image set;

[0040] Select B quality perception vectors ft of each image from the quality perception vector sets corresponding to the reference image set, the medium image set and the poor image set. R To form the reference feature vector Medium eigenvector Sum and Difference Eigenvectors

[0041] Construct a layer consisting of two fully connected layers and a Relu activation function with parameters φ P The quality regression network to be trained is repeatedly trained; wherein the process of repeated training is:

[0042] The reference feature vector, the medium feature vector and the difference feature vector are input into the regression network to be trained, so that the first fully connected layer maps the reference feature vector, the medium feature vector and the difference feature vector to a low-dimensional hidden layer to obtain a reference hidden layer output vector F R′, medium hidden equal output vector F M ′ and difference hidden layer output vector F P ′, and the second fully connected layer performs mapping processing on the reference hidden layer output vector, the medium hidden layer output vector and the difference hidden layer output vector to obtain a reference quality mapping feature Medium quality mapping features Quality mapping features such as sum and difference

[0043] Use the optimizer to update the quality regression network to be trained The weight φ P , calculate the preset quality regression loss function L proj About weight φ P Gradient To update the first-order moment and the second-order moment, and correct the deviation, and use the corrected first-order moment and second-order moment to update the weight φ P , until the regression loss function L proj When the preset regression loss function threshold is reached, the update is stopped and the last updated quality regression network to be trained is used as the quality regression network; the quality regression loss function L proj for:

[0044]

[0045] in, represents the cosine similarity between two vectors, α, β are hyperparameters that control the degree of feature aggregation, i,j∈[1,B].

[0046] Preferably, the step of training the quality prediction network comprises:

[0047] Construct a layer consisting of two fully connected layers and a Relu activation function with parameters φ a The quality prediction network to be trained

[0048] Mapping reference mass to features Medium quality mapping features Quality mapping features such as sum and difference Input into the quality prediction network to be trained to generate a reference image quality score, a medium image quality score and a poor image quality score respectively;

[0049] A quality prediction loss function is created based on the reference image quality score, the medium image quality score and the poor image quality score, and the quality prediction network to be trained is repeatedly trained until the quality prediction loss function reaches a preset threshold, then the training is stopped, and the quality prediction network to be trained that was last updated is used as the quality prediction network.

[0050] The present invention also provides a lightweight contrastive learning image quality assessment system based on quality decoupling, which implements the lightweight contrastive learning image quality assessment method based on quality decoupling as described above. The system includes a semantic quality decoupling module, a quality regression network and a quality prediction network; wherein,

[0051] The semantic quality decoupling module includes:

[0052] The backbone network is used to obtain the scale features of the four scales of the image to be evaluated;

[0053] A reconstruction network is used to reconstruct the pre-scale features of each scale feature to obtain the reconstructed pre-scale features of three dimensions;

[0054] A quality perception module, used for modeling the local conditional probabilities of each scale feature and the previous scale feature at a local position based on the pre-reconstruction scale feature, and obtaining the overall conditional entropy of the scale feature and the previous scale feature according to the conditional probability; obtaining a weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula; multiplying the pre-reconstruction scale features of the three dimensions with their respective corresponding weight matrices for spatial transformation to obtain a first transformation feature, a second transformation feature, and a third transformation feature, and performing splicing and pooling processing on the first transformation feature, the second transformation feature, the third transformation feature, and the scale feature of the fourth scale to obtain a quality perception vector;

[0055] The quality regression network and the quality prediction network are used to perform mapping and prediction processing on the quality perception vector to obtain the quality prediction score of the image to be evaluated.

[0056] From the above technical solution, it can be seen that the present invention provides a lightweight contrast learning image quality evaluation method and system based on quality decoupling, which reconstructs the pre-scale features of each scale feature based on the reconstruction network in the semantic quality decoupling module to obtain the reconstructed pre-scale features of three dimensions, and then models the local conditional probabilities of each scale feature and the pre-scale feature at the local position based on the reconstructed pre-scale features, and obtains the overall conditional entropy of the scale feature and the pre-scale feature according to the conditional probability, and then obtains the weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula, so that the pre-scale features of the three dimensions are respectively multiplied by their respective corresponding weight matrices for spatial transformation to obtain the first transformation feature, the second transformation feature and the first transformation feature. Three transformation features are obtained, and the first transformation feature, the second transformation feature, the third transformation feature, and the scale feature of the fourth scale are spliced ​​and pooled to obtain a quality perception vector. The quality perception vector is mapped and predicted by a pre-trained quality regression network and a quality prediction network to obtain a quality prediction score of the image to be evaluated. In this way, a lightweight quality contrast learning strategy is adopted, which can be efficiently trained on a single GPU. The entire training process does not require the participation of subjective opinion scores, and the semantic quality decoupling model is modeled as an optimization problem. Only a lightweight reconstruction network needs to be introduced to calculate the feature weight map with the largest quality representation ability, thereby improving the objectivity of image quality evaluation, reducing resource waste, and improving the accuracy of quality evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] By referring to the following description in conjunction with the accompanying drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more clear and easy to understand. In the accompanying drawings:

[0058] Figure 1 It is a flowchart of a lightweight contrast learning image quality assessment method based on quality decoupling according to an embodiment of the present invention;

[0059] Figure 2 A schematic diagram of semantic quality decoupling of a lightweight contrastive learning image quality assessment method based on quality decoupling according to an embodiment of the present invention;

[0060] Figure 3 4 is a system block diagram of a lightweight contrastive learning image quality assessment system based on quality decoupling according to an embodiment of the present invention. DETAILED DESCRIPTION

[0061] Most existing BIQA models are trained using supervised learning, which places high demands on the quality and quantity of the Mean Opinion Score (MOS) in subjective experiments, and requires end-to-end supervised training when the model is migrated to a new dataset. To enhance the generalization and robustness of the BIQA model, some recent methods use contrastive learning to train the backbone network in a self-supervised manner, and only supervise the training of the regression network when testing on a new dataset. However, these contrastive learning-based methods consume a lot of computing resources during training and cannot be applied to resource-constrained environments.

[0062] In view of the above problems, the present invention provides a lightweight contrast learning image quality assessment method and system based on quality decoupling. The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0063] In order to illustrate the lightweight contrast learning image quality assessment method and system based on quality decoupling provided by the present invention, Figure 1-Figure 3 The embodiments of the present invention are exemplarily indicated.

[0064] The following description of exemplary embodiments is merely illustrative in nature and is not intended to limit the present invention and its application or use. Technologies and devices known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the technologies and devices should be considered as part of the specification.

[0065] like Figure 1 As shown, the present invention provides a lightweight contrast learning image quality assessment method based on quality decoupling, comprising:

[0066] S1: acquiring scale features of four scales of the image to be evaluated through a preset semantic quality decoupling module, and reconstructing pre-scale features of each scale feature based on a reconstruction network in the semantic quality decoupling module to obtain pre-reconstructed scale features of three dimensions;

[0067] S2: Modeling the local conditional probability of each scale feature and the previous scale feature at a local position based on the reconstructed scale feature, and obtaining the overall conditional entropy of the scale feature and the previous scale feature according to the conditional probability;

[0068] S3: Obtaining a weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula;

[0069] S4: multiplying the pre-reconstruction scale features of the three dimensions by their respective corresponding weight matrices for spatial transformation to obtain a first transformation feature, a second transformation feature, and a third transformation feature, and performing splicing and pooling processing on the first transformation feature, the second transformation feature, the third transformation feature, and the scale feature of the fourth scale to obtain a quality perception vector;

[0070] S5: Mapping and predicting the quality perception vector through a pre-trained quality regression network and a quality prediction network to obtain a quality prediction score of the image to be evaluated.

[0071] Step S1 is a process of obtaining scale features of four scales of the image to be evaluated through a preset semantic quality decoupling module, and reconstructing the pre-scale features of each scale feature based on the reconstruction network in the semantic quality decoupling module to obtain pre-reconstructed scale features of three dimensions; wherein,

[0072] The steps of obtaining scale features of four scales of the image to be evaluated through a preset semantic quality decoupling module, and reconstructing the pre-scale features of each scale feature based on the reconstruction network in the semantic quality decoupling module to obtain the pre-reconstructed scale features of three dimensions include:

[0073] S11: inputting the image to be evaluated into the backbone network of the semantic quality decoupling module, so that the pooling layer in the backbone network performs feature extraction on the image to be evaluated to obtain scale features of four scales, namely, a first scale feature, a second scale feature, a third scale feature and a fourth scale feature;

[0074] S12: Input the second scale feature, the third scale feature, and the fourth scale feature into the trained reconstruction network to obtain the pre-reconstruction scale features for predicting reconstruction: reconstructing one-scale features, reconstructing two-scale features, and reconstructing three-scale features.

[0075] Wherein, when using the method to evaluate the image to be evaluated, the reconstruction network used is pre-trained, and the steps of pre-training the reconstruction network include:

[0076] A preset number of images are selected to form a reference image set, and feature extraction is performed on the images in the reference image set to obtain scale features of four scales, which are called training scale features;

[0077] Construct a reconstruction network to be trained with parameter γ consisting of a fully connected layer The training scale feature X of the k+1th scale in the training scale feature k+1 ={x i ,i∈H x ×W x} Input to the reconstruction network to be trained In order to predict the pre-training scale feature Y of the k+1th scale k ={y i ,i∈H y ×W y}, and record the predicted reconstructed pre-training scale feature as the pre-reconstructed training scale feature Y k ′={y i ′,i∈H y ×W y}, where k = 1, 2, 3; H x ,W x and H y ,W y are the k+1th training scale features X k+1 and the kth training scale feature Y k or Y k ' height and width;

[0078] The pre-reconstruction training scale feature Y k ′ and the true pre-training scale feature Y k The Euclidean norm of is used as the loss function L; where L = ||Y k ′-Y k ||2;

[0079] Use Adam optimizer to update the reconstructed network to be trained Parameter γ, calculate the gradient of the loss function L with respect to the parameter γ Reconstruct the network to be trained by updating and correcting it The first and second moments of

[0080] Use the updated first-order moment and second-order moment to update the reconstructed network to be trained The parameter γ is adjusted until the loss function reaches a preset threshold, then the training is stopped and the last updated network to be trained is reconstructed As a reconstruction network

[0081] It should be noted that in this embodiment, since the same processing is performed on the image in both the training stage and the later use stage, the scale feature is also expressed by the same formula as the training scale feature. For example, the scale feature of the k+1th scale is expressed as X k+1 ={x i ,i∈H x ×W x}, the previous scale feature representation of the k+1th scale is Y k ={y i ,i∈H y ×W y}, the scale feature before reconstruction is represented by Y k ′={y i ′,i∈H y ×W y}, where k = 1, 2, 3; H x ,W x and H y ,W y are the k+1th scale features X k+1 and the k-th scale feature Y k or Y k ′height and width.

[0082] In a more specific embodiment, the training process of each module, especially the detailed process of training the semantic quality decoupling module is as follows:

[0083] First, 100,000 images without obvious quality distortion were selected from the KADID dataset to construct a reference image set R. All images in the image set were cropped to 384×512 with their aspect ratios maintained. Ten types of distortion, including Gaussian blur, motion blur, compression distortion, image noise, denoising distortion, and jitter, were applied to the reference image set, and two levels of degradation were performed to generate a "medium" image set M and a "poor" image set P. B images were randomly selected from the reference image set R and input into the backbone network f with a parameter of θ. θ (·), here we take ResNet50 as an example, you can also choose other backbone networks, such as Swin Transformer or InceptionV4, which has been pre-trained on the ImageNet dataset. θ After each pooling layer of (·), multi-scale features are extracted. Four-scale features are obtained for each of the B images. A reconstruction network with a parameter γ consisting of a fully connected layer is constructed. Compared with the backbone network composed of convolutional layers, it is a lighter network. k+1 ={x i ,i∈H x ×W x} Input to the reconstruction network In, it is used to predict the feature Y of the kth scale k ={y i ,i∈H y ×W y}, and record the reconstructed feature as Y k ′={y i ′,i∈H y ×W y}, where H x ,W x and Hy ,W y are k+1 scale features X respectively k+1 and the k-th scale feature Y k The loss function L uses the predicted value Y k ′ and the true value Y k The Euclidean norm is used to calculate, that is, L = || Y k ′-Y k ||2, use Adam optimizer to update and reconstruct the network The parameter γ of the loss function L is first calculated with respect to the gradient of γ. Update the first-order moment and the second-order moment and correct the deviation. More specifically, in this embodiment, the corrected first-order moment and the second-order moment can be used to update the parameter γ, the learning rate is set to 1e-4, and the weight decay is set to 5e-4; repeat steps 2.1 to 2.3 for the images in the reference image set R until all images are traversed, and the images with less than B images are discarded. At this time, the reconstruction network The training is completed, and the parameters after training are recorded as γ * , set the gradient of the reconstruction network parameters to zero to fix the reconstruction network The parameters in , so that these parameters will not be updated in the subsequent gradient descent steps. Randomly select an image I from the reference image set R R , through the backbone network f θ (·) and then use the feature (scale feature) X of the k+1th (k=1, 2, 3)th scale k+1 ={x i ,i∈H x ×W x} Input to the reconstruction network Predict the k-th scale feature (previous scale feature) Y k ={y i ,i∈H y ×W y}, the reconstructed feature (scale feature before reconstruction) is recorded as Y k ′={y i ′,i∈H y ×W y}.

[0084] It should be noted that no matter whether it is the (training) scale feature, the pre-(training) scale feature or the (training) pre-reconstruction scale feature, their essence is scale feature, but they are named for the convenience of distinction. The scale feature is a known scale feature, the pre-scale feature is a concept, which means the scale feature before the known scale feature, and the pre-reconstruction scale feature is the predicted pre-scale feature, which is the predicted scale feature.

[0085] Step S2 is a process of modeling the local conditional probabilities of each scale feature and the previous scale feature at the local position based on the reconstructed scale feature, and obtaining the overall conditional entropy of the scale feature and the previous scale feature according to the conditional probability; wherein,

[0086] The process of modeling the local conditional probabilities of each scale feature and the previous scale feature at a local position based on the pre-reconstructed scale feature, and obtaining the overall conditional entropy of the scale feature and the previous scale feature according to the conditional probability includes:

[0087] Based on the scale feature X k+1 With the previous scale feature Y k Perform Gaussian modeling to form a Gaussian model The pre-reconstruction scale feature y predicted by the reconstruction network is i ′ is used as the mean of the Gaussian model to reconstruct the pre-scale feature y i ′ and the previous scale feature y i The Euclidean norm of i as the standard deviation of the Gaussian model; I is the identity matrix of the Gaussian model;

[0088] Based on the Gaussian model By formula Calculate the scale feature X k+1 With the previous scale feature Y k The local conditional entropy at the local position i; where, Represents the reconstruction network;

[0089] For each local position i, the local conditional entropy h(y i |x i ) to obtain the scale feature X k+1 With the previous scale feature Y k The overall conditional entropy Among them, H y ,W y is the k-th scale feature Y k height and width.

[0090] For the convenience of description, in describing this specific embodiment, the scale feature, pre-scale feature and pre-reconstruction scale feature mentioned above are expressed in the form of "feature + expression". That is, when calculating entropy, first use the Gaussian model Modeling Feature X k+1 With feature Y k The conditional probability p(y i |x i ), where y i ′ is the mean of the Gaussian model, δ i Yes i′ and y i The Euclidean norm of , is the standard deviation of the Gaussian model, and I is the identity matrix.

[0091] By formula Calculate feature X k+1 With feature Y k The local conditional entropy at the local position i, where h(y i |x i ) represents the conditional entropy, that is, when x is known i Under the condition of y i The entropy of y i Obey y i ′ is the mean value, is the multivariate normal distribution of the covariance matrix, so the entropy expression of the normal distribution can be used, that is Where log represents the natural logarithm, 2πe is a constant, and δ i is the standard deviation of the Gaussian model. The local conditional entropy h(y i |x i ) is the exponential sum of the feature X k+1 With feature Y k The overall conditional entropy Indicates the quality characterization capability of the overall characteristics, where H y ,W y is the k-th scale feature Y k The height and width, h(y i |x i ) is the local conditional entropy at the local position i.

[0092] Step S3 is a process of obtaining a weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula, that is, in order to obtain a quality perception vector, it is necessary to first obtain a weight matrix corresponding to the image. Then, how to obtain the weight matrix is ​​to firstly build a model, and then introduce other formulas to solve it according to the formula generated by the modeling. Then, the step of obtaining the weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula includes:

[0093] S31: Previous scale feature Y k ={y i ,i∈H y ×W y} Perform spatial transformation to obtain the actual transformation feature Z after the k-th scale transformation k =ω k ×Y k ={ω i y i ,i∈H y ×W y};in, is the unknown weight matrix to be determined;

[0094] S32: According to the preset modeling optimization formula, the scale feature X of the k+1th scale is k+1 With the previous scale feature Y k The actual transformation feature Z after spatial transformation k For the overall conditional entropy H e (Z k |X k+1 ) is maximized to calculate the unknown weight matrix to be calculated; wherein the modeling optimization formula is:

[0095]

[0096] in, It means to find the parameter ω that makes the function in the brackets reach the maximum value. k , H e (ω k Y k |X k+1 ) is the scale feature X of the k+1th scale k+1 Actual transformation feature Z k The overall conditional entropy, H y ,W y is the actual transformation feature Z k The height and width of is the scale feature X of the k+1th scale k+1 Actual transformation feature Z k The local conditional entropy at position i, 2πe is a constant, e is the base of the natural logarithm, δ i is the scale feature y before reconstruction i ′ and the previous scale feature y i The Euclidean norm of , the sum after st is the preset constraint function, and c in the formula is a preset constant.

[0097] The process of solving the modeling optimization formula includes:

[0098] Constructing the solution function based on the Lagrange multiplier method where λ is the Lagrange multiplier, is the objective function of the modeling optimization formula, A constraint function for the modeling optimization formula;

[0099] when and When the scale feature X of the k+1th scale k+1 Actual transformation feature Z k The overall conditional entropy H e (ω k Y k |Xk+1 ) to obtain the maximum value;

[0100] Will and When the optimal weight of the modeling optimization formula is As the weight matrix; where δ i is the scale feature y before reconstruction i ′ and the previous scale feature y i The Euclidean norm of , c is a constant.

[0101] More specific embodiments are as follows:

[0102] First, the feature Y of the k-th scale k ={y i ,i∈H y ×W y} Perform spatial transformation to obtain the feature Z after the k-th scale transformation k =ω k ×Y k ={ω i y i ,i∈H y ×W y},in is the weight matrix, which is an unknown quantity. Maximize the feature X of the k+1th scale k+1 With Y k The feature Z after spatial transformation k The overall conditional entropy H e (Z k |X k+1 ) to increase the quality representation ability of the feature, thereby achieving the purpose of semantic quality decoupling. This task can be modeled as an optimization problem, as shown in Formula 1:

[0103]

[0104] in, It means to find the parameter ω that makes the function in the brackets reach the maximum value. k , H e (ω k Y k |X k+1 ) is X k+1 With Z k The overall conditional entropy, H y ,W y Characteristic Z k The height and width of For X k+1 With Z kThe local conditional entropy at position i is obtained by substituting the local conditional entropy formula in 2.7 into formula 1 to obtain the formula after the second equal sign, which is called the objective function. 2πe is a constant, where e is the base of the natural logarithm, and δ i Yes i ′ and y i The Euclidean norm of st is the constraint function, and c in the formula is a constant.

[0105] Then, according to the Lagrange multiplier method, the loss function is constructed where λ is the Lagrange multiplier, is the objective function in Formula 1, is the constraint function in Formula 1. and When feature X k+1 With feature Z k The overall conditional entropy H e (ω k Y k |X k+1 ) to obtain the maximum value, at this time let the optimal weight of formula 1 where δ i Yes i ′ and y i The Euclidean norm of , c is a constant. In this way, the weight matrix that maximizes the overall conditional entropy can be obtained.

[0106] Step S4 is a process of spatially transforming the pre-reconstruction scale features of the three dimensions by multiplying them with their respective corresponding weight matrices to obtain a first transformation feature, a second transformation feature, and a third transformation feature, and concatenating and pooling the first transformation feature, the second transformation feature, the third transformation feature, and the scale feature of the fourth scale to obtain a quality perception vector.

[0107] In a specific embodiment, for feature Y with scale k=1, 2, 3 k (Pre-scale feature or scale feature) Calculate the feature Z after spatial transformation k , the first, second, and third transformation features Z1, Z2, Z3 obtained after spatial transformation of the scale features of scale k = 1, 2, 3 and the scale feature X4 of k = 4 are spliced ​​in the channel dimension to obtain the spliced ​​features, and the global maximum pooling is used to pool them into a one-dimensional vector, which is the image to be evaluated I R The quality perception vector ft R .

[0108] Step S5 is a process of mapping and predicting the quality perception vector through a pre-trained quality regression network and a quality prediction network to obtain a quality prediction score of the image to be evaluated; wherein,

[0109] The step of mapping and predicting the quality perception vector by using a pre-trained quality regression network and a quality prediction network to obtain a quality prediction score of the image to be evaluated includes:

[0110] S51: Mapping the quality perception vector through a preset quality regression network to obtain a quality mapping feature;

[0111] S52: Inputting the quality mapping feature into a quality prediction network so that the quality prediction network performs activation mapping on the quality mapping feature to obtain a quality prediction score of the image to be evaluated.

[0112] The steps of training the quality regression network include:

[0113] Performing a cropping process of a specific specification on all images in the reference image set to form a standard image, and performing two levels of degradation processing on the standard image to obtain a medium image set and a poor image set;

[0114] Select B quality perception vectors ft of each image from the quality perception vector sets corresponding to the reference image set, the medium image set and the poor image set. R To form the reference feature vector Medium eigenvector Sum and Difference Eigenvectors

[0115] Construct a layer consisting of two fully connected layers and a Relu activation function with parameters φ P The quality regression network to be trained is repeatedly trained; wherein the process of repeated training is:

[0116] The reference feature vector, the medium feature vector and the difference feature vector are input into the regression network to be trained, so that the first fully connected layer maps the reference feature vector, the medium feature vector and the difference feature vector to a low-dimensional hidden layer to obtain a reference hidden layer output vector F R ′, medium hidden equal output vector F M ′ and difference hidden layer output vector F P ′, and the second fully connected layer performs mapping processing on the reference hidden layer output vector, the medium hidden layer output vector and the difference hidden layer output vector to obtain a reference quality mapping feature Medium quality mapping features Quality mapping features such as sum and difference

[0117] Use the optimizer to update the quality regression network to be trained The weight φP , calculate the preset quality regression loss function L proj About weight φ P Gradient To update the first-order moment and the second-order moment, and correct the deviation, and use the corrected first-order moment and second-order moment to update the weight φ P , until the regression loss function L proj When the preset regression loss function threshold is reached, the update is stopped and the last updated quality regression network to be trained is used as the quality regression network; the quality regression loss function L proj for:

[0118]

[0119] in, represents the cosine similarity between two vectors, α, β are hyperparameters that control the degree of feature aggregation, i,j∈[1,B].

[0120] In a specific embodiment, the reference image set R, the medium image set M and the poor image set P generated above are firstly used to obtain the quality feature set R of the reference image set R. ft Randomly select B feature vectors from From the set of “medium” images M, we have a set of quality features M ft and the quality feature set P of the “bad” image set P ft The corresponding distorted image features are selected and recorded as Construct a layer consisting of two fully connected layers and a Relu activation function with parameters φ P Quality regression network Compared with the backbone network, it is also a lightweight network. R , F M and F P At the same time, it is fed into the quality regression network Through the first fully connected layer of the regression network, the three input features are mapped to a lower-dimensional hidden layer respectively, and the hidden layer output F is obtained. R ′,F M ′ and F P ′. Then the Relu activation function is applied to the output of the hidden layer to obtain the activated hidden layer output. Finally, the hidden layer output passes through the second fully connected layer to obtain the quality mapping feature In subjective quality assessment, the scores of extremely poor and excellent images have high confidence. Based on this feature, the loss function L shown in Formula 2 is adopted. proj , use the Adam optimizer to update the quality regression network The weight φ P , first calculate the loss function L projAbout weight φ P Gradient Update the first-order moment and the second-order moment and correct the deviation, and finally use the corrected first-order moment and second-order moment to update the weight φ P , the learning rate is set to 1e-3, and the weight decay is set to 5e-4; Formula 2 is as follows

[0121]

[0122] In formula 2 is the cosine similarity between two vectors, α, β are hyperparameters that control the degree of feature aggregation, i,j∈[1,B].

[0123] The step of training the quality prediction network comprises:

[0124] Construct a layer consisting of two fully connected layers and a Relu activation function with parameters φ a The quality prediction network to be trained

[0125] Mapping reference mass to features Medium quality mapping features Quality mapping features such as sum and difference Input into the quality prediction network to be trained to generate a reference image quality score, a medium image quality score and a poor image quality score respectively;

[0126] A quality prediction loss function is created based on the reference image quality score, the medium image quality score and the poor image quality score, and the quality prediction network to be trained is repeatedly trained until the quality prediction loss function reaches a preset threshold, then the training is stopped, and the quality prediction network to be trained that was last updated is used as the quality prediction network.

[0127] In a specific embodiment, a layer consisting of two fully connected layers and a ReLU activation function with a parameter of φ is constructed. a Quality prediction network The quality mapping feature F R_proj , F M_proj and F P_proj Feed Quality Prediction Network The first fully connected layer maps the input features to a lower-dimensional hidden layer, and then applies the Relu activation function to the output of the hidden layer to obtain the activated hidden layer output. Finally, the activated hidden layer output passes through the second fully connected layer and is mapped to the image quality score. and

[0128] For images with the same content, the quality will decrease as the distortion level increases. Therefore, the loss function L shown in Formula 3 is used agr , use the Adam optimizer to update the quality prediction network The weight φ a , first calculate the loss function L agr About weight φ a Gradient Update the first-order moment and the second-order moment and correct the deviation, and finally use the corrected first-order moment and second-order moment to update the weight φ a , the learning rate is set to 1e-3, and the weight decay is set to 5e-4. In Formula 3, i∈[1,B].

[0129]

[0130] For the reference image set R, the feature set R of the “medium” image set M and the “poor” image set P ft , M ft and P ft Repeat steps 3.2 to 4.2 until all features are traversed, and the features that are less than B in the end are discarded. And fix the quality regression network by setting the gradient of the network parameters to zero and quality prediction network The parameters after training are and Therefore, the trained regression network can be used directly later.

[0131] The specific training and use process is as described above. In order to further highlight the use process, another specific embodiment is provided. Characters different from the above embodiment are used to represent each feature to clearly reflect the specific use logic. First, for an input image I test , first scale the input image to 384×512 and input it into the backbone network f θ (·) Extract features from the 1st to 4th scales Among them, H 1 , W 1 The height and width of the first scale feature are similar. Input features f2, f3, and f4 into the reconstruction network of step 2.3. Get the reconstructed features and Then the reconstructed features Multiply the pre-acquired weight matrices respectively to perform spatial transformation w to obtain the transformed features and The features after spatial transformation The concatenated features are concatenated with the feature f4 of scale 4 in the channel dimension, and then pooled into a one-dimensional vector ft using global maximum pooling, which is the image I test The quality perception vector ft is fed into the quality regression network Get the quality map feature ft proj . Then ft proj Feed into the quality prediction network in step 4 The output q of the network is the image I test The quality prediction score of .

[0132] As described above, the lightweight contrastive learning image quality assessment method based on quality decoupling provided by the present invention reconstructs the pre-scale features of each scale feature based on the reconstruction network in the semantic quality decoupling module to obtain the reconstructed pre-scale features of three dimensions, and then models the local conditional probabilities of each scale feature and the pre-scale feature at the local position based on the reconstructed pre-scale features, and obtains the overall conditional entropy of the scale feature and the pre-scale feature according to the conditional probability, and then obtains the weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula, so that the pre-scale features of the three dimensions are respectively multiplied by their respective corresponding weight matrices for spatial transformation to obtain the first transformation feature, the second transformation feature and the third transformation feature. The first transformation feature, the second transformation feature, the third transformation feature, and the scale feature of the fourth scale are concatenated and pooled to obtain a quality perception vector. The quality perception vector is mapped and predicted through a pre-trained quality regression network and a quality prediction network to obtain a quality prediction score of the image to be evaluated. In this way, a lightweight quality contrast learning strategy is adopted, which can be efficiently trained on a single GPU. The entire training process does not require the participation of subjective opinion scores, and the semantic quality decoupling model is modeled as an optimization problem. Only a lightweight reconstruction network needs to be introduced to calculate the feature weight map with the largest quality representation ability, thereby improving the objectivity of image quality evaluation, reducing resource waste, and improving the accuracy of quality evaluation.

[0133] like Figure 3 As shown, the present invention also provides a lightweight contrastive learning image quality assessment system 100 based on quality decoupling, which implements the lightweight contrastive learning image quality assessment method based on quality decoupling as described above, including a semantic quality decoupling module 101, a quality regression network 102 and a quality prediction network 103; wherein,

[0134] The semantic quality decoupling module 101 includes:

[0135] The backbone network 1011 is used to obtain scale features of four scales of the image to be evaluated;

[0136] A reconstruction network 1012 is used to reconstruct the pre-scale features of each scale feature to obtain the reconstructed pre-scale features of three dimensions;

[0137] The quality perception module 1013 is used to model the local conditional probabilities of each scale feature and the previous scale feature at the local position based on the pre-reconstruction scale feature, and obtain the overall conditional entropy of the scale feature and the previous scale feature according to the conditional probability; obtain the weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula; multiply the pre-reconstruction scale features of the three dimensions by their respective corresponding weight matrices for spatial transformation to obtain a first transformation feature, a second transformation feature and a third transformation feature, and perform splicing and pooling processing on the first transformation feature, the second transformation feature, the third transformation feature, and the scale feature of the fourth scale to obtain a quality perception vector;

[0138] The quality regression network 102 and the quality prediction network 103 are used to perform mapping and prediction processing on the quality perception vector to obtain the quality prediction score of the image to be evaluated.

[0139] The specific implementation method refers to the above-mentioned embodiment of the lightweight contrast learning image quality assessment method based on quality decoupling, which will not be described in detail here.

[0140] As described above, the lightweight contrastive learning image quality assessment system based on quality decoupling provided by the present invention reconstructs the pre-scale features of each scale feature based on the reconstruction network 1012 in the semantic quality decoupling module 101 to obtain the reconstructed pre-scale features of three dimensions, and then models the local conditional probabilities of each scale feature and the pre-scale feature at the local position based on the reconstructed pre-scale features, and obtains the overall conditional entropy of the scale feature and the pre-scale feature according to the conditional probability, and then obtains the weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula, so that the reconstructed pre-scale features of the three dimensions are respectively multiplied by their respective corresponding weight matrices to perform spatial transformation to obtain the first transformation feature, the second transformation feature and the third transformation feature. , and concatenate and pool the first transformation feature, the second transformation feature, the third transformation feature, and the scale feature of the fourth scale to obtain a quality perception vector, and map and predict the quality perception vector through the pre-trained quality regression network 102 and the quality prediction network 103 to obtain the quality prediction score of the image to be evaluated. In this way, a lightweight quality comparison learning strategy is adopted, which can be efficiently trained on a single GPU. The entire training process does not require the participation of subjective opinion scores, and the semantic quality decoupling model is modeled as an optimization problem. Only a lightweight reconstruction network needs to be introduced to calculate the feature weight map with the largest quality representation ability, thereby improving the objectivity of image quality evaluation, reducing resource waste, and improving the accuracy of quality evaluation.

[0141] As described above with reference to the accompanying drawings, the lightweight contrast learning image quality assessment method and system based on quality decoupling proposed in accordance with the present invention are described in an exemplary manner. However, those skilled in the art should understand that various improvements can be made to the lightweight contrast learning image quality assessment method and system based on quality decoupling proposed in the present invention without departing from the content of the present invention. Therefore, the protection scope of the present invention should be determined by the content of the attached claims.

Claims

1. A lightweight contrastive learning image quality assessment method based on quality decoupling, characterized in that: include: Acquire scale features of four scales of the image to be evaluated through a preset semantic quality decoupling module, and reconstruct pre-scale features of each scale feature based on a reconstruction network in the semantic quality decoupling module to obtain pre-reconstructed scale features of three dimensions; Modeling local conditional probabilities of each scale feature and the previous scale feature at a local position based on the pre-reconstructed scale feature, and obtaining overall conditional entropy of the scale feature and the previous scale feature according to the conditional probability; Obtaining a weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula; The pre-reconstruction scale features of the three dimensions are respectively multiplied by the corresponding weight matrices to perform spatial transformation to obtain a first transformation feature, a second transformation feature, and a third transformation feature, and the first transformation feature, the second transformation feature, the third transformation feature, and the scale feature of the fourth scale are concatenated and pooled to obtain a quality perception vector; The quality perception vector is mapped and predicted by a pre-trained quality regression network and a quality prediction network to obtain a quality prediction score of the image to be evaluated.

2. The lightweight contrastive learning image quality assessment method based on quality decoupling according to claim 1, characterized in that: The steps of obtaining scale features of four scales of the image to be evaluated through a preset semantic quality decoupling module, and reconstructing the pre-scale features of each scale feature based on the reconstruction network in the semantic quality decoupling module to obtain the pre-reconstructed scale features of three dimensions include: Inputting the image to be evaluated into the backbone network of the semantic quality decoupling module, so that the pooling layer in the backbone network performs feature extraction on the image to be evaluated to obtain scale features of four scales, namely, a first scale feature, a second scale feature, a third scale feature and a fourth scale feature; The second scale feature, the third scale feature, and the fourth scale feature are input into the trained reconstruction network. to obtain the pre-reconstruction scale features for predicting reconstruction: reconstructing one-scale features, reconstructing two-scale features, and reconstructing three-scale features.

3. The lightweight contrastive learning image quality assessment method based on quality decoupling according to claim 2, characterized in that: The step of pre-training the reconstruction network comprises: A preset number of images are selected to form a reference image set, and feature extraction is performed on the images in the reference image set to obtain scale features of four scales, which are called training scale features; Construct a reconstruction network to be trained with parameter γ consisting of a fully connected layer The training scale feature X of the k+1th scale in the training scale feature k+1 ={x i ,i∈H x ×W x } Input to the reconstruction network to be trained In order to predict the pre-training scale feature Y of the k+1th scale k ={y i ,i∈H y ×W y }, and record the predicted reconstructed pre-training scale feature as the pre-reconstructed training scale feature Y k ′={y i ′,i∈H y ×W y }, where k = 1, 2, 3; H x ,W x and H y ,W y are the k+1th training scale features X k+1 and the kth training scale feature Y k or Y k ' height and width; The pre-reconstruction training scale feature Y′ k With the true pre-training scale feature Y k The Euclidean norm of is used as the loss function L; where L = ||Y′ k -Y k ||2; Use Adam optimizer to update the reconstructed network to be trained Parameter γ, calculate the gradient of the loss function L with respect to the parameter γ Reconstruct the network to be trained by updating and correcting it The first and second moments of Use the updated first-order moment and second-order moment to update the reconstructed network to be trained The parameter γ is adjusted until the loss function reaches a preset threshold, then the training is stopped and the last updated network to be trained is reconstructed As a reconstruction network 4. The lightweight contrastive learning image quality assessment method based on quality decoupling according to claim 3 is characterized in that: The process of modeling the local conditional probabilities of each scale feature and the previous scale feature at a local position based on the pre-reconstructed scale feature, and obtaining the overall conditional entropy of the scale feature and the previous scale feature according to the conditional probability includes: Based on the scale feature X k+1 With the previous scale feature Y k Perform Gaussian modeling to form a Gaussian model The pre-reconstruction scale feature y predicted by the reconstruction network is i ′ is used as the mean of the Gaussian model to reconstruct the pre-scale feature y i ′ and the previous scale feature y i The Euclidean norm of i as the standard deviation of the Gaussian model; I is the identity matrix of the Gaussian model; Based on the Gaussian model By formula Calculate the scale feature X k+1 With the previous scale feature Y k The local conditional entropy at the local position i; where, Stands for Rebuilding the Network. For each local position i, the local conditional entropy h(y i |x i ) to obtain the scale feature X k+1 With the previous scale feature Y k The overall conditional entropy Among them, H y ,W y is the k-th scale feature Y k height and width.

5. The lightweight contrastive learning image quality assessment method based on quality decoupling according to claim 4, characterized in that: The step of obtaining a weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula includes: The previous scale feature Y k ={y i ,i∈H y ×W y } Perform spatial transformation to obtain the actual transformation feature Z after the k-th scale transformation k =ω k ×Y k ={ω i y i ,i∈H y ×W y };in, is the unknown weight matrix to be determined; Through the preset modeling optimization formula, the scale feature X of the k+1th scale is k+1 The actual transformed feature Zk after spatial transformation with the previous scale feature Yk has an effect on the overall conditional entropy H e (Z k |X k+1 ) is maximized to calculate the unknown weight matrix to be calculated; wherein the modeling optimization formula is: in, It means to find the parameter ω that makes the function in the brackets reach the maximum value. k , H e (ω k Y k |X k+1 ) is the scale feature X of the k+1th scale k+1 Actual transformation feature Z k The overall conditional entropy, H y ,W y is the actual transformation feature Z k The height and width of is the scale feature X of the k+1th scale k+1 Actual transformation feature Z k The local conditional entropy at position i, 2πe is a constant, e is the base of the natural logarithm, δ i is the scale feature y before reconstruction i ′ and the previous scale feature y i The Euclidean norm of , the sum after st is the preset constraint function, and c in the formula is a preset constant.

6. The lightweight contrastive learning image quality assessment method based on quality decoupling according to claim 5, characterized in that: The process of solving the modeling optimization formula includes: Constructing the solution function based on the Lagrange multiplier method where λ is the Lagrange multiplier, is the objective function of the modeling optimization formula, A constraint function for the modeling optimization formula; when and When the scale feature X of the k+1th scale k+1 Actual transformation feature Z k The overall conditional entropy H e (ω k Y k |X k+1 ) to obtain the maximum value; Will and When the optimal weight of the modeling optimization formula is As the weight matrix; where δ i is the scale feature y before reconstruction i ′ and the previous scale feature y i The Euclidean norm of , c is a constant.

7. The lightweight contrastive learning image quality assessment method based on quality decoupling according to claim 6, characterized in that: The step of mapping and predicting the quality perception vector by using a pre-trained quality regression network and a quality prediction network to obtain a quality prediction score of the image to be evaluated includes: Performing mapping processing on the quality perception vector through a preset quality regression network to obtain quality mapping features; The quality map feature is input into a quality prediction network so that the quality prediction network performs activation mapping on the quality map feature to obtain a quality prediction score of the image to be evaluated.

8. The lightweight contrastive learning image quality assessment method based on quality decoupling according to claim 7, characterized in that: The steps of training the quality regression network include: Performing a cropping process of a specific specification on all images in the reference image set to form a standard image, and performing two levels of degradation processing on the standard image to obtain a medium image set and a poor image set; Select B quality perception vectors ft of each image from the quality perception vector sets corresponding to the reference image set, the medium image set and the poor image set. R To form the reference feature vector Medium eigenvector Sum and Difference Eigenvectors Construct a layer consisting of two fully connected layers and a Relu activation function with parameters φ P The quality regression network to be trained is repeatedly trained; wherein the process of repeated training is: The reference feature vector, the medium feature vector and the difference feature vector are input into the regression network to be trained, so that the first fully connected layer maps the reference feature vector, the medium feature vector and the difference feature vector to a low-dimensional hidden layer to obtain a reference hidden layer output vector F R ′, medium hidden equal output vector F M ′ and difference hidden layer output vector F P ′, and the second fully connected layer performs mapping processing on the reference hidden layer output vector, the medium hidden layer output vector and the difference hidden layer output vector to obtain a reference quality mapping feature Medium quality mapping features Quality mapping features such as sum and difference Use the optimizer to update the quality regression network to be trained The weight φ P , calculate the preset quality regression loss function L proj About weight φ P Gradient To update the first-order moment and the second-order moment, and correct the deviation, and use the corrected first-order moment and second-order moment to update the weight φ P , until the regression loss function L proj When the preset regression loss function threshold is reached, the update is stopped and the last updated quality regression network to be trained is used as the quality regression network; the quality regression loss function L proj for: in, represents the cosine similarity between two vectors, α, β are hyperparameters that control the degree of feature aggregation, i,j∈[1,B].

9. The lightweight contrastive learning image quality assessment method based on quality decoupling according to claim 8, characterized in that: The step of training the quality prediction network comprises: Construct a layer consisting of two fully connected layers and a Relu activation function with parameters φ a The quality prediction network to be trained Mapping reference mass to features Medium quality mapping features Quality mapping features such as sum and difference Input into the quality prediction network to be trained to generate a reference image quality score, a medium image quality score and a poor image quality score respectively; A quality prediction loss function is created based on the reference image quality score, the medium image quality score and the poor image quality score, and the quality prediction network to be trained is repeatedly trained until the quality prediction loss function reaches a preset threshold, then the training is stopped, and the quality prediction network to be trained that was last updated is used as the quality prediction network.

10. A lightweight contrastive learning image quality assessment system based on quality decoupling, implementing the lightweight contrastive learning image quality assessment method based on quality decoupling as described in any one of claims 1 to 9, the system comprising a semantic quality decoupling module, a quality regression network and a quality prediction network; wherein, The semantic quality decoupling module includes: The backbone network is used to obtain the scale features of the four scales of the image to be evaluated; A reconstruction network is used to reconstruct the pre-scale features of each scale feature to obtain the reconstructed pre-scale features of three dimensions; A quality perception module, used for modeling the local conditional probabilities of each scale feature and the previous scale feature at a local position based on the pre-reconstruction scale feature, and obtaining the overall conditional entropy of the scale feature and the previous scale feature according to the conditional probability; obtaining a weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula; multiplying the pre-reconstruction scale features of the three dimensions with their respective corresponding weight matrices for spatial transformation to obtain a first transformation feature, a second transformation feature, and a third transformation feature, and performing splicing and pooling processing on the first transformation feature, the second transformation feature, the third transformation feature, and the scale feature of the fourth scale to obtain a quality perception vector; The quality regression network and the quality prediction network are used to perform mapping and prediction processing on the quality perception vector to obtain the quality prediction score of the image to be evaluated.

Images