Lightweight contrastive learning image quality assessment method and system based on quality decoupling

Abstract

The application provides a lightweight contrast learning image quality evaluation method and system based on quality decoupling, reconstructs the pre-scale features of each scale feature based on a reconstruction network in a semantic quality decoupling module to obtain three-dimensional reconstructed pre-scale features, models the local conditional probability of each scale feature and the pre-scale feature at a local position, obtains the overall conditional entropy of the scale feature and the pre-scale feature according to the conditional probability, obtains a weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula, makes the reconstructed pre-scale features respectively multiply the weight matrix to perform spatial transformation, and performs feature splicing, pooling and other processing to obtain a quality perception vector, and then maps and predicts the quality perception vector through a quality regression network and a quality prediction network to obtain a quality prediction score of the image to be evaluated. The whole training process does not need to involve subjective opinion scores, improves the objectivity of image quality evaluation, and reduces resource waste.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to an image quality evaluation method, more particularly to a lightweight contrast learning image quality evaluation method and system based on quality decoupling. BACKGROUND

[0002] Image quality assessment (IQA) aims to understand and evaluate the perception state of human when interpreting images. It can not only measure and optimize the quality of experience of video transmission system, but also promote the development of visual tasks such as image repair, image generation and super-resolution. With the powerful representation ability of deep neural network, no-reference image quality assessment has achieved great success. This kind of method can evaluate image quality without the participation of reference images.

[0003] Due to the long time consumption of subjective quality evaluation experiment, the scale of images with MOS annotation is relatively limited. To cope with this small sample learning problem, many IQA methods use a deep visual backbone network pre-trained on other visual tasks, and add a quality regression network for end-to-end supervised training. Due to the differences in experimental conditions, equipment and data processing procedures involved in the establishment process of different data sets, the MOS distribution is inconsistent. When the model is transferred to a new data set, it needs to be retrained. To improve the generalization ability and robustness of the BIQA model, some methods train the visual backbone network in a self-supervised contrast learning manner. These methods only need to train the quality regression network in a supervised learning manner when migrating to a new data set, without the need to retrain the deep backbone network. However, since these labels are inaccurate when applied to real distortion data, it is not suitable to learn features from distortion type labels.

[0004] In short, existing BIQA models are mostly trained in a supervised learning manner, which requires high quality and quantity of MOS in subjective experiments, and needs to be retrained in an end-to-end supervised manner when the model is migrated to a new data set. To enhance the generalization ability and robustness of the BIQA model, some recent methods train the backbone network in a self-supervised manner through contrast learning. When testing on a new data set, only the regression network needs to be supervised. However, these contrast learning-based methods consume a large amount of computing resources during training, which cannot be applied to resource-constrained environments.

[0005] Therefore, there is an urgent need for a lightweight contrast learning image quality evaluation method and system based on quality decoupling, which can not only be efficiently trained on a single GPU, but also does not require any MOS data during the entire training process, can reduce resource waste, and improve the accuracy of quality evaluation. SUMMARY

[0006] In view of the above problems, the purpose of the present application is to provide a lightweight contrast learning image quality evaluation method based on quality decoupling to solve the problem that the prior art is not accurate when applied to real distortion data, consumes a large amount of computing resources in the training process, and cannot be applied to resource-limited environments.

[0007] The present application provides a lightweight contrast learning image quality evaluation method based on quality decoupling, which comprises:

[0008] The four scale features of the image to be evaluated are obtained through a preset semantic quality decoupling module, and the pre-scale features of each scale feature are reconstructed based on the reconstruction network in the semantic quality decoupling module to obtain three-dimensional reconstructed pre-scale features.

[0009] The local conditional probability of each scale feature and the pre-scale feature at the local position is modeled based on the reconstructed pre-scale features, and the overall conditional entropy of the scale feature and the pre-scale feature is obtained according to the conditional probability.

[0010] The weight matrix that maximizes the overall conditional entropy is obtained through a preset modeling optimization formula.

[0011] The three-dimensional reconstructed pre-scale features are multiplied by their respective weight matrices for spatial transformation to obtain first, second and third transformed features, and the first, second, third transformed features and the scale features of the fourth scale are spliced and pooled to obtain a quality perception vector.

[0012] The quality perception vector is mapped and predicted by a pre-trained quality regression network and a quality prediction network to obtain the quality prediction score of the image to be evaluated.

[0013] Preferably, the step of obtaining four scale features of the image to be evaluated through a preset semantic quality decoupling module, and reconstructing the pre-scale features of each scale feature based on the reconstruction network in the semantic quality decoupling module to obtain three-dimensional reconstructed pre-scale features, comprises:

[0014] The image to be evaluated is input into the backbone network of the semantic quality decoupling module, and the pooling layer in the backbone network extracts features from the image to be evaluated to obtain four scale features, namely first, second, third and fourth scale features.

[0015] The second, third and fourth scale features are input into the trained reconstruction network In order to obtain the pre-reconstruction scale features for predictive reconstruction: reconstruction scale one features, reconstruction scale two features, and reconstruction scale three features.

[0016] Preferably, the step of pre-training the reconstruction network includes:

[0017] A predetermined number of images are selected to form a reference image set, and feature extraction is performed on the images in the reference image set to obtain scale features at four scales, which are called training scale features.

[0018] Construct a training reconstruction network with parameter γ consisting of fully connected layers. The training scale feature X of the (k+1)th scale in the training scale features. k+1 ={x i ,i∈H x ×W x Input to the network to be trained and reconstructed In the middle, the pre-trained scale features Y are used to predict the (k+1)th scale. k ={y i ,i∈H y ×W y The predicted pre-training scale features are denoted as the pre-training scale features Y after reconstruction. k ′={y i ′,i∈H y ×W y}, where k = 1, 2, 3; H x W x and H y W y These are the (k+1)th training scale features X k+1 With the k-th training scale feature Y k Or Y k The height and width of ′;

[0019] The pre-reconstruction training scale feature Y k ′ and the true pre-training scale features Y k The Euclidean norm is used as the loss function L; where L = ||Y k ′-Y k ||2;

[0020] Update the network to be rebuilt using the Adam optimizer. Given the parameter γ, calculate the gradient of the loss function L with respect to the parameter γ. To update and correct the network to be trained and reconstructed The first and second moments;

[0021] The network to be trained and reconstructed is updated using the corrected first and second moments. The parameter γ is adjusted until the loss function reaches a preset threshold, at which point training stops, and the network is reconstructed using the last updated data. As a network reconstruction

[0022] Preferably, the process of modeling the local conditional probabilities of each scale feature and the pre-scale feature at a local location based on the reconstructed pre-scale features, and obtaining the overall conditional entropy of the scale features and the pre-scale features according to the conditional probabilities, includes:

[0023] Based on scale feature X k+1 Compared with pre-scale features Y k Perform Gaussian modeling to form a Gaussian model Among them, the pre-reconstruction scale features y predicted by the reconstruction network are... i ′, as the mean of the Gaussian model, will reconstruct the pre-scale features y i ′ and prescale features y i The Euclidean norm δ i I is the standard deviation of the Gaussian model; I is the identity matrix of the Gaussian model.

[0024] Based on the Gaussian model Through formula Calculate the scale feature X k+1 Compared with pre-scale features Y k The local conditional entropy at local position i; where, Represents rebuilding the network;

[0025] The local conditional entropy h(y) at each local location i i |x i The scale feature X is obtained by summing the exponents of ). k+1 Compared with pre-scale features Y k Overall conditional entropy Where H y W y For the k-th scale feature Y k The height and width.

[0026] Preferably, the step of obtaining the weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula includes:

[0027] For prescale features Y k ={y i ,i∈H y ×W y Perform spatial transformation to obtain the actual transformation feature Z after the k-th scale transformation. k =ω k ×Y k ={ω i yi ,i∈H y ×W y};in, The weight matrix is ​​unknown and needs to be determined.

[0028] The modeling optimization formula is based on the scale feature X at the (k+1)th scale. k+1 The actual transformed feature Zk after spatial transformation of the pre-scale feature Yk with respect to the global conditional entropy H e (Z k |X k+1 The modeling optimization formula is as follows: (The formula is to be maximized to calculate the unknown weight matrix.)

[0029]

[0030] in, This indicates the search for the parameter ω that maximizes the function within the parentheses. k H e (ω k Y k |X k+1 ) represents the scale feature X at the (k+1)th scale. k+1 Actual transformation feature Z k The overall conditional entropy, H y W y For the actual transformation feature Z k Height and width, The scale feature X at the (k+1)th scale k+1 Actual transformation feature Z k The local conditional entropy at position i, where 2πe is a constant, e is the base of the natural logarithm, and δ i To reconstruct prescale features y i ′ and prescale features y i The Euclidean norm of , the summation after st is the preset constraint function, and c in the formula is the preset constant.

[0031] Preferably, the process of solving the modeling optimization formula includes:

[0032] The solution function is constructed based on the Lagrange multiplier method. Where λ is a Lagrange multiplier. The objective function of the modeling optimization formula is... The constraint function for the modeling optimization formula;

[0033] when and At that time, the scale feature X at the (k+1)th scale k+1 Actual transformation feature Z k The overall conditional entropy H e(ω k Y k |X k+1 ) reaches maximum value;

[0034] When and , the optimal weight of the modeling optimization formula is as a weight matrix; wherein δ i is the Euclidean norm of the pre-reconstruction scale feature y i ′ and the pre-reconstruction scale feature y i , and c is a constant.

[0035] Preferably, the step of mapping and predicting the quality perception vector by the pre-trained quality regression network, the quality prediction network to obtain the quality prediction score of the image to be evaluated, comprises:

[0036] Mapping the quality perception vector by the pre-set quality regression network to obtain quality mapping features;

[0037] Inputting the quality mapping features into the quality prediction network to enable the quality prediction network to perform activation mapping on the quality mapping features to obtain the quality prediction score of the image to be evaluated.

[0038] Preferably, the step of training the quality regression network comprises:

[0039] Performing specific specification cropping processing on all images in the reference image set to form specification images, and performing two-level degradation processing on the specification images to obtain a medium image set and a poor image set;

[0040] Selecting B quality perception vectors ft R of images from the quality perception vector sets corresponding to the reference image set, the medium image set and the poor image set, respectively, to form a reference feature vector a medium feature vector and a poor feature vector

[0041] Constructing a to-be-trained quality regression network with two fully connected layers and a Relu activation function, the to-be-trained quality regression network having parameters φ P , and repeatedly training the to-be-trained quality regression network; wherein the process of repeated training is:

[0042] Inputting the reference feature vector, the medium feature vector and the poor feature vector into the to-be-trained regression network, and enabling the first fully connected layer to map the reference feature vector, the medium feature vector and the poor feature vector to a low-dimensional hidden layer to obtain a reference hidden layer output vector F RThe output vector F with medium hiding time is F'. M The output vector F of the hidden layer is equal to the sum and difference. P The second fully connected layer maps the output vector of the reference hidden layer, the output vector of the intermediate hidden layer, and the output vector of the inferior hidden layer to obtain the reference quality mapping feature. Medium quality mapping features Sum and difference quality mapping features

[0043] Update the untrained quality regression network using an optimizer. weight φ P Calculate the preset quality regression loss function L proj Regarding the weight φ P gradient This updates the first and second moments, corrects the biases, and updates the weights φ of the corrected first and second moments. P Until the regression loss function L proj If the preset regression loss function threshold is reached, updates stop, and the last updated quality regression network to be trained is used as the quality regression network; where the quality regression loss function L... proj for:

[0044]

[0045] in, The cosine similarity between two vectors is represented by α and β, which are hyperparameters that control the degree of feature aggregation, i,j∈[1,B].

[0046] Preferably, the step of training the quality prediction network includes:

[0047] Construct a layer with parameters φ consisting of two fully connected layers and a ReLU activation function. a The quality prediction network to be trained

[0048] Map the reference quality features Medium quality mapping features Sum and difference quality mapping features The input is fed into the quality prediction network to be trained to generate reference image quality scores, medium image quality scores, and poor image quality scores, respectively.

[0049] A quality prediction loss function is created based on the reference image quality score, the medium image quality score, and the poor image quality score. The quality prediction network to be trained is repeatedly trained until the quality prediction loss function reaches a preset threshold. Then, training stops, and the last updated quality prediction network to be trained is used as the quality prediction network.

[0050] The application also provides a lightweight contrast learning image quality evaluation system based on quality decoupling, which implements the lightweight contrast learning image quality evaluation method based on quality decoupling as described above, and comprises a semantic quality decoupling module, a quality regression network and a quality prediction network.

[0051] The semantic quality decoupling module comprises:

[0052] a backbone network configured to obtain scale features of four scales of an image to be evaluated;

[0053] a reconstruction network configured to reconstruct a front scale feature of each scale feature to obtain reconstructed front scale features of three dimensions;

[0054] a quality perception module configured to model a local conditional probability of each scale feature and a front scale feature at a local position based on the reconstructed front scale features, and obtain an overall conditional entropy of the scale features and the front scale features according to the conditional probability; obtain a weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula; multiply the reconstructed front scale features of the three dimensions with the respective corresponding weight matrices respectively to perform spatial transformation to obtain first transformed features, second transformed features and third transformed features, and perform splicing and pooling processing on the first transformed features, the second transformed features, the third transformed features and the scale features of the fourth scale to obtain a quality perception vector;

[0055] The quality regression network and the quality prediction network are configured to perform mapping and prediction processing on the quality perception vector to obtain a quality prediction score of the image to be evaluated.

[0056] As can be seen from the above technical solution, the present invention provides a lightweight contrastive learning image quality assessment method and system based on quality decoupling. It reconstructs pre-scale features of various scales based on the reconstruction network in the semantic quality decoupling module to obtain three-dimensional reconstructed pre-scale features. Then, it models the local conditional probabilities of each scale feature and the pre-scale feature at local locations based on the reconstructed pre-scale features, and obtains the overall conditional entropy of the scale features and the pre-scale features according to the conditional probabilities. Finally, it obtains a weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula, and performs spatial transformation by multiplying the three-dimensional reconstructed pre-scale features with their respective weight matrices to obtain the first transformed feature, the second transformed feature, and the third transformed feature. The three transformation features, along with the first, second, and third transformation features and the scale feature of the fourth scale, are concatenated and pooled to obtain a quality-perceived vector. A pre-trained quality regression network and quality prediction network are then used to map and predict the quality-perceived vector to obtain a quality prediction score for the image to be evaluated. This lightweight quality contrast learning strategy allows for efficient training on a single GPU. The entire training process does not require subjective opinion scores, and semantic quality is decoupled and modeled as an optimization problem. Only a lightweight reconstruction network is needed to calculate the feature weight map with the greatest quality representation capability, improving the objectivity of image quality evaluation, reducing resource waste, and increasing the accuracy of quality evaluation. Attached Figure Description

[0057] Other objects and results of the invention will become more apparent and readily understood by referring to the following description taken in conjunction with the accompanying drawings, and with a more complete understanding of the invention. In the drawings:

[0058] Fig. 1 This is a flowchart of a lightweight contrastive learning image quality assessment method based on quality decoupling according to an embodiment of the present invention;

[0059] Fig. 2 This is a schematic diagram of semantic quality decoupling in a lightweight contrastive learning image quality assessment method based on quality decoupling according to an embodiment of the present invention.

[0060] Fig. 3 This is a system block diagram of a lightweight contrastive learning image quality assessment system based on quality decoupling according to an embodiment of the present invention. Detailed Implementation

[0061] Most of the existing BIQA models are trained in a supervised learning manner, which requires a large amount of quality and quantity of Mean Opinion Score (MOS) in subjective experiments, and needs to be retrained end-to-end when the model is migrated to a new dataset. To enhance the generalization ability and robustness of the BIQA model, some recent methods train the backbone network in a self-supervised manner through contrastive learning, and only need to supervise the training of the regression network when testing on a new dataset. However, these contrastive learning-based methods consume a large amount of computing resources during training and cannot be applied to resource-constrained environments.

[0062] To solve the above problems, the present application provides a lightweight contrastive learning image quality evaluation method and system based on quality decoupling, which will be described in detail below in conjunction with the accompanying drawings.

[0063] To illustrate the lightweight contrastive learning image quality evaluation method and system based on quality decoupling provided by the present application, Figs. 1-3 The embodiments of the present application are exemplarily indicated.

[0064] The following exemplary embodiments are actually only illustrative, but never as any limitation on the present application and its applications or uses. Techniques and devices known to those of ordinary skill in the relevant art can not be discussed in detail, but should be considered as part of the specification when appropriate.

[0065] As Fig. 1 shown, the present application provides a lightweight contrastive learning image quality evaluation method based on quality decoupling, comprising:

[0066] S1: obtaining four scale features of a to-be-evaluated image through a preset semantic quality decoupling module, and reconstructing a pre-scale feature of each scale feature based on a reconstruction network in the semantic quality decoupling module to obtain three-dimensional reconstructed pre-scale features;

[0067] S2: modeling a local conditional probability of each scale feature and pre-scale feature at a local position based on the reconstructed pre-scale features, and obtaining an overall conditional entropy of the scale features and the pre-scale features according to the conditional probability;

[0068] S3: obtaining a weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula;

[0069] S4: multiplying the pre-reconstruction scale features of the three dimensions with the respective corresponding weight matrix respectively to obtain a first transformed feature, a second transformed feature and a third transformed feature, and performing splicing and pooling processing on the first transformed feature, the second transformed feature, the third transformed feature, and the scale feature of the fourth scale to obtain a quality perception vector;

[0070] S5: performing mapping and prediction processing on the quality perception vector by using a pre-trained quality regression network and a quality prediction network to obtain a quality prediction score of the image to be evaluated.

[0071] Step S1 is a process of obtaining four scale features of an image to be evaluated by using a preset semantic quality decoupling module, and reconstructing pre-scale features of each scale feature based on a reconstruction network in the semantic quality decoupling module to obtain pre-reconstruction scale features of three dimensions; wherein,

[0072] The step of obtaining four scale features of an image to be evaluated by using a preset semantic quality decoupling module, and reconstructing pre-scale features of each scale feature based on a reconstruction network in the semantic quality decoupling module to obtain pre-reconstruction scale features of three dimensions, comprises:

[0073] S11: inputting the image to be evaluated into a backbone network of the semantic quality decoupling module, and enabling a pooling layer in the backbone network to extract features of the image to be evaluated to obtain four scale features, which are a first scale feature, a second scale feature, a third scale feature and a fourth scale feature respectively;

[0074] S12: inputting the second scale feature, the third scale feature and the fourth scale feature into a trained reconstruction network to obtain pre-reconstruction scale features: a reconstruction one scale feature, a reconstruction two scale feature and a reconstruction three scale feature.

[0075] When the method is used to evaluate an image to be evaluated, the reconstruction network used is pre-trained. The step of pre-training the reconstruction network comprises:

[0076] A preset number of images are selected and constructed into a reference image set, and features of the images in the reference image set are extracted to obtain four scale features, which are referred to as training scale features;

[0077] A to-be-trained reconstruction network with a parameter of γ is constructed, which is composed of a full connection layer The training scale feature X k+1 k+1 i in the training scale features is inputted into the to-be-trained reconstruction network x ×W xInput to the network to be trained and reconstructed In the middle, the pre-trained scale features Y are used to predict the (k+1)th scale. k ={y i ,i∈H y ×W y The predicted pre-training scale features are denoted as the pre-training scale features Y after reconstruction. k ′={y i ′,i∈H y ×W y}, where k = 1, 2, 3; H x W x and H y W y These are the (k+1)th training scale features X k+1 With the k-th training scale feature Y k Or Y k The height and width of ′;

[0078] The pre-reconstruction training scale feature Y k ′ and the true pre-training scale features Y k The Euclidean norm is used as the loss function L; where L = ||Y k ′-Y k ||2;

[0079] Update the network to be rebuilt using the Adam optimizer. Given the parameter γ, calculate the gradient of the loss function L with respect to the parameter γ. To update and correct the network to be trained and reconstructed The first and second moments;

[0080] The network to be trained and reconstructed is updated using the corrected first and second moments. The parameter γ is adjusted until the loss function reaches a preset threshold, at which point training stops, and the network is reconstructed using the last updated data. As a network reconstruction

[0081] It should be noted that in this embodiment, since the same processing is performed on the image in both the training and later use phases, the scale features are represented by the same formula as the training scale features. For example, the scale feature of the (k+1)th scale is represented as X. k+1 ={x i ,i∈H x ×W x}, the prescale feature representation Y at the (k+1)th scale k ={y i ,i∈H y ×W y}, the pre-reconstruction scale features are represented as Y k ′={y i ′,i∈H y ×W y}, where k = 1, 2, 3; H x W x and H y W y X represents the (k+1)th scale feature. k+1 With the k-th scale feature Y k Or Y k The height and width of ′.

[0082] In a more specific embodiment, the training process of each module, especially the detailed process of training the semantic quality decoupling module, is as follows:

[0083] First, 100,000 images without significant quality distortion were selected from the KADID dataset to construct a reference image set R. All images in this set were cropped to 384×512 pixels while maintaining their aspect ratio. Ten distortion types were applied—Gaussian blur, motion blur, compression distortion, image noise, denoising distortion, and jitter—to perform two levels of quality degradation on the reference image set, generating a "medium" image set M and a "poor" image set P. B images were then randomly selected from the reference image set R and input into a backbone network f with parameter θ. θ (·), Here, ResNet50 is used as an example, but other backbone networks can also be chosen, such as Swing Transformer or InceptionV4. This network has already been pre-trained on the ImageNet dataset. From the backbone network f θ After each pooling layer (·), multi-scale features are extracted, resulting in four scale features for each of the B images. A reconstruction network with parameter γ consisting of fully connected layers is then constructed. Compared to backbone networks composed of convolutional layers, it is a more lightweight network. The features X at the (k+1)th scale (k = 1, 2, 3) are... k+1 ={x i ,i∈H x ×W x Input to the reconstructed network In the middle, the feature Y is used to predict the k-th scale. k ={y i ,i∈H y ×W y Let Y be the reconstructed feature. k ′={y i ′,i∈H y ×W y}, where H x W x and Hy ,W y respectively, are the height and width of the k+1th scale feature X k+1 k k k k k * R θ k+1 i x x k i y y k i y y

[0084]

[0085] ​​​​​​​​​​​​​​​​​​​​​​​​​​​Step S2 involves modeling the local conditional probabilities of each scale feature and the pre-scale feature at local locations based on the reconstructed pre-scale features, and obtaining the overall conditional entropy of the scale features and the pre-scale features according to the conditional probabilities; wherein,

[0086] The process of modeling the local conditional probabilities of each scale feature and the pre-scale feature at local locations based on the reconstructed pre-scale features, and obtaining the overall conditional entropy of the scale features and the pre-scale features based on the conditional probabilities, includes:

[0087] Based on scale feature X k+1 Compared with pre-scale features Y k Perform Gaussian modeling to form a Gaussian model Among them, the pre-reconstruction scale features y predicted by the reconstruction network are... i ′, as the mean of the Gaussian model, will reconstruct the pre-scale features y i ′ and prescale features y i The Euclidean norm δ i I is the standard deviation of the Gaussian model; I is the identity matrix of the Gaussian model.

[0088] Based on the Gaussian model Through formula Calculate the scale feature X k+1 Compared with pre-scale features Y k The local conditional entropy at local position i; where, Represents rebuilding the network;

[0089] The local conditional entropy h(y) at each local location i i |x i The scale feature X is obtained by summing the exponents of ). k+1 Compared with pre-scale features Y k Overall conditional entropy Where H y W y For the k-th scale feature Y k The height and width.

[0090] For ease of description, in this specific embodiment, the aforementioned scale features, pre-scale features, and reconstructed pre-scale features are represented in the form of "feature + expression." That is, when calculating entropy, a Gaussian model is first used. Modeling feature X k+1 With feature Y k The conditional probability p(y) at local location i i |x i ), where y i ' is the mean of the Gaussian model, δ i It is y iand y i is the Euclidean norm, σ is the standard deviation of the Gaussian model, and I is the identity matrix.

[0091] The feature X is calculated by the formula k+1 and the feature Y k is the local conditional entropy at the local position i, where h(y i |x i ) represents the conditional entropy, i.e., the entropy of y i given x i , because y i obeys a multivariate normal distribution with mean y i ' and covariance matrix , the entropy of a normal distribution can be expressed as where log represents the natural logarithm, 2πe is a constant, and δ i is the standard deviation of the Gaussian model. The exponential sum of the local conditional entropy h(y i |x i ) at each local position i is the overall conditional entropy of the feature X k+1 and the feature Y k represents the quality representation capability of the overall feature, where H y , W y is the height and width of the kth scale feature Y k , and h(y i |x i ) is the local conditional entropy at the local position i.

[0092] Step S3 is a process of obtaining a weight matrix that maximizes the overall conditional entropy by a preset modeling optimization formula. In order to obtain a quality perception vector, a weight matrix corresponding to the image needs to be obtained first. Then, how to obtain the weight matrix needs to be determined. First, modeling is performed, and then other formulas are introduced for solving according to the formula generated by modeling. Therefore, the step of obtaining a weight matrix that maximizes the overall conditional entropy by a preset modeling optimization formula includes:

[0093] S31: performing spatial transformation on the front scale feature Y k ={y i , i∈H y ×W y} to obtain the actual transformed feature Z k = ω k × Y k = {ω i y i , i∈H y ×W y}; where ​The weight matrix is ​​unknown and needs to be determined.

[0094] S32: Based on the scale feature X of the (k+1)th scale, the modeling optimization formula is used. k+1 Compared with pre-scale features Y k The actual transformation characteristic Z after spatial transformation k The overall conditional entropy H e (Z k |X k+1 The modeling optimization formula is as follows: (The formula is to be maximized to calculate the unknown weight matrix.)

[0095]

[0096] in, This indicates the search for the parameter ω that maximizes the function within the parentheses. k H e (ω k Y k |X k+1 ) represents the scale feature X at the (k+1)th scale. k+1 Actual transformation feature Z k The overall conditional entropy, H y W y For the actual transformation feature Z k Height and width, The scale feature X at the (k+1)th scale k+1 Actual transformation feature Z k The local conditional entropy at position i, where 2πe is a constant, e is the base of the natural logarithm, and δ i To reconstruct prescale features y i ′ and prescale features y i The Euclidean norm of , the summation after st is the preset constraint function, and c in the formula is the preset constant.

[0097] The process of solving the modeling optimization formula includes:

[0098] The solution function is constructed based on the Lagrange multiplier method. Where λ is a Lagrange multiplier. The objective function of the modeling optimization formula is... The constraint function for the modeling optimization formula;

[0099] when and At that time, the scale feature X at the (k+1)th scale k+1 Actual transformation feature Z k The overall conditional entropy H e (ω k Y k |Xk+1 (To obtain the maximum value)

[0100] Will and When, the optimal weight of the modeling optimization formula As the weight matrix; where δ i To reconstruct prescale features y i ′ and prescale features y i The Euclidean norm, where c is a constant.

[0101] More specific embodiments are as follows:

[0102] First, consider the feature Y at the k-th scale. k ={y i ,i∈H y ×W y Perform a spatial transformation to obtain the feature Z after the k-th scale transformation. k =ω k ×Y k ={ω i y i ,i∈H y ×W y},in Let X be the weight matrix, and let X be the unknown quantity. Maximize the feature X at the (k+1)th scale. k+1 With Y k Features Z after spatial transformation k The overall conditional entropy H e (Z k |X k+1 This enhances the quality representation capability of features, thereby achieving semantic quality decoupling. This task can be modeled as an optimization problem, as shown in Equation 1:

[0103]

[0104] in, This indicates the search for the parameter ω that maximizes the function within the parentheses. k H e (ω k Y k |X k+1 ) is X k+1 With Z k The overall conditional entropy, H y W y For feature Z k Height and width, For X k+1 With Z kThe local conditional entropy at position i, which is the local conditional entropy formula in 2.7, is brought into formula 1 to obtain the formula after the second equality, which is called the objective function, and 2πe is a constant, where e is the base of the natural logarithm, δ i is the Euclidean norm of y i ′ and y i , and s.t. is the constraint function, where c in the formula is a constant.

[0105] According to the Lagrange multiplier method, a loss function L is constructed where λ is the Lagrange multiplier, is the objective function in formula 1, is the constraint function in formula 1. When and , the overall conditional entropy H k+1 (ω k Y k |X k+1 ) of the feature X i and the feature Z i reaches the maximum, and the weight matrix that optimizes formula 1 is where δ i is the Euclidean norm of y k ′ and y k , and c is a constant. In this way, the weight matrix that maximizes the overall conditional entropy can be obtained.

[0106] Step S4 is a process of multiplying the three-dimensional pre-scale features by the respective corresponding weight matrices to obtain first, second and third transformed features, and splicing and pooling the first, second, third transformed features and the scale feature of the fourth scale to obtain a quality perception vector.

[0107] In one specific embodiment, the features Y R (scale features or pre-scale features) of scales k = 1, 2, and 3 are calculated, and the spatial transformed features Z R are obtained. The first, second and third transformed features Z1, Z2 and Z3 obtained by spatially transforming the scale features of scales k = 1, 2 and 3 and the scale feature X4 of scale k = 4 are spliced in the channel dimension to obtain a spliced feature, which is pooled into a one-dimensional vector by global maximum pooling. The vector is the quality perception vector ft R of the image I P to be evaluated.

[0108] Step S5 is a process of mapping and predicting the quality perception vector by a pre-trained quality regression network or quality prediction network to obtain a quality prediction score of the image to be evaluated; wherein,

[0109] The steps of mapping and predicting the quality-perceived vector using a pre-trained quality regression network and a quality prediction network to obtain the quality prediction score of the image to be evaluated include:

[0110] S51: The quality perception vector is mapped using a preset quality regression network to obtain quality mapping features;

[0111] S52: Input the quality mapping features into the quality prediction network so that the quality prediction network activates the quality mapping features to obtain the quality prediction score of the image to be evaluated.

[0112] The steps for training the quality regression network include:

[0113] All images in the reference image set are cropped to specific specifications to form standard images, and the standard images are then subjected to two levels of degradation processing to obtain a medium-quality image set and a poor-quality image set.

[0114] B quality perception vectors ft of images are selected from the quality perception vector sets corresponding to the reference image set, the intermediate image set, and the inferior image set, respectively. R To form a reference feature vector Medium eigenvectors Sum and difference eigenvectors

[0115] Construct a layer with parameters φ consisting of two fully connected layers and a ReLU activation function. P The quality regression network to be trained is repeatedly trained; wherein the repeated training process is as follows:

[0116] The reference feature vector, the intermediate feature vector, and the differential feature vector are input into the regression network to be trained, so that the first fully connected layer maps the reference feature vector, the intermediate feature vector, and the differential feature vector to a low-dimensional hidden layer to obtain the reference hidden layer output vector F. R The output vector F with medium hiding time is ''. M The output vector F of the hidden layer is equal to the sum and difference. P The second fully connected layer maps the output vector of the reference hidden layer, the output vector of the intermediate hidden layer, and the output vector of the inferior hidden layer to obtain the reference quality mapping feature. Medium quality mapping features Sum and difference quality mapping features

[0117] Update the untrained quality regression network using an optimizer. weight φP , calculate a preset quality regression loss function L proj about the weight φ P Gradient to update the first and second moments, and correct the bias, and update the weight φ P of the first and second moments after correction, until the regression loss function L proj reaches a preset regression loss function threshold, then stop updating, and take the last updated quality regression network as the quality regression network; wherein the quality regression loss function L proj is:

[0118]

[0119] wherein, denotes the cosine similarity between two vectors, and α, β are hyperparameters controlling the aggregation degree of features, i, j ∈ [1, B].

[0120] In one specific embodiment, first use the generated reference image set R, medium image set M and poor image set P, randomly select B feature vectors from the quality feature set R ft of the reference image set R, denoted as Select the corresponding features of the distorted images from the quality feature set M ft of the “medium” image set M and the quality feature set P ft of the “poor” image set P, respectively denoted as Construct a quality regression network with two fully connected layers and a Relu activation function, with parameters φ P It is also a lightweight network compared to the backbone network. F R , F M and F P are simultaneously sent to the quality regression network Through the first fully connected layer of the regression network, the input three features are mapped to a lower-dimensional hidden layer respectively, and the hidden layer output F R ', F M ' and F P ' are obtained. Then apply the Relu activation function to the output of the hidden layer to obtain the activated hidden layer output. Finally, the hidden layer output is passed through the second fully connected layer to obtain the quality mapping feature In subjective quality assessment, the scores of extremely poor and extremely good images have high confidence. According to this feature, the loss function L proj shown in formula 2 is used to update the weight φ P of the quality regression network using the Adam optimizer, first calculate the loss function L proj ​Gradient of the weight φ P Update the first and second moments and correct the bias, and finally use the corrected first and second moments to update the weight φ P , the learning rate is set to 1e-3, and the weight decay is set to 5e-4; formula 2 is as follows

[0121]

[0122] In formula 2 is the cosine similarity between two vectors, and α and β are hyperparameters that control the degree of feature aggregation, i, j ∈ [1, B].

[0123] The step of training the quality prediction network includes:

[0124] A parameter φ a is constructed by two fully connected layers and a Relu activation function

[0125] The reference quality mapping feature The medium quality mapping feature And the poor quality mapping feature are input into the to-be-trained quality prediction network to generate reference image quality scores, medium image quality scores, and poor image quality scores, respectively.

[0126] Based on the reference image quality scores, medium image quality scores, and poor image quality scores, a quality prediction loss function is created, and the to-be-trained quality prediction network is repeatedly trained until the quality prediction loss function reaches a preset threshold, then the training is stopped, and the last updated to-be-trained quality prediction network is taken as the quality prediction network.

[0127] In one specific embodiment, a quality prediction network with a parameter φ a is constructed by two fully connected layers and a Relu activation function The quality mapping features F R_proj , F M_proj , and F P_proj are sent to Through the first fully connected layer of the quality prediction network , the input features are mapped to a lower-dimensional hidden layer, and then the Relu activation function is applied to the output of the hidden layer to obtain the activated hidden layer output. Finally, the activated hidden layer output is mapped to the image quality score and

[0128] ​For images with the same content, quality decreases as the distortion level increases. Therefore, the loss function L shown in Equation 3 is used. agr Update the quality prediction network using the Adam optimizer. weight φ a First, calculate the loss function L. agr Regarding the weight φ a gradient Update the first and second moments and correct for biases. Finally, use the corrected first and second moments to update the weights φ. a The learning rate is set to 1e-3, and the weight decay is set to 5e-4. In Formula 3, i∈[1,B].

[0129]

[0130] For a reference image set R, a "medium" image set M, and a "poor" image set P, the feature set R is... ft M ft and P ft Repeat steps 3.2 to 4.2 until all features have been explored, discarding any features with fewer than B features. The quality regression network is stabilized by setting the gradient of its parameters to zero. and quality prediction network The parameters, and the parameters after training are respectively and Therefore, the trained regression network can be used directly in subsequent training.

[0131] The specific training and usage process is as described above. To further highlight the usage process, another specific embodiment is provided, again using characters different from those in the above embodiment to represent each feature, so as to clearly reflect the specific usage logic; First, for an input image I test First, the input image is scaled to 384×512 and then input into the backbone network f. θ (·) Extract features from the first to the fourth scale. Among them, H 1 W 1 The height and width of the features at the first scale are defined, and so on. Features f2, f3, and f4 are then input into the reconstruction network in step 2.3. Obtain the reconstructed features and Then, the reconstructed features Multiply each feature by its pre-acquired weight matrix and perform a spatial transformation w to obtain the transformed features. and Features after spatial transformation and the feature of scale 4 f4 is spliced in the channel dimension to obtain a spliced feature, and the spliced feature is pooled into a one-dimensional vector ft by global maximum pooling, that is, the quality perception vector of the image I test . ft is sent to the quality regression network to obtain a quality mapping feature ft proj . ft proj is sent to the quality prediction network of step 4 to obtain the quality prediction score q of the image I test .

[0132] As described above, the image quality evaluation method based on quality decoupling and lightweight contrast learning provided by the present application reconstructs the pre-scale features of the scale features of each scale based on the reconstruction network in the semantic quality decoupling module to obtain three-dimensional reconstructed pre-scale features, models the local conditional probability of each scale feature and the pre-scale feature at the local position based on the reconstructed pre-scale features, and obtains the overall conditional entropy of the scale feature and the pre-scale feature according to the conditional probability. Then, the weight matrix that maximizes the overall conditional entropy is obtained through a preset modeling optimization formula, and the three-dimensional reconstructed pre-scale features are multiplied by the respective corresponding weight matrix to perform spatial transformation to obtain first transformed features, second transformed features and third transformed features. The first transformed features, the second transformed features, the third transformed features and the scale features of the fourth scale are spliced and pooled to obtain a quality perception vector. The quality perception vector is mapped and predicted by the pre-trained quality regression network and quality prediction network to obtain the quality prediction score of the image to be evaluated. In this way, the lightweight quality contrast learning strategy can be efficiently trained on a single GPU, the entire training process does not require subjective opinion scores, the semantic quality decoupling is modeled as an optimization problem, and only a lightweight reconstruction network is needed to calculate the feature weight map with the maximum quality representation capability. The objectivity of image quality evaluation is improved, resource waste is reduced, and quality evaluation accuracy is improved.

[0133] As shown in Fig. 3 , the present application also provides a lightweight contrast learning image quality evaluation system 100 based on quality decoupling, which implements the lightweight contrast learning image quality evaluation method based on quality decoupling as described above, and includes a semantic quality decoupling module 101, a quality regression network 102 and a quality prediction network 103.

[0134] The semantic quality decoupling module 101 includes:

[0135] a backbone network 1011 for obtaining scale features of four scales of an image to be evaluated;

[0136] a reconstruction network 1012 for reconstructing pre-scale features of the scale features of each scale to obtain three-dimensional reconstructed pre-scale features;

[0137] a quality perception module 1013, configured to model a local conditional probability of each scale feature and a pre-scale feature at a local position based on the reconstructed pre-scale feature, and acquire an overall conditional entropy of the scale feature and the pre-scale feature according to the conditional probability; acquire a weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula; multiply the reconstructed pre-scale features in three dimensions with the respective corresponding weight matrices respectively to perform spatial transformation to acquire a first transformed feature, a second transformed feature, and a third transformed feature, and perform splicing and pooling processing on the first transformed feature, the second transformed feature, the third transformed feature, and a scale feature of a fourth scale to acquire a quality perception vector;

[0138] the quality regression network 102 and the quality prediction network 103 are configured to perform mapping and prediction processing on the quality perception vector to acquire a quality prediction score of the image to be evaluated.

[0139] The specific implementation refers to the above embodiment of the image quality evaluation method based on quality decoupling and lightweight contrast learning, and will not be repeated here.

[0140] As described above, the image quality evaluation system based on quality decoupling and lightweight contrast learning provided by the application reconstructs the pre-scale features of each scale feature through the reconstruction network 1012 in the semantic quality decoupling module 101 to acquire reconstructed pre-scale features in three dimensions, then models a local conditional probability of each scale feature and a pre-scale feature at a local position based on the reconstructed pre-scale feature, and acquires an overall conditional entropy of the scale feature and the pre-scale feature according to the conditional probability, and then acquires a weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula, and multiplies the reconstructed pre-scale features in three dimensions with the respective corresponding weight matrices respectively to perform spatial transformation to acquire a first transformed feature, a second transformed feature, and a third transformed feature, and perform splicing and pooling processing on the first transformed feature, the second transformed feature, the third transformed feature, and a scale feature of a fourth scale to acquire a quality perception vector, and perform mapping and prediction processing on the quality perception vector through the pre-trained quality regression network 102 and the quality prediction network 103 to acquire a quality prediction score of the image to be evaluated. In this way, the lightweight quality contrast learning strategy is adopted, which can be efficiently trained on a single GPU, the entire training process does not require subjective opinion scores, and the semantic quality decoupling is modeled as an optimization problem, and only a lightweight reconstruction network is required to calculate the feature weight map with the maximum quality representation capability, thereby improving the objectivity of image quality evaluation, reducing resource waste, and improving the quality evaluation precision.

[0141] The quality-decoupled lightweight contrastive learning image quality evaluation method and system according to the present application are described above with reference to the accompanying drawings by way of example. However, those skilled in the art should understand that various improvements can be made to the quality-decoupled lightweight contrastive learning image quality evaluation method and system according to the present application described above without departing from the content of the present application. Therefore, the protection scope of the present application should be determined by the content of the appended claims.

Claims

1. A lightweight contrastive learning-based image quality assessment method based on quality decoupling, characterized in that, include: The image to be evaluated is obtained by using a preset semantic quality decoupling module to acquire scale features of four scales, and the pre-scale features of each scale feature are reconstructed based on the reconstruction network in the semantic quality decoupling module to obtain three-dimensional reconstructed pre-scale features. Based on the reconstructed pre-scale features, model the local conditional probabilities of each scale feature and the pre-scale feature at local locations, and obtain the overall conditional entropy of the scale feature and the pre-scale feature according to the conditional probabilities. The weight matrix that maximizes the overall conditional entropy is obtained through a preset modeling optimization formula; The pre-reconstruction scale features of the three dimensions are multiplied by their respective weight matrices to perform spatial transformation to obtain the first transformation feature, the second transformation feature, and the third transformation feature. The first transformation feature, the second transformation feature, the third transformation feature, and the scale feature of the fourth scale are then concatenated and pooled to obtain the quality-perceived vector. The quality perception vector is mapped and predicted using a pre-trained quality regression network and a quality prediction network to obtain the quality prediction score of the image to be evaluated.

2. The lightweight contrastive learning image quality assessment method based on quality decoupling as described in claim 1, characterized in that, The steps include obtaining scale features of the image to be evaluated at four scales through a preset semantic quality decoupling module, and reconstructing pre-scale features of each scale feature based on the reconstruction network in the semantic quality decoupling module to obtain three-dimensional reconstructed pre-scale features, including: The image to be evaluated is input into the backbone network of the semantic quality decoupling module, so that the pooling layer in the backbone network extracts features from the image to be evaluated to obtain scale features at four scales, namely the first scale feature, the second scale feature, the third scale feature and the fourth scale feature. The second scale feature, the third scale feature, and the fourth scale feature are input into the trained reconstruction network. In order to obtain the pre-reconstruction scale features for predictive reconstruction: reconstruction scale one features, reconstruction scale two features, and reconstruction scale three features.

3. The lightweight contrastive learning image quality assessment method based on quality decoupling as described in claim 2, characterized in that, The steps of pre-training the reconstruction network include: A predetermined number of images are selected to form a reference image set, and feature extraction is performed on the images in the reference image set to obtain scale features at four scales, which are called training scale features. Construct a fully connected layer with the following parameters: Network to be trained and rebuilt The training scale feature of the first Training scale features at each scale Input to the network to be trained and reconstructed In the middle, to predict the first Pre-trained scalar features at each scale The predicted pre-training scale features are denoted as the pre-training scale features after reconstruction. ,in, ; and The first Training scale features With the Training scale features or Height and width; The pre-reconstruction training scale features Compared with real pre-trained scale features Euclidean norm as loss function ;in ; Update the network to be rebuilt using the Adam optimizer. parameters Calculate the loss function Regarding parameters gradient To update and correct the network to be rebuilt during training The first and second moments; The network to be trained and reconstructed is updated using the corrected first and second moments. parameters Training continues until the loss function reaches a preset threshold, at which point training stops, and the network is reconstructed using the last updated data. As a network reconstruction .

4. The lightweight contrastive learning image quality assessment method based on quality decoupling as described in claim 3, characterized in that, The process of modeling the local conditional probabilities of each scale feature and the pre-scale feature at local locations based on the reconstructed pre-scale features, and obtaining the overall conditional entropy of the scale features and the pre-scale features based on the conditional probabilities, includes: Based on scale features Compared with prescale features Perform Gaussian modeling to form a Gaussian model ; wherein, the pre-reconstruction scale features predicted by the reconstruction network are... As the mean of the Gaussian model, it will reconstruct the pre-scale features. Compared with prescale features Euclidean norm As the standard deviation of the Gaussian model; Let be the identity matrix of the Gaussian model; Based on the Gaussian model Through formula Computational scale features Compared with prescale features In local locations The local conditional entropy at a given location; where, Represents rebuilding the network; For each local location Local conditional entropy at a point Summing the exponents to obtain scale features Compared with prescale features Overall conditional entropy ;in For the first Individual scale features The height and width.

5. The lightweight contrastive learning image quality assessment method based on quality decoupling as described in claim 4, characterized in that, The steps of obtaining the weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula include: For prescale features Perform spatial transformation to obtain the actual transformation features after the k-th scale transformation. ;in, For actual transformation characteristics Height and width, The weight matrix is ​​unknown and needs to be determined. The modeling optimization formula is based on the scale features of the (k+1)th scale. Compared with prescale features Actual transformation characteristics after spatial transformation global conditional entropy Maximization is performed to calculate the unknown weight matrix; wherein the modeling optimization formula is: ； in, This indicates finding the parameter that makes the function within the parentheses reach its maximum value. , Scale features at the (k+1)th scale Actual transformation characteristics The overall conditional entropy, Scale features at the (k+1)th scale Actual transformation characteristics In position Local conditional entropy at a given location It is a constant. is the base of the natural logarithm. To reconstruct prescale features Compared with prescale features The Euclidean norm, st The subsequent summation is a pre-defined constraint function, in which... c This is a preset constant.

6. The lightweight contrastive learning image quality assessment method based on quality decoupling as described in claim 5, characterized in that, The process of solving the modeling optimization formula includes: The solution function is constructed based on the Lagrange multiplier method. ;in For Lagrange multipliers, The objective function of the modeling optimization formula is... The constraint function for the modeling optimization formula; when and At that time, the scale feature of the (k+1)th scale Actual transformation characteristics Overall conditional entropy To obtain the maximum value; Will and When, the optimal weight of the modeling optimization formula As the weight matrix; where To reconstruct prescale features Compared with prescale features The Euclidean norm, c It is a constant.

7. The lightweight contrastive learning image quality assessment method based on quality decoupling as described in claim 6, characterized in that, The steps of mapping and predicting the quality-perceived vector using a pre-trained quality regression network and a quality prediction network to obtain the quality prediction score of the image to be evaluated include: The quality perception vector is mapped using a preset quality regression network to obtain quality mapping features. The quality mapping features are input into the quality prediction network so that the quality prediction network activates and maps the quality mapping features to obtain the quality prediction score of the image to be evaluated.

8. The lightweight contrastive learning image quality assessment method based on quality decoupling as described in claim 7, characterized in that, The steps for training the quality regression network include: All images in the reference image set are cropped to specific specifications to form standard images, and the standard images are then subjected to two levels of degradation processing to obtain a medium-quality image set and a poor-quality image set. B quality-perceived vectors of images are selected from the quality-perceived vector sets corresponding to the reference image set, the intermediate image set, and the inferior image set, respectively. To form a reference feature vector Medium eigenvectors Sum and difference eigenvectors ; Construct a layer consisting of two fully connected layers and a ReLU activation function with the following parameters: The quality regression network to be trained is repeatedly trained; wherein the repeated training process is as follows: The reference feature vector, the intermediate feature vector, and the inferior feature vector are input into the quality regression network to be trained, so that the first fully connected layer maps the reference feature vector, the intermediate feature vector, and the inferior feature vector to the low-dimensional hidden layer to obtain the output vector of the reference hidden layer. Output vectors with medium hiding, etc. The output vectors of the hidden layers, such as sum and difference, are... The second fully connected layer then maps the output vectors of the reference hidden layer, the intermediate hidden layer, and the inferior hidden layer to obtain reference quality mapping features. Medium quality mapping characteristics Sum and difference quality mapping features ; Update the untrained quality regression network using an optimizer. weight Calculate the preset quality regression loss function Regarding weight gradient This updates the first and second moments, corrects biases, and updates the weights of the corrected first and second moments. until the regression loss function If the preset regression loss function threshold is reached, updates stop, and the last updated training quality regression network is used as the final quality regression network; where the quality regression loss function... for: ； in, The cosine similarity between two vectors is represented by... To control the degree of feature aggregation, .

9. The lightweight contrastive learning image quality assessment method based on quality decoupling as described in claim 8, characterized in that, The steps for training the quality prediction network include: Construct a layer consisting of two fully connected layers and a ReLU activation function with the following parameters: The quality prediction network to be trained ; Map reference quality features Medium quality mapping characteristics Sum and difference quality mapping features The input is fed into the quality prediction network to be trained to generate reference image quality scores, medium image quality scores, and poor image quality scores, respectively. A quality prediction loss function is created based on the reference image quality score, the medium image quality score, and the poor image quality score. The quality prediction network to be trained is repeatedly trained until the quality prediction loss function reaches a preset threshold. Then, training stops, and the last updated quality prediction network to be trained is used as the quality prediction network.

10. A lightweight contrastive learning image quality assessment system based on quality decoupling, implementing the lightweight contrastive learning image quality assessment method based on quality decoupling as described in any one of claims 1-9, the system comprising a semantic quality decoupling module, a quality regression network, and a quality prediction network; wherein, The semantic quality decoupling module includes: The backbone network is used to obtain scale features of the image to be evaluated at four scales. A reconstruction network is used to reconstruct the prescale features of each scale to obtain three-dimensional reconstructed prescale features. The quality-aware module is used to model the local conditional probabilities of each scale feature and the pre-scale feature at local locations based on the pre-reconstruction scale features, and to obtain the overall conditional entropy of the scale features and the pre-scale features according to the conditional probabilities; to obtain a weight matrix that maximizes the overall conditional entropy through a preset modeling optimization formula; to perform spatial transformation by multiplying the pre-reconstruction scale features of the three dimensions with their respective weight matrices to obtain a first transformed feature, a second transformed feature, and a third transformed feature; and to concatenate and pool the first transformed feature, the second transformed feature, the third transformed feature, and the scale feature of the fourth scale to obtain a quality-aware vector. The quality regression network and the quality prediction network are used to map and predict the quality perception vector to obtain the quality prediction score of the image to be evaluated.

Images