A full-reference image quality assessment method based on Conformer and meta-learning

By introducing Conformer and meta-learning technology into the image quality evaluation method, the problem of low generalization ability in the existing technology is solved, and accurate prediction of image quality of small sample data sets of various distortion types is achieved.

CN115661114BActive Publication Date: 2025-03-14CHONGQING UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211400068.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-11-09
Publication Date
2025-03-14
Estimated Expiration
2042-11-09

AI Technical Summary

Technical Problem

The existing full reference image quality evaluation method faces small sample data sets of multiple distortion types, and it is difficult to accurately perceive image quality.

Method used

Using a full reference image quality evaluation method based on Conformer and meta-learning, image quality evaluation is realized through feature extraction module, similarity calculation module, weight calculation module, evaluation score module and meta-learning-based training and testing module.

Benefits of technology

On the basis of ensuring the generalization ability of the model, the quality score of the image can be accurately predicted, which is suitable for small sample data sets facing multiple types of distortion.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115661114B_ABST
    Figure CN115661114B_ABST
Patent Text Reader

Abstract

The present invention discloses a full-reference image quality assessment method based on Conformer and meta-learning, comprising the following steps: obtaining a distorted image to be predicted and a corresponding reference image; inputting the distorted image and the corresponding reference image into a pre-trained image quality assessment model; the image quality assessment model comprises: a feature extraction module, a similarity calculation module, a weight calculation module, an evaluation score module, and a training and testing module based on meta-learning; and outputting a prediction score corresponding to the distorted image. The present invention is based on an image quality assessment model, and through the feature extraction module, similarity calculation module, weight calculation module, evaluation score module, and training and testing module based on meta-learning constructed by the present invention, when faced with a small sample data set of multiple distortion types, the quality score of the distorted image to be predicted can be accurately predicted while ensuring the generalization ability of the model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image and video processing technology, and in particular to a full-reference image quality evaluation method based on Conformer and meta-learning. Background Art

[0002] Image distortion has become a key evaluation indicator in the field of image processing in areas such as image denoising, image super-resolution, and image compression. The image perception quality score obtained by human observers is accurate, but it requires a lot of manpower and material resources.

[0003] Traditional full-reference image quality assessment methods generally use manually obtained feature descriptors to extract distortion-related features from the distorted image that are different from the reference image. However, manual feature descriptors are difficult to cover all distortion types.

[0004] In the full-reference image quality assessment method based on deep learning, if the conventional single gradient descent method is used in the training process, when the training dataset and the test dataset belong to two different types of small sample datasets, the distortion type judgment ability learned by the model from the current dataset is actually difficult to apply to another dataset, which will lead to a decrease in the generalization ability of the model.

[0005] Therefore, based on the existing full-reference image quality evaluation method, when faced with a small sample data set with multiple distortion types, how to accurately perceive the quality of the image while ensuring the generalization ability of the model has become an urgent problem to be solved in this field. Summary of the invention

[0006] The purpose of the present invention is to provide a full-reference image quality evaluation method based on Conformer and meta-learning, which can accurately predict the quality score of the image while ensuring the generalization ability of the model when facing a small sample data set with different distortion types.

[0007] To achieve the above object, the technical solution adopted by the present invention is:

[0008] The present invention provides a full-reference image quality assessment method based on Conformer and meta-learning, comprising the following steps:

[0009] S1. Obtain a distorted image to be predicted and a corresponding reference image;

[0010] S2, inputting the distorted image and the corresponding reference image into a pre-trained image quality assessment model; the image quality assessment model comprises: a feature extraction module, a similarity calculation module, a weight calculation module, an evaluation score module and a training and testing module based on meta-learning;

[0011] S3. Output the prediction score corresponding to the distorted image.

[0012] Furthermore, the image quality assessment model training process in step S2 includes:

[0013] S21, multiple types of distorted images and corresponding reference images are input as training data sets into a twin network with Conformer as a feature extraction module to extract feature vectors corresponding to the distorted images and the reference images respectively;

[0014] S22, in a similarity calculation module, the feature vectors corresponding to the distorted image and the reference image are respectively calculated using two preset methods to obtain similarities between the feature vectors of the distorted image and the reference image;

[0015] S23, in the weight calculation module, after fusing the feature vector of the distorted image and the feature vector of the reference image, the fused feature is mapped into the weight of the feature vector similarity through multiple fully connected layers and activation functions;

[0016] S24, combining the similarity of the feature vectors of the distorted image and the reference image in step S22 and the weight of the feature vector similarity in step S23, calculating the final feature vector similarity of the distorted image and the reference image, and obtaining the final quality evaluation score of the distorted image;

[0017] S25. Add meta-learning to the training and testing process of the model: In the training phase, by utilizing the double gradient descent method from the support set to the query set, the model is allowed to learn to adjust the current model initialization parameters according to a part of the distortion types; in the testing phase, a small number of images of another part of other distortion types are used to fine-tune the parameters of the model that has the ability to identify distortion types after the training phase, so as to obtain the final image quality evaluation model.

[0018] Furthermore, in the step S21, the feature vectors corresponding to the distorted image and the reference image are extracted by inputting into the twin network with Conformer as the feature extraction module; including:

[0019] The distorted image and the corresponding reference image are respectively input into the feature extraction network with Conformer as the backbone, and the corresponding global features are extracted using the self-attention mechanism; and the corresponding local features are extracted using the convolution operation;

[0020] After the global features and the local features are embedded into each other, their dimensions are changed, and the feature vectors corresponding to the distorted image and the reference image are extracted.

[0021] Furthermore, in step S22, two preset methods are used to calculate the similarity between the feature vectors of the distorted image and the reference image, including:

[0022] The ProjectedDotProductSimilarity method is used to calculate the similarity1 between the distorted image feature vector and the reference image feature vector:

[0023]

[0024] (1)Where V1 represents the feature vector obtained after the reference image passes through the twin network; V2 represents the feature vector obtained after the distorted image passes through the twin network, and w1 and w2 represent random initialization vectors for adjusting the dimension of the feature vector;

[0025] The TriLinearSimilarity method is used to calculate the similarity2 between the distorted image feature vector and the reference image feature vector:

[0026] similarity2=w T [V1,V2,V1*V2]+b (2)

[0027] (2)In the formula, V1*V2 represents the vector obtained by multiplying the corresponding elements of two feature vectors; [V1, V2, V1*V2] represents the result of concatenating the three vectors according to the last dimension; w represents the random initialization vector for adjusting the dimension of the concatenated vector, and b represents the bias term for fine-tuning the similarity result.

[0028] Further, the step S23 includes:

[0029] S231, the feature vector V1 of the reference image and the feature vector V2 of the distorted image are fused using the concat operation:

[0030] V fuse =cat(V1,V2,1) (3)

[0031] S232, obtain fusion feature V fuse After that, V is transformed through multiple fully connected layers and activation functions. fuse The result of the mapping is used as the weight parameter of the similarity

[0032]

[0033] Among them, FC1 and FC2 are fully connected layers, and f represents the sigmoid activation function.

[0034] Furthermore, the step S24 includes:

[0035] Combined weight parameters Calculate the similarity between the distorted image feature vector and the reference image feature vector:

[0036]

[0037] Similarity represents the similarity between the distorted image and the reference image, which serves as the final distorted image quality prediction score.

[0038] Furthermore, the step S25 adds meta-learning to the training and testing process of the model, specifically including:

[0039] S251. During the training process, a part of the distortion types is selected, and the small sample data set is divided into subtasks, and each subtask is divided into a support set and a query set;

[0040] S252, setting the loss function of the support set and the query set in the subtask;

[0041] S253, during the training process, the first-order gradient of the loss function corresponding to the current subtask support set is calculated, and the current model parameters are updated through the Adam optimizer to complete the gradient optimization corresponding to the support set;

[0042] S254: During the training process, based on the model parameters after step S253, the query set of the current subtask is input, and the current model parameters are updated again through the Adam optimizer, thereby completing the gradient optimization corresponding to the query set; by integrating the gradient update results of all subtasks, the prior model parameters that have a guiding role in the test phase are obtained;

[0043] S255. During the test, a small number of images of other distortion types are input, and the model with prior parameters is updated using the Adam optimizer to obtain the final image quality evaluation model.

[0044] Furthermore, the step S252 includes:

[0045] Set the loss function for the support set and query set in the subtask:

[0046]

[0047] Among them, e represents the quality label score corresponding to the distorted image, and s represents the quality prediction score corresponding to the distorted image.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] The embodiment of the present invention provides a full-reference image quality assessment method based on Conformer and meta-learning, comprising the following steps: obtaining a distorted image to be predicted and a corresponding reference image; inputting the distorted image and the corresponding reference image into a pre-trained image quality assessment model; the image quality assessment model comprises: a feature extraction module, a similarity calculation module, a weight calculation module, an evaluation score module and a training and testing module based on meta-learning; and outputting a prediction score corresponding to the distorted image. The present invention is based on an image quality assessment model, and through the feature extraction module, similarity calculation module, weight calculation module, evaluation score module and training and testing module based on meta-learning constructed by the present invention, when faced with a small sample data set of multiple distortion types, it can accurately predict the quality score of the distorted image to be predicted while ensuring the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A flowchart of a full-reference image quality assessment method based on Conformer and meta-learning provided in an embodiment of the present invention;

[0051] Figure 2 A flow chart of the image quality assessment model training process provided by an embodiment of the present invention;

[0052] Figure 3 Schematic diagram of the full-reference image quality assessment method based on Conformer and meta-learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0054] In the description of the present invention, it should be noted that the terms "upper", "lower", "inner", "outer", "front end", "rear end", "two ends", "one end", "the other end" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0055] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "provided with", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0056] Reference Figure 1 As shown, the present invention provides a full-reference image quality evaluation method based on Conformer and meta-learning, comprising the following steps:

[0057] S1. Obtain a distorted image to be predicted and a corresponding reference image;

[0058] S2, inputting the distorted image and the corresponding reference image into a pre-trained image quality assessment model; the image quality assessment model comprises: a feature extraction module, a similarity calculation module, a weight calculation module, an evaluation score module and a training and testing module based on meta-learning;

[0059] S3. Output the prediction score corresponding to the distorted image.

[0060] Among them, in step S1, for example, in the field of image compression, the distorted image refers to the compressed image, and the reference image refers to the image before compression; the image content of the two is consistent, and the distorted image has noise and / or lower resolution than the reference image, resulting in a certain blur in human visual perception. The image quality evaluation model in step S2 includes: a feature extraction module, a similarity calculation module, a weight calculation module, an evaluation score module, and a training and testing module based on meta-learning;

[0061] The feature extraction module is used to extract the feature vector of the image, which is a fusion of the global features and local features of the image. When extracting features from the distorted image and the reference image, the two share the weight parameters of the network. The similarity calculation module is used to calculate the similarity of the image feature vectors; the weight calculation module is used to calculate the similarity weight after the fusion of the feature vector of the distorted image and the feature vector of the reference image. The evaluation score module calculates the final quality evaluation score of the distorted image based on the outputs of the similarity calculation module and the weight calculation module. The training and testing module based on meta-learning adds meta-learning to the training and testing process of the model, allowing the model to learn to adjust its current initialization parameters according to different distortion types to improve the generalization ability of the model.

[0062] The above step S3 outputs the prediction score corresponding to the distorted image based on the pre-trained image quality assessment model. This method can accurately predict the quality score of the distorted image to be predicted while ensuring the generalization ability of the model when facing a small sample data set with multiple distortion types.

[0063] In one embodiment, referring to Figure 2 As shown, the image annotation model training process in step S2 above includes the following steps:

[0064] S21, multiple types of distorted images and corresponding reference images are input as training data sets into a twin network with Conformer as a feature extraction module to extract feature vectors corresponding to the distorted images and the reference images respectively;

[0065] S22, in a similarity calculation module, the feature vectors corresponding to the distorted image and the reference image are respectively calculated using two preset methods to obtain similarities between the feature vectors of the distorted image and the reference image;

[0066] S23, in the weight calculation module, after fusing the feature vector of the distorted image and the feature vector of the reference image, the fused feature is mapped into the weight of the feature vector similarity through multiple fully connected layers and activation functions;

[0067] S24, combining the similarity of the feature vectors of the distorted image and the reference image in step S22 and the weight of the feature vector similarity in step S23, calculating the final feature vector similarity of the distorted image and the reference image, and obtaining the final quality evaluation score of the distorted image;

[0068] S25. Add meta-learning to the training and testing process of the model: In the training phase, by utilizing the double gradient descent method from the support set to the query set, the model is allowed to learn to adjust the current model initialization parameters according to a part of the distortion types; in the testing phase, a small number of images of another part of other distortion types are used to fine-tune the parameters of the model that has the ability to identify distortion types after the training phase, so as to obtain the final image quality evaluation model.

[0069] In step S21, various types of distorted images and corresponding reference images are used as training data sets. For example, the distortion types include: white noise, Gaussian blur, image compression, etc.

[0070] The distorted image and the corresponding reference image are input into a twin network with Conformer as the feature extraction module. The feature vectors corresponding to the reference image and the distorted image are V1 and V2, respectively, which are composed of the global features and local features of the image. The feature extraction modules of the distorted image and the reference image share the weight parameters of the network. Specifically, the reference image is input into a feature extraction network with Conformer as the backbone, the global features of the reference image are extracted using the self-attention mechanism, the local features of the reference image are extracted using the convolution operation, the global features and local features of the reference image are embedded in each other, and their dimensions are changed to obtain the feature vector V1 of the reference image. A twin network that shares the internal structural parameters of Conformer is constructed, and the distorted image is also sent into the same feature extraction network. The feature vector V2 of the distorted image is obtained in the same way as the reference image.

[0071] In step S22, after passing through the feature extraction network, the ProjectedDotProductSimilarity method is used to calculate the similarity1 between the distorted image feature vector and the reference image feature vector.

[0072]

[0073] In the above formula, V1 represents the feature vector obtained after the reference image passes through the twin network, V2 represents the feature vector obtained after the distorted image passes through the twin network, w1 and w2 represent random initialization vectors for adjusting the dimension of the feature vector. The TriLinearSimilarity method is used to calculate the similarity2 between the feature vector of the distorted image and the feature vector of the reference image.

[0074] similarity2=w T [V1,V2,V1*V2]+b (2)

[0075] In the above formula, V1*V2 represents the vector obtained by multiplying the corresponding elements of two feature vectors, [V1,V2,V1*V2] represents the result of concatenating the three vectors according to the last dimension, w represents the random initialization vector for adjusting the dimension of the concatenated vector, and b represents the bias term for fine-tuning the similarity result.

[0076] In step S23, in order to obtain the weight parameter of similarity The feature vector V1 of the reference image and the feature vector V2 of the distorted image are concatenated using the concat operation on the first dimension of the vector.

[0077] V fuse =cat(V1,V2,1) (3)

[0078] Get the fusion feature Vfuse After that, V is transformed through multiple fully connected layers and activation functions. fuse The result of the mapping is used as the weight parameter of the similarity

[0079]

[0080] Among them, FC1 and FC2 are fully connected layers, and their input and output dimensions correspond to (4000, 512) and (512, 1) respectively, and f represents the sigmoid activation function.

[0081] In step S24, after obtaining the weight parameter After that, the final similarity between the distorted image feature vector and the reference image feature vector is calculated.

[0082]

[0083] Similarity indicates the similarity between the distorted image and the reference image. The closer the distorted image is to the reference image, the greater its similarity value is. Therefore, the present invention uses similarity as the final picture quality prediction score.

[0084] In step S25, meta-learning is added to the training and testing process of the model, and the double gradient descent method from the support set to the query set in meta-learning is used to allow the model to learn to optimize and adjust the model parameters.

[0085] During the training process, the training data set consists of N subtask sets, each of which consists of a support set and a query set. The training data set can be expressed as:

[0086]

[0087] in, Represents different sets of subtasks, represents the support set and query set corresponding to different subtask sets, and N is the total number of subtasks. The loss function corresponding to the support set and query set is set as:

[0088]

[0089] Among them, e represents the quality label score corresponding to the distorted image, and s represents the quality prediction score corresponding to the distorted image. The support set in the i-th subtask can be expressed as The corresponding loss function can be expressed as

[0090] During the training process, the dual gradient descent method from the support set to the query set is used to allow the model to learn and adjust the current model initialization parameters according to a part of the distortion types, such as white noise, Gaussian blur, image compression and other distortion types.

[0091] First, gradient optimization is performed on the support set, using θ to represent all parameters of the model. The first-order gradient with respect to θ is:

[0092]

[0093] Among them, f θ represents the model with parameter θ. The support set of the ith subtask is obtained by the Adam optimizer The update process is divided into T steps. After the update, the parameters of the model are:

[0094]

[0095] Among them, a is the internal learning rate. To prevent the denominator from becoming 0, a correction term is added set up m θ(t) and v θ(t) They represent the first-order moment and the second-order moment of the gradient respectively, which can be expressed as:

[0096] m θ(t) =y1*m θ(t-1) +(1-y1)*g θ(t)

[0097]

[0098] Among them, y1 and y2 are m θ(t) and v θ(t) The exponential decay rate, g θ(t) is the gradient updated in step t (t∈{1, 2, ..., T}). Next, the corresponding gradient optimization is performed on the query set. Based on the optimization results of the support set, the Adam optimizer is used to optimize the query set of the i-th subtask. After updating, the parameters of the model are:

[0099]

[0100] Among them, θ i ′ is the model parameter optimized on the i-th subtask support set, θ i ″ is the model parameter optimized on the i-th subtask query set. For the N tasks in the training process, the gradient update results of all tasks are integrated to obtain the prior model parameters that guide the testing phase.

[0101]

[0102] Where β is the external learning rate, θ f Indicates the prior model parameters obtained after the gradient optimization is completed. During the test, a small number of other distorted images, such as ringing effect images, are input, and the Adam optimizer is used to update the model with prior parameters. The update is completed in P steps. After the update, the parameters of the model are:

[0103]

[0104] a f is the external learning rate during the fine-tuning of the above model parameters, θ te is the model parameter after complete meta-learning. Finally, the distorted image to be predicted is input, and the optimal score prediction of the distorted image can be expressed as,

[0105]

[0106] Here, x represents a single distorted image input to the model.

[0107] Combine the following Figure 3 The full-reference image quality evaluation method based on Conformer and meta-learning of the present invention is specifically described in detail: the public data set is divided according to the distortion type, and the white noise image, Gaussian blurred image, compressed image, etc. and their reference images are sent to the feature extraction module in batches to obtain the feature vector V1 and the feature vector V2, and the prediction score of the current distorted image is obtained after passing through the similarity calculation module and the weight calculation module. When the distorted image and its reference image are used for model training, the parameters of the model are gradient updated twice; a small number of other distortion types are selected, such as the ringing effect image and its reference image, and the trained model parameters are gradient updated once to complete the fine-tuning of the model, and the best model can be obtained. The remaining ringing effect images and their reference images are input to obtain the best prediction score.

[0108] The full-reference image quality evaluation method based on Conformer and meta-learning provided in this embodiment can accurately predict the quality score of the distorted image while ensuring the generalization ability of the model when facing a small sample data set with multiple distortion types.

[0109] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A full-reference image quality assessment method based on Conformer and meta-learning, characterized in that: The steps include: S1. Obtain a distorted image to be predicted and a corresponding reference image; S2, inputting the distorted image and the corresponding reference image into a pre-trained image quality assessment model; The image quality evaluation model includes: a feature extraction module, a similarity calculation module, a weight calculation module, an evaluation score module and a training and testing module based on meta-learning; S3, outputting the prediction score corresponding to the distorted image; The image quality assessment model training process in step S2 includes: S21, multiple types of distorted images and corresponding reference images are input as training data sets into a twin network with Conformer as a feature extraction module to extract feature vectors corresponding to the distorted images and the reference images respectively; S22, in a similarity calculation module, the feature vectors corresponding to the distorted image and the reference image are respectively calculated using two preset methods to obtain similarities between the feature vectors of the distorted image and the reference image; S23, in the weight calculation module, after fusing the feature vector of the distorted image and the feature vector of the reference image, the fused feature is mapped into the weight of the feature vector similarity through multiple fully connected layers and activation functions; S24, combining the similarity of the feature vectors of the distorted image and the reference image in step S22 and the weight of the feature vector similarity in step S23, calculating the final feature vector similarity of the distorted image and the reference image, and obtaining the final quality evaluation score of the distorted image; S25. Add meta-learning to the training and testing process of the model: In the training phase, by using the double gradient descent method from the support set to the query set, the model is allowed to learn to adjust the current model initialization parameters according to a part of the distortion types; in the testing phase, a small number of images of other distortion types are used to fine-tune the parameters of the model that has the ability to identify the distortion type after the training phase, so as to obtain the final image quality evaluation model; In the step S21, the feature vectors corresponding to the distorted image and the reference image are extracted by inputting the twin network with Conformer as the feature extraction module; including: The distorted image and the corresponding reference image are respectively input into the feature extraction network with Conformer as the backbone, and the corresponding global features are extracted using the self-attention mechanism; and the corresponding local features are extracted using the convolution operation; After the global features and the local features are embedded into each other, their dimensions are changed, and the feature vectors corresponding to the distorted image and the reference image are extracted.

2. The full-reference image quality assessment method based on Conformer and meta-learning according to claim 1, characterized in that: In step S22, two preset methods are used to calculate the similarity between the feature vectors of the distorted image and the reference image, including: The ProjectedDotProductSimilarity method is used to calculate the similarity1 between the distorted image feature vector and the reference image feature vector: (1)Where V1 represents the feature vector obtained after the reference image passes through the twin network; V2 represents the feature vector obtained after the distorted image passes through the twin network, and w1 and w2 represent random initialization vectors for adjusting the dimension of the feature vector; The TriLinearSimilarity method is used to calculate the similarity2 between the distorted image feature vector and the reference image feature vector: <h2 style=";text-align:left;direction:ltr">similarity2=w<h2 style=";text-align:left;direction:ltr"> T <h2 style=";text-align:left;direction:ltr"> [V1,V2,V1*V2]+b (2) (2)In the formula, V1*V2 represents the vector obtained by multiplying the corresponding elements of two feature vectors; [V1, V2, V1*V2] represents the result of concatenating the three vectors according to the last dimension; w represents the random initialization vector for adjusting the dimension of the concatenated vector, and b represents the bias term for fine-tuning the similarity result.

3. The full-reference image quality assessment method based on Conformer and meta-learning according to claim 2, characterized in that: The step S23 comprises: S231, the feature vector V1 of the reference image and the feature vector V2 of the distorted image are fused using the concat operation: V fuse =cat(V1,V2,1) (3) S232, obtain fusion feature V fuse After that, V is transformed through multiple fully connected layers and activation functions. fuse The result of the mapping is used as the weight parameter of the similarity Among them, FC1 and FC2 are fully connected layers, and f represents the sigmoid activation function.

4. The full-reference image quality assessment method based on Conformer and meta-learning according to claim 3, characterized in that: The step S24 comprises: Combined weight parameters Calculate the similarity between the distorted image feature vector and the reference image feature vector: Similarity represents the similarity between the distorted image and the reference image, which serves as the final distorted image quality prediction score.

5. The full-reference image quality assessment method based on Conformer and meta-learning according to claim 4, characterized in that: The step S25 adds meta-learning to the training and testing process of the model, specifically including: S251. During the training process, a part of the distortion types is selected, and the small sample data set is divided into subtasks, and each subtask is divided into a support set and a query set; S252, setting the loss function of the support set and the query set in the subtask; S253, during the training process, the first-order gradient of the loss function corresponding to the current subtask support set is calculated, and the current model parameters are updated through the Adam optimizer to complete the gradient optimization corresponding to the support set; S254: During the training process, based on the model parameters after step S253, the query set of the current subtask is input, and the current model parameters are updated again through the Adam optimizer, thereby completing the gradient optimization corresponding to the query set; by integrating the gradient update results of all subtasks, the prior model parameters that have a guiding role in the test phase are obtained; S255. During the test, a small number of images of other distortion types are input, and the model with prior parameters is updated using the Adam optimizer to obtain the final image quality evaluation model.

6. The full-reference image quality assessment method based on Conformer and meta-learning according to claim 5, characterized in that: The step S252 comprises: Set the loss function for the support set and query set in the subtask: Among them, e represents the quality label score corresponding to the distorted image, and s represents the quality prediction score corresponding to the distorted image.

Citation Information

Patent Citations

  • Three-dimensional image quality objective evaluation method based on feature extraction and ensemble learning

    CN112651922A

  • Full reference image quality assessment based on convolutional neural network

    US20160358321A1