Method and system for evaluating adaptive image quality during testing based on prior knowledge, terminal and medium

Abstract

The invention discloses a priori knowledge-based adaptive image quality evaluation method and system during testing, a terminal and a medium, and the method comprises the steps: obtaining image samples of a current batch, inputting all the image samples into a frozen image quality evaluation model, outputting a plurality of groups of prediction quality scores, carrying out clustering processing on all the predicted quality scores to obtain quality priori knowledge; updating the image quality evaluation model according to the quality priori knowledge to obtain a target image quality evaluation model; and then, inputting a to-be-evaluated image into the target image quality evaluation model, and outputting a final image quality evaluation result. According to the method, adaptive learning during testing is carried out by utilizing the quality priori knowledge, high-confidence relative quality is extracted by utilizing the quality priori knowledge so as to improve the stability of the model, and the influence of noise labels is avoided through a selective feature alignment method, so that the accuracy of image quality evaluation is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image quality evaluation, and particularly to a test-time adaptation image quality evaluation method, system, terminal and computer-readable storage medium based on prior knowledge. Background Art

[0002] Image quality directly affects people's visual perception and understanding. In application fields such as image compression, transmission, and storage, it is crucial to construct an accurate and reliable image quality evaluation model. However, in actual test scenarios, the distribution of test data often changes due to various factors, such as lighting conditions, capture devices, and environmental backgrounds. The emergence of this distribution change poses a huge challenge to existing image quality evaluation models because these models are usually trained for specific data distributions and lack adaptability to test data with distribution changes. In addition, in actual situations, since it is difficult to obtain source domain data and complete test data simultaneously, traditional domain adaptation methods become ineffective. Therefore, it is necessary to improve the robustness and accuracy of image quality evaluation models in real test scenarios.

[0003] However, in existing methods, test-time adaptation of image quality evaluation is achieved by introducing group contrast loss and permutation loss. That is, the group loss function divides a batch of images into high-quality and low-quality groups, and images in the same quality group have similar feature representations, and vice versa. The permutation loss applies different degrees of distortion to test images and makes images with higher degrees of distortion show a larger feature distance from the original images. However, simply aligning features in this way leads to incorrect knowledge generalization. At the same time, due to the lack of an effective filtering strategy to mitigate the impact of noisy labels.

[0004] Therefore, the prior art still needs to be improved and developed. Summary of the Invention

[0005] The main purpose of the present invention is to provide a test-time adaptation image quality evaluation method, system, terminal and medium based on prior knowledge, aiming to solve the problem that the prior art mainly focuses on simple feature alignment strategies, resulting in incorrect knowledge generalization, thus being unable to avoid the impact of noisy labels and leading to inaccurate evaluation of image quality.

[0006] To achieve the above object, the present invention provides a test-time adaptation image quality evaluation method based on prior knowledge, and the test-time adaptation image quality evaluation method based on prior knowledge includes the following steps:

[0007] Obtain the image samples of the current batch, input all the image samples into the frozen image quality evaluation model, output multiple groups of predicted quality scores, and perform clustering processing on all the predicted quality scores to obtain quality prior knowledge;

[0008] Update the image quality evaluation model according to the quality prior knowledge to obtain a target image quality evaluation model;

[0009] Obtain an image to be evaluated, input the image to be evaluated into the target image quality evaluation model, and output a final image quality evaluation result.

[0010] Optionally, in the method for test-time adaptation of image quality evaluation based on prior knowledge, before obtaining the current batch of image samples, inputting all the image samples into the frozen image quality evaluation model, outputting multiple groups of predicted quality scores, and performing clustering processing on all the predicted quality scores to obtain quality prior knowledge, further includes:

[0011] Construct an image quality evaluation test model and obtain the user's image quality evaluation requirements;

[0012] Freeze the parameters of the image quality evaluation test model according to the image quality evaluation requirements to obtain a frozen image quality evaluation model.

[0013] Optionally, in the method for test-time adaptation of image quality evaluation based on prior knowledge, where obtaining the current batch of image samples, inputting all the image samples into the frozen image quality evaluation model, outputting predicted quality scores, and performing clustering processing on all the predicted quality scores to obtain quality prior knowledge specifically includes:

[0014] Obtain the current batch of image samples of the target object, input all the image samples into the image quality evaluation model, and output multiple groups of predicted quality scores;

[0015] Perform clustering processing on all the predicted quality scores according to a clustering algorithm to obtain multiple different quality clusters, and use all the quality clusters as quality prior knowledge.

[0016] Optionally, in the method for test-time adaptation of image quality evaluation based on prior knowledge, where updating the image quality evaluation model according to the quality prior knowledge to obtain a target image quality evaluation model specifically includes:

[0017] Construct a relative quality matrix, and optimize the relative quality matrix according to the quality prior knowledge to obtain a high-confidence relative quality matrix;

[0018] Obtain multiple image features of all the image samples, perform selective alignment processing on all the image features to obtain a selective feature alignment result;

[0019] Combine the first loss function of the high-confidence relative quality matrix and the second loss function of the selective feature alignment result to obtain an objective loss function, and update the image quality evaluation model according to the objective loss function to obtain an objective image quality evaluation model.

[0020] Optionally, in the method for test-time adaptation image quality evaluation based on prior knowledge, wherein constructing a relative quality matrix and optimizing the relative quality matrix according to the quality prior knowledge to obtain a high-confidence relative quality matrix specifically includes:

[0021] Calculate the difference between the predicted quality scores of each pair of image samples to obtain a quality score difference result, and construct a matrix according to the quality score difference result to obtain a relative quality matrix;

[0022] Generate a confidence filtering strategy according to the quality prior knowledge, and optimize the relative quality matrix according to the confidence filtering strategy to obtain a high-confidence relative quality matrix.

[0023] Optionally, in the method for test-time adaptation image quality evaluation based on prior knowledge, wherein obtaining multiple image features of all the image samples and performing selective alignment processing on all the image features to obtain a selective feature alignment result specifically includes:

[0024] Extract features from all the image samples to obtain multiple image features, and obtain the cosine similarity between all the image features;

[0025] Determine the feature distance between all the image features according to the cosine similarity, and perform selective alignment processing on some adjacent image features according to the feature distance to obtain a selective feature alignment result.

[0026] Optionally, in the method for test-time adaptation image quality evaluation based on prior knowledge, wherein the expression of the objective loss function is:

[0027]

[0028] The expression of the first loss function is:

[0029]

[0030] The expression of the second loss function is:

[0031]

[0032] Wherein, is the objective loss function, is the first loss function, is the second loss function, is the probability mapped by the function, is the high confidence part of the high confidence relative quality matrix, θ is the network parameter, |A| is the total number of combinations, K is the number of quality clusters, A k is the set of nearest neighbor samples in the kth cluster, sim(z i ,z j ) is the image feature z i and image feature z j The cosine similarity between .

[0033] Optionally, the method for adaptive image quality assessment during testing based on prior knowledge, wherein the system for adaptive image quality assessment during testing based on prior knowledge comprises:

[0034] A data clustering module is used to obtain the current batch of image samples, input all the image samples into the image quality assessment model, output multiple groups of predicted quality scores, and perform clustering processing on all the predicted quality scores to obtain quality prior knowledge;

[0035] A model updating module, used for updating the image quality evaluation model according to the quality prior knowledge to obtain a target image quality evaluation model;

[0036] The quality assessment module is used to obtain the image to be evaluated, input the image to be evaluated into the target image quality assessment model, and output the final image quality assessment result.

[0037] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and a test-time adaptive image quality assessment program based on prior knowledge stored in the memory and executable on the processor, wherein the test-time adaptive image quality assessment program based on prior knowledge implements the steps of the test-time adaptive image quality assessment method based on prior knowledge as described above when executed by the processor.

[0038] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a test-time adaptive image quality assessment program based on prior knowledge, and when the test-time adaptive image quality assessment program based on prior knowledge is executed by a processor, it implements the steps of the test-time adaptive image quality assessment method based on prior knowledge as described above.

[0039] In the present invention, multiple sets of image samples are obtained, all the image samples are input into the image quality evaluation model, multiple sets of predicted quality scores are output, and clustering processing is performed on all the predicted quality scores to obtain quality prior knowledge; the image quality evaluation model is updated according to the quality prior knowledge to obtain a target image quality evaluation model; a to-be-evaluated image is obtained, and the to-be-evaluated image is input into the target image quality evaluation model to output a final image quality evaluation result. The present invention adapts learning during testing by using quality prior knowledge. Specifically, by using quality prior knowledge, relatively high-confidence quality is extracted to improve the stability of model training, and by using a selective feature alignment method, the influence of noisy labels is avoided, thereby improving the accuracy of image quality evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flowchart of a preferred embodiment of the method for test-time adaptation of image quality evaluation based on prior knowledge of the present invention;

[0041] Figure 2 is a schematic structural diagram corresponding to the method for test-time adaptation of image quality evaluation based on prior knowledge of the present invention;

[0042] Figure 3 is a schematic diagram of a relative quality matrix in a preferred embodiment of the present invention;

[0043] Figure 4 is a structural diagram of a preferred embodiment of the test-time adaptation image quality evaluation system based on prior knowledge of the present invention;

[0044] Figure 5 is a schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0045] To make the objectives, technical solutions and advantages of the present invention clearer and more definite, the following further describes the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0046] It should be noted that if there are directional indications (such as up, down, left, right, front, back...) involved in the embodiments of the present invention, the directional indications are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the drawings). If the specific posture changes, the directional indications will also change accordingly.

[0047] In addition, if there are descriptions such as "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only for descriptive purposes and should not be construed as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions between various embodiments may be combined with each other, but it must be based on the ability of those of ordinary skill in the art to implement. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0048] The method for evaluating the quality of test-time adaptation images based on prior knowledge according to a preferred embodiment of the present invention, as Figure 1 shown, the method for evaluating the quality of test-time adaptation images based on prior knowledge includes the following steps:

[0049] Step S10: Obtain the image samples of the current batch, input all the image samples into the frozen image quality evaluation model, output multiple groups of predicted quality scores, and perform clustering processing on all the predicted quality scores to obtain quality prior knowledge.

[0050] The step S10 includes:

[0051] Step S11: Obtain the image samples of the current batch of the target object, input all the image samples into the image quality evaluation model, and output multiple groups of predicted quality scores;

[0052] Step S12: Perform clustering processing on all the predicted quality scores according to the clustering algorithm to obtain multiple different quality clusters, and use all the quality clusters as quality prior knowledge.

[0053] Specifically, the current image quality evaluation models usually lack the ability to adapt to the test data with distribution shift. To solve the problem that the distribution change between the training data and the test data leads to the degradation of the model performance; the existing methods mainly focus on simple feature alignment strategies, which will result in incorrect knowledge generalization. In the embodiments of the present invention, to solve the above problems, a method for evaluating the quality of test-time adaptation images based on prior knowledge is proposed, and the architecture diagram corresponding to the method for evaluating the quality of test-time adaptation images based on prior knowledge is as Figure 2As shown, the prior - knowledge - based test - time adaptation image quality evaluation method can be divided into a training stage and a testing stage. In the training stage, a batch of image samples are used to adjust the model. Specifically, first, an image quality evaluation test model is constructed, and the user's image quality evaluation requirements are obtained. Then, the state of the image quality evaluation test model needs to be changed to the test state to ensure that the parameters of the normalization layer are not affected. According to the image quality evaluation requirements, the parameters of the image quality evaluation test model are fixed to obtain a frozen image quality evaluation model.

[0054] After that, quality prior knowledge needs to be extracted through the frozen image quality evaluation model. Specifically, the current batch of image samples of the target object is obtained, and all pre - processed image samples are input into the frozen image quality evaluation model to output multiple groups of predicted quality scores (i.e., predicted values), such as predicted value 1, predicted value 2,..., predicted value n. Subsequently, all the predicted quality scores are clustered. By grouping images with similar quality into the same cluster, data can be better understood and organized. Specifically, using a clustering algorithm, images with closer quality are grouped into one cluster, that is, multiple different quality clusters are obtained, and the images in the same quality cluster have similar quality. All the quality clusters are used as quality prior knowledge.

[0055] Step S20: Update the image quality evaluation model according to the quality prior knowledge to obtain a target image quality evaluation model.

[0056] The step S20 includes:

[0057] Step S21: Construct a relative quality matrix and optimize the relative quality matrix according to the quality prior knowledge to obtain a high - confidence relative quality matrix;

[0058] Step S22: Obtain multiple image features of all the image samples, perform selective alignment processing on all the image features to obtain a selective feature alignment result;

[0059] Step S23: Combine the first loss function of the high - confidence relative quality matrix and the second loss function of the selective feature alignment result to obtain a target loss function, and update the image quality evaluation model according to the target loss function to obtain a target image quality evaluation model.

[0060] Specifically, after obtaining the quality prior knowledge, which is the basis for subsequent model optimization. In the present invention, the corresponding optimization is carried out from two complementary perspectives, namely, between clusters and within clusters of quality clusters; from the perspective between clusters, a high-confidence ranking learning method is proposed, which consists of a relative quality matrix and a confidence filtering strategy to generate a high-confidence quality ranking; from the perspective within clusters, a selective feature alignment method is proposed, that is, only the nearest adjacent samples within the same quality cluster are aligned to reduce the influence of noisy labels. This two-layer design takes into account the requirements of global ranking and local alignment, effectively improving the performance of the model. To avoid overfitting, only the normalization layer is updated; for the high-confidence ranking learning method, since the existing test-time adaptation research directly uses entropy as the loss function for learning, but image quality assessment, as a regression task, cannot directly use the existing test-time adaptation loss function. Therefore, in the implementation of the invention, the concept of relative quality is proposed, and a relative quality matrix is formed by using the predicted quality scores of a batch of images; specifically, a relative quality matrix is generated by calculating the predicted quality differences of each pair of image samples. The corresponding process is to calculate the difference in the predicted quality scores between each pair of image samples. For example, subtract the predicted quality score corresponding to the i-th image from the predicted quality score corresponding to the j-th image to obtain the result of the quality score difference, and construct a matrix according to the quality score difference result to obtain the relative quality matrix, where the relative quality matrix is as Figure 3 shown. In addition, due to the distribution change in the test data, the relative quality matrix may not accurately reflect the relative quality between two images. By utilizing the quality prior knowledge, a confidence filtering strategy is further proposed, believing that the relative quality generated by different clusters in the matrix is highly confident, and vice versa. Specifically, a confidence filtering strategy is generated according to the quality prior knowledge, and the relative quality matrix is optimized according to the confidence filtering strategy to obtain a high-confidence relative quality matrix.

[0061] For the selective feature alignment method, the cosine similarity of image features is used to measure the distance between different image samples within a cluster in the feature space. Specifically, feature extraction is performed on all the image samples to obtain multiple image features, and the cosine similarity between all the image features is obtained; then, the feature distance between all the image features is determined according to the cosine similarity, and all adjacent image features are selectively aligned according to the feature distance to obtain the selective feature alignment result. The present invention avoids the influence of noisy labels by only aligning the nearest feature samples within a cluster.

[0062] After that, it is necessary to update the image quality evaluation model. Specifically, the first loss function of the high-confidence relative quality matrix and the second loss function of the selective feature alignment result are respectively obtained. Among them, the expression of the first loss function is: And for The expression of is: For The expression of is: And the expression of M is: Among them, Is the first loss function, Is the probability mapped by the function, Is the high-confidence part in the high-confidence relative quality matrix, Φ is the standard normal cumulative distribution function, and α is the modulation parameter. Is the masked high-confidence relative quality matrix, RQM is the relative quality matrix, M is the mask matrix, x i And x j Are both indices in the training images, c k Is the cluster of clustering; the expression of the second loss function is: Among them, Is the second loss function, |A| is the total number of combinations, K is the number of quality clusters, A k Is the set of nearest neighbor samples in the k-th cluster, sim(z i , z j ) is the cosine similarity between the image feature z i And the image feature z j ; θ is the network parameter.

[0063] Subsequently, the first loss function and the second loss function are combined to obtain the target loss function. Among them, the expression of the target loss function is: After that, the image quality evaluation model is updated according to the target loss function to obtain the target image quality evaluation model.

[0064] Furthermore, after obtaining the target image quality evaluation model, in the embodiments of the present invention, in order to test the performance of the target image quality evaluation model, experiments are carried out on multiple image quality models and data sets, and the corresponding experimental results are shown in Table 1.

[0065] Table 1: Performance results of multiple image quality models under the distribution change of test data

[0066]

[0067] As shown in Table 1, the bolded ones in Table 1 are the optimal results. Among them, SRCC is the Spearman rank correlation coefficient, and PLCC is the Pearson linear correlation coefficient. These two coefficients are important criteria for evaluating the quality of an image quality evaluation model, ranging from [0, 1]. The higher the value of the index, the better the performance. Experiments were conducted on multiple image quality evaluation models, including the TRes model, MUSIQ model, HyperIQA model, and MetaIQA model. At the same time, multiple commonly used image quality evaluation datasets were involved in the experiments, such as the KonIQ-10K dataset, BID dataset, CLIVE dataset, and LIVE dataset. For the selection of the training source domain, a large-scale real dataset (FLIVE) was selected. Baseline in Table 1 represents the original effect of the model on this dataset, TTA-IQA is the current latest method, and ours is the method proposed in the present invention. The results in Table 1 show the superiority of the method proposed in the present invention on multiple models and datasets, indicating that the method of the present invention can more effectively adjust the model and improve the performance of the model under the distribution change of the test data.

[0068] The present invention proposes to extract the quality prior knowledge of a pre-trained image quality model by means of clustering and use the quality prior knowledge for learning during testing adaptation. Specifically, a high-confidence ranking learning method is proposed between clusters, that is, in the face of the problem that the existing loss function for testing adaptation cannot be directly applied to the image quality evaluation model for regression tasks, it is proposed to use relative quality for learning; a selective feature alignment method is proposed within clusters, that is, the cosine similarity of features is used to measure the distance between different samples within the cluster in the feature space, and by only aligning the closest feature samples within the cluster, the influence of noisy labels is avoided, thereby improving the accuracy of the evaluation of image quality.

[0069] Step S30: Obtain the image to be evaluated, input the image to be evaluated into the target image quality evaluation model, and output the final image quality evaluation result.

[0070] Specifically, in the embodiment of the present invention, after obtaining the updated model (i.e., the target image quality evaluation model), it is necessary to predict the quality score of the corresponding image. Specifically, obtain the image to be evaluated, input the image to be evaluated into the target image quality evaluation model, output the corresponding quality score, and obtain the corresponding image quality evaluation result according to the quality score.

[0071] Furthermore, as Figure 4 shown, based on the above test-time adaptation image quality evaluation method based on prior knowledge, the present invention also correspondingly provides a test-time adaptation image quality evaluation system based on prior knowledge, wherein the test-time adaptation image quality evaluation system based on prior knowledge includes:

[0072] A data clustering module 51, configured to obtain image samples of the current batch, input all the image samples into a frozen image quality evaluation model, output multiple groups of predicted quality scores, and perform clustering processing on all the predicted quality scores to obtain quality prior knowledge;

[0073] A model update module 52, configured to update the image quality evaluation model according to the quality prior knowledge to obtain a target image quality evaluation model;

[0074] A quality evaluation module 53, configured to obtain an image to be evaluated, input the image to be evaluated into the target image quality evaluation model, and output a final image quality evaluation result.

[0075] Further, as Figure 5 shown, based on the above prior-knowledge-based test-time adaptation image quality evaluation method, the present invention also correspondingly provides a terminal, which includes a processor 10, a memory 20, and a display 30. Figure 5 Only some components of the terminal are shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0076] The memory 20 may be an internal storage unit of the terminal in some embodiments, such as a hard disk or a memory of the terminal. The memory 20 may also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the terminal. Further, the memory 20 may also include both the internal storage unit and the external storage device of the terminal. The memory 20 is used to store application software installed on the terminal and various types of data, such as program codes installed on the terminal, etc. The memory 20 may also be used to temporarily store data that has been output or will be output. In one embodiment, a prior-knowledge-based test-time adaptation image quality evaluation program 40 is stored on the memory 20, and the prior-knowledge-based test-time adaptation image quality evaluation program 40 can be executed by the processor 10, so as to implement the prior-knowledge-based test-time adaptation image quality evaluation method in the present application.

[0077] The processor 10 may be a central processing unit (CPU), a microprocessor, or other data processing chips in some embodiments, and is used to run program codes stored in the memory 20 or process data, such as executing the prior-knowledge-based test-time adaptation image quality evaluation method, etc.

[0078] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) toucher, etc. The display 30 is used to display information of the terminal and to display a visual user interface. Components of the terminal communicate with each other via a system bus.

[0079] In one embodiment, when the processor 10 executes the test based on prior knowledge in the memory 20 and adapts to the image quality evaluation program 40, the following steps are implemented:

[0080] Obtain image samples of the current batch, input all the image samples into the frozen image quality evaluation model, output multiple sets of predicted quality scores, and perform clustering processing on all the predicted quality scores to obtain quality prior knowledge;

[0081] Update the image quality evaluation model according to the quality prior knowledge to obtain a target image quality evaluation model;

[0082] Obtain an image to be evaluated, input the image to be evaluated into the target image quality evaluation model, and output a final image quality evaluation result.

[0083] Among them, before the step of obtaining multiple sets of image samples, inputting all the image samples into the image quality evaluation model, outputting multiple sets of predicted quality scores, and performing clustering processing on all the predicted quality scores to obtain quality prior knowledge, the following steps are further included:

[0084] Construct an image quality evaluation test model and obtain the user's image quality evaluation requirements;

[0085] Freeze the parameters of the image quality evaluation test model according to the image quality evaluation requirements to obtain a frozen image quality evaluation model.

[0086] Among them, the step of obtaining image samples of the current batch, inputting all the image samples into the frozen image quality evaluation model, outputting multiple sets of predicted quality scores, and performing clustering processing on all the predicted quality scores to obtain quality prior knowledge specifically includes:

[0087] Obtain image samples of the current batch of the target object, input all the image samples into the image quality evaluation model, and output multiple sets of predicted quality scores;

[0088] Perform clustering processing on all the predicted quality scores according to a clustering algorithm to obtain multiple different quality clusters, and use all the quality clusters as quality prior knowledge.

[0089] Among them, updating the image quality evaluation model according to the quality prior knowledge to obtain a target image quality evaluation model specifically includes:

[0090] Construct a relative quality matrix, and optimize the relative quality matrix according to the quality prior knowledge to obtain a high-confidence relative quality matrix;

[0091] Obtain multiple image features of all the image samples, perform selective alignment processing on all the image features to obtain a selective feature alignment result;

[0092] Combine the first loss function of the high-confidence relative quality matrix and the second loss function of the selective feature alignment result to obtain a target loss function, and update the image quality evaluation model according to the target loss function to obtain a target image quality evaluation model.

[0093] Among them, constructing a relative quality matrix, and optimizing the relative quality matrix according to the quality prior knowledge to obtain a high-confidence relative quality matrix specifically includes:

[0094] Calculate the difference in predicted quality scores between each group of image samples to obtain a quality score difference result, and construct a matrix according to the quality score difference result to obtain a relative quality matrix;

[0095] Generate a confidence filtering strategy according to the quality prior knowledge, and optimize the relative quality matrix according to the confidence filtering strategy to obtain a high-confidence relative quality matrix.

[0096] Among them, obtaining multiple image features of all the image samples, performing selective alignment processing on all the image features to obtain a selective feature alignment result specifically includes:

[0097] Extract features from all the image samples to obtain multiple image features, and obtain the cosine similarity between all the image features;

[0098] Determine the feature distance between all image features according to the cosine similarity, and perform selective alignment processing on some adjacent image features according to the feature distance to obtain a selective feature alignment result.

[0099] Among them, the expression of the target loss function is:

[0100]

[0101] The expression of the first loss function is:

[0102]

[0103] The expression of the second loss function is as follows:

[0104]

[0105] Wherein, is the target loss function, is the first loss function, is the second loss function, is the probability mapped by the function, is the high-confidence part in the high-confidence relative quality matrix, |A| is the total number of combinations, K is the number of quality clusters, and A k is the set of nearest neighbor samples in the k-th cluster, sim(z i , z j ) is the cosine similarity between the image feature z i and the image feature z j , and θ is the network parameter.

[0106] The present invention also provides a computer-readable storage medium. Wherein, the computer-readable storage medium stores a test-time adaptation image quality evaluation program based on prior knowledge. When the test-time adaptation image quality evaluation program based on prior knowledge is executed by a processor, the steps of the test-time adaptation image quality evaluation method based on prior knowledge as described above are implemented.

[0107] In summary, the present invention provides a test-time adaptation image quality evaluation method, system, terminal and medium based on prior knowledge. The method includes: obtaining multiple groups of image samples, inputting all the image samples into the image quality evaluation model, outputting multiple groups of predicted quality scores, and performing clustering processing on all the predicted quality scores to obtain quality prior knowledge; updating the image quality evaluation model according to the quality prior knowledge to obtain a target image quality evaluation model; obtaining an image to be evaluated, and inputting the image to be evaluated into the target image quality evaluation model to output a final image quality evaluation result. The present invention performs test-time adaptation learning by using quality prior knowledge. Specifically, by using quality prior knowledge, high-confidence relative quality is extracted to improve the stability of the model, and the influence of noise labels is avoided through a selective feature alignment method, thereby improving the accuracy of image quality evaluation.

[0108] It should be noted that in this text, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or terminal. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or terminal comprising such element.

[0109] Of course, those of ordinary skill in the art can understand that all or part of the processes of implementing the above-described embodiments of the method can be completed by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program. The program can be stored in a computer-readable storage medium that can be read by a computer. When the program is executed, it can include the processes of the above-described method embodiments. The computer-readable storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0110] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description. All such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. A test-time adaptive image quality assessment method based on prior knowledge, characterized in that: The test-time adaptive image quality evaluation method based on prior knowledge includes: Obtaining image samples of the current batch, inputting all the image samples into a frozen image quality assessment model, outputting multiple groups of predicted quality scores, and performing clustering processing on all the predicted quality scores to obtain quality prior knowledge; The image quality assessment model is updated according to the quality prior knowledge to obtain a target image quality assessment model; The image to be evaluated is obtained, the image to be evaluated is input into the target image quality evaluation model, and a final image quality evaluation result is output.

2. The test-time adaptive image quality assessment method based on prior knowledge according to claim 1, characterized in that: The method further includes obtaining image samples of the current batch, inputting all the image samples into a frozen image quality assessment model, outputting multiple groups of predicted quality scores, and performing clustering processing on all the predicted quality scores to obtain quality prior knowledge; Build an image quality evaluation test model and obtain users' image quality evaluation requirements; The parameters of the image quality assessment test model are frozen according to the image quality assessment requirements to obtain a frozen image quality assessment model.

3. The test-time adaptive image quality assessment method based on prior knowledge according to claim 1, characterized in that: The method of obtaining the image samples of the current batch, inputting all the image samples into the frozen image quality assessment model, outputting multiple groups of predicted quality scores, and performing clustering processing on all the predicted quality scores to obtain quality prior knowledge specifically includes: Obtaining a current batch of image samples of the target object, inputting all of the image samples into the image quality assessment model, and outputting multiple groups of predicted quality scores; All the predicted quality scores are clustered according to a clustering algorithm to obtain a plurality of different quality clusters, and all the quality clusters are used as quality prior knowledge.

4. The test-time adaptive image quality assessment method based on prior knowledge according to claim 1, characterized in that: The updating process of the image quality assessment model according to the quality prior knowledge to obtain a target image quality assessment model specifically includes: Constructing a relative mass matrix, and optimizing the relative mass matrix according to the mass prior knowledge to obtain a high-confidence relative mass matrix; Acquire multiple image features of all the image samples, perform selective alignment processing on all the image features, and obtain a selective feature alignment result; The first loss function of the high confidence relative quality matrix and the second loss function of the selective feature alignment result are combined to obtain a target loss function, and the image quality assessment model is updated according to the target loss function to obtain a target image quality assessment model.

5. The test-time adaptive image quality assessment method based on prior knowledge according to claim 4, characterized in that: The step of constructing a relative quality matrix and optimizing the relative quality matrix according to the quality prior knowledge to obtain a high confidence relative quality matrix specifically includes: Calculating the difference of the predicted quality scores between each group of image samples to obtain a quality score difference result, and constructing a matrix according to the quality score difference result to obtain a relative quality matrix; A confidence filtering strategy is generated according to the quality prior knowledge, and the relative quality matrix is ​​optimized according to the confidence filtering strategy to obtain a high-confidence relative quality matrix.

6. The test-time adaptive image quality assessment method based on prior knowledge according to claim 4, characterized in that: The step of acquiring multiple image features of all the image samples and performing selective alignment processing on all the image features to obtain a feature alignment result specifically includes: Performing feature extraction on all the image samples to obtain multiple image features, and obtaining cosine similarities between all the image features; The feature distances between all image features are determined according to the cosine similarity, and selective alignment processing is performed on some adjacent image features according to the feature distance to obtain a selective feature alignment result.

7. The test-time adaptive image quality assessment method based on prior knowledge according to claim 4, characterized in that: The expression of the objective loss function is: The expression of the first loss function is: The expression of the second loss function is: in, is the target loss function, is the first loss function, is the second loss function, is the probability mapped by the function, is the high confidence part of the high confidence relative quality matrix, θ is the network parameter, |A| is the total number of combinations, K is the number of quality clusters, A k is the set of nearest neighbor samples in the kth cluster, sim(z i ,z j ) is the image feature z i and image feature z j The cosine similarity between .

8. A test-time adaptive image quality assessment system based on prior knowledge, characterized in that: The test-time adaptive image quality assessment system based on prior knowledge includes: A data clustering module is used to obtain image samples of the current batch, input all the image samples into a frozen image quality assessment model, output multiple groups of predicted quality scores, and perform clustering processing on all the predicted quality scores to obtain quality prior knowledge; A model updating module, used for updating the image quality evaluation model according to the quality prior knowledge to obtain a target image quality evaluation model; The quality assessment module is used to obtain the image to be evaluated, input the image to be evaluated into the target image quality assessment model, and output the final image quality assessment result.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a test-time adaptive image quality assessment program based on prior knowledge stored in the memory and executable on the processor. When the test-time adaptive image quality assessment program based on prior knowledge is executed by the processor, the steps of the test-time adaptive image quality assessment method based on prior knowledge as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program for adaptive image quality assessment during testing based on prior knowledge, and when the program for adaptive image quality assessment during testing based on prior knowledge is executed by a processor, the steps of the method for adaptive image quality assessment during testing based on prior knowledge as described in any one of claims 1-7 are implemented.

Images