A no-reference image quality assessment method based on local manifold learning

CN118229633BActive Publication Date: 2026-08-21XIAMEN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410298833.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-03-15
Publication Date
2026-08-21
Estimated Expiration
2044-03-15

AI Technical Summary

Technical Problem

而且,在现实世界的场景中,由于观众对不同内容的偏好,图像质量与其内容密切相关,这也意味着相同图像的不同裁剪对应的质量感知可能是不同的,并且局部图像补丁的感知得分往往与整个图像的得分不同

Benefits of technology

通过对创建的教师模型以及学生模型进行初始化;接着获取训练图像,对训练图像进行裁剪得到训练子图像,将各训练图像输入教师模型得到注意力图,基于注意力图获取各训练图像的视觉显著区域图,将各训练子图像以及视觉显著区域图输入学生模型得到子图像特征以及显著区域特征;接着对子图像特征以及显著区域特征进行局部流形学习并计算对比损失,得到对比损失值;将训练子图像输入学生模型计算标签损失,得到标签损失值;接着基于对比损失值以及标签损失值计算总损失值,基于总损失值更新学生模型,基于EMA算法更新教师模型,直至学生模型的图像质量评估精度大于预设的精度阈值,最后利用学生模型进行图像质量评估;即为缓解局部流形崩溃问题,引入了视觉显著性裁剪和教师模型与学生模型相互学习机制,相比传统将同一图像的随机裁剪图像视为正类,创新地采用训练图像的视觉显著区域图作为正类,而将训练图像的其他随机裁剪视为类内负类,这确保了局部流形的一致性,而不会导致局部流形塌陷;此外,为了区分整体流形,将所有其他图像的裁剪标记为类间负类;还设计了师生裁剪机制,即使用教师模型进行视觉显著性裁剪,为学生模型提供对比学习的正样本,然后学生模型使用EMA算法更新教师模型,这种双重学习提高了视觉显著性裁剪的准确性和稳定性,有助于后续的比较学习,最终极大的提升了图像质量评估的准确性。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118229633B_ABST
    Figure CN118229633B_ABST
Patent Text Reader

Abstract

The application provides a kind of local manifold learning-based no-reference image quality assessment method in the technical field of computer vision, comprising: step S1, initializing teacher model and student model;Step S2, obtain training image and cut to obtain training sub-image;Step S3, input training image into teacher model to obtain attention map, and obtain visual saliency region map of training image based on attention map;Step S4, input training sub-image and visual saliency region map into student model to obtain sub-image feature and saliency region feature;Step S5, local manifold learning is carried out to each feature, and contrast loss is calculated, and training sub-image is input into student model to calculate label loss;Step S6, total loss value is calculated based on contrast loss value and label loss value to update student model, and teacher model is updated based on EMA algorithm;Step S7, image quality assessment is carried out using student model.The application has the advantage that the accuracy of image quality assessment is greatly improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a no-reference image quality assessment method based on local manifold learning. Background Technology

[0002] Image quality assessment is a crucial problem in computer vision technology, aiming to quantify and simulate the subjective evaluation of distorted images by the human perceptual system. In the digital age, a vast number of images are disseminated daily on social media platforms such as Facebook and TikTok. However, during transmission, storage, and processing, images are often affected by various noises, distortions, and compression, which can impact their visual quality to varying degrees. To ensure accurate image transmission, improve user experience, and support decision-making and analysis across various fields, image quality assessment has become increasingly important. Although full-reference image quality assessment (IQA) has made considerable progress, its requirement for high-quality reference images makes no-reference image quality assessment (NR-IQA) more attractive and applicable. In recent years, researchers have focused on solving the NR-IQA problem, with the following main methods: Method 1: Directly use or fine-tune a model pre-trained on a large dataset as a feature extractor to further predict image quality scores. However, these pre-trained models may be suboptimal for the NR-IQA task because images with the same content share the same semantic labels, but their quality may differ.

[0003] Method 2: This method employs a self-supervised approach, first performing contrastive learning on a large-scale dataset to distinguish samples with different perceptual qualities, and then fine-tuning on the IQA dataset. This method often categorizes different croppings of the same image into the same level of perceptual quality, treating all croppings from the same image as positive samples and the rest as negative samples. If multiple croppings are performed on an image to expand the training data, manifold collapse occurs because there are no constraints on intra-class features; that is, although class boundaries are clearly distinguished, intra-class features cluster together. Furthermore, in real-world scenarios, image quality is closely related to content due to viewer preferences, meaning that different croppings of the same image may correspond to different perceived quality, and the perceptual score of a local image patch often differs from the score of the entire image.

[0004] Therefore, how to provide a referenceless image quality assessment method based on local manifold learning to improve the accuracy of image quality assessment has become an urgent technical problem to be solved. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a no-reference image quality assessment method based on local manifold learning, thereby improving the accuracy of image quality assessment.

[0006] This invention is implemented as follows: a no-reference image quality assessment method based on local manifold learning, comprising the following steps: Step S10: Create a teacher model and a student model for image quality assessment, and initialize the teacher model and the student model; Step S20: Obtain training images and crop the training images to obtain training sub-images; Step S30: Input each of the training images into the teacher model to obtain an attention map, and obtain a visual salient region map of each training image based on the attention map; Step S40: Input each of the training sub-images and the visual salient region map into the student model to obtain sub-image features and salient region features; Step S50: Perform local manifold learning on the sub-image features and salient region features and calculate the contrast loss to obtain the contrast loss value; input the training sub-image into the student model to calculate the label loss to obtain the label loss value; Step S60: Calculate the total loss value based on the contrast loss value and the label loss value, update the student model based on the total loss value, and update the teacher model based on the EMA algorithm until the image quality assessment accuracy of the student model is greater than the preset accuracy threshold. Step S70: Use the student model to perform image quality assessment.

[0007] Furthermore, in step S10, the encoders of the teacher model and the student model use a pre-trained Transformer model, and the decoders use a Transformer model decoder.

[0008] Furthermore, step S20 specifically includes: Acquire training images For the training images Ten random cropping operations of size 224*224 were performed to obtain training sub-images. ;in, , h represents the cropping number, n represents the training image number, and N represents the total number of training images.

[0009] Furthermore, step S30 specifically includes: Step S31: Transfer each of the training sub-images Cut into T image blocks, and then cut each image block into T image blocks. pieceInputting the projection layer of the teacher model yields T image patch tokens, and then adding an additional class token yields T+1 tokens; Each of the aforementioned tokens is input into two different transmission layers of the teacher model to obtain Q and K; Calculate the global attention matrix based on Q and K: ; Where Q represents the Query matrix for attention calculation; K represents the Key matrix for attention calculation; and t represents the transpose. , R represents a real number; D represents the feature dimension. Represents the global attention matrix; This represents the normalized exponential function; Step S32: Based on the global attention matrix Calculate the attention weight matrix of class tokens to image patch tokens: ; in, This represents the attention weight matrix of the class token to the image patch token, i.e., the degree of attention. ; In this context, "0" represents the class token index, and "1:T+1" represents the image block token index; The shape of the attention weight matrix of the aforementioned token class for image patch tokens is transformed from one dimension to two dimensions, i.e. Change to ; The attention weight matrix of class tokens for image patch tokens is extracted from the last O layers of the encoder of the teacher model, and its average value is calculated to obtain the average attention level. ; in, Indicates the average level of attention; The teacher model encoder is represented by the first... The attention weight matrix of the class tokens of the layer network for the image patch tokens; Indicates the first layer; Find the average level of attention. The region with the largest sum of values ​​and a size of M*M is used as the training image. Visual salient region map: ; in, q represents the visual saliency region map; p represents the starting row coordinate of the top left corner of the visual saliency region map; q represents the starting column coordinate of the top left corner of the visual saliency region map; i represents the row increment of the visual saliency region map; j represents the column increment of the visual saliency region map.

[0010] Furthermore, in step S40, the calculation formulas for the sub-image features and salient region features are as follows: ; ; in, Representing sub-image features; Indicates the features of visually salient regions; The encoder representing the student model; Represents the training sub-image; Represents the visually salient region map; || is the L1 norm.

[0011] Furthermore, in step S50, the calculation process of the contrast loss value is as follows: Based on the sub-image features Calculate the negative class within the class: ; Based on the sub-image features Compute the negative class between classes: ; The salient region features As a positive class, the comparative loss value is calculated by combining the negative classes within the class and the negative classes between the classes: ; in, Indicates a negative class within a class; Indicates a negative class between classes; This represents the contrast loss value; H represents the number of times an image is cropped. Indicates the cropping index of the image; This represents the nth image. Image features of a cropped image; The temperature hyperparameter represents the contrast loss; n represents the image index; Indicates the image index; Indicates the cropping index of the image; Represents the nth image. A cropped image feature.

[0012] Furthermore, in step S50, the calculation process for the tag loss value is as follows: The training sub-image The student model is input, and its encoder, decoder, and multilayer perceptron process are used to obtain the predicted score. ; The label loss value of the training sub-image is calculated based on the predicted score: ; in, Indicates the predicted score; Indicates the label loss value; This represents a multilayer perceptron; The decoder representing the student model; The encoder represents the student model; H represents the number of times an image is cropped; h represents the cropping number. This represents the prediction score for the h-th crop of the n-th image; This represents L1 loss.

[0013] Furthermore, in step S60, the formula for calculating the total loss value is: ; in, This represents the total loss value; Indicates the label loss value; Indicates the comparative loss value; represents the balance factor; N represents the total number of training images; n represents the training image number.

[0014] Furthermore, in step S60, updating the teacher model based on the EMA algorithm specifically involves: ; in, This represents the updated teacher model parameters; This represents the teacher model parameters before the update; Indicates the parameters of the student model; This represents the balance factor.

[0015] The advantages of this invention are: The process begins by initializing the created teacher and student models. Next, training images are acquired, cropped to obtain sub-images, and input into the teacher model to generate attention maps. Based on these attention maps, visual salient region maps are obtained for each training image. These sub-images and their corresponding visual salient region maps are then input into the student model to obtain sub-image features and salient region features. Local manifold learning is then performed on these sub-image and salient region features, and contrastive loss is calculated to obtain contrastive loss values. The training sub-images are then input into the student model to calculate label loss values. Finally, the total loss value is calculated based on the contrastive and label loss values. The student model is updated based on the total loss value, and the teacher model is updated using the EMA algorithm until the image quality assessment accuracy of the student model exceeds a preset accuracy threshold. Finally, the student model is used to perform image quality assessment. To mitigate the local manifold collapse problem, quality assessment employs visual saliency cropping and a teacher-student model mutual learning mechanism. Compared to the traditional approach of treating randomly cropped images of the same image as positive classes, this innovative approach uses visually salient regions of the training image as positive classes, while treating other random cropped images of the training image as intra-class negative classes. This ensures the consistency of the local manifold without causing collapse. Furthermore, to distinguish the overall manifold, all other image croppings are labeled as inter-class negative classes. A teacher-student cropping mechanism is also designed, where the teacher model performs visual saliency cropping, providing positive samples for comparative learning to the student model. The student model then updates the teacher model using the EMA algorithm. This dual learning improves the accuracy and stability of visual saliency cropping, aids subsequent comparative learning, and ultimately greatly enhances the accuracy of image quality assessment. Attached Figure Description

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0017] Figure 1 This is a flowchart of a no-reference image quality assessment method based on local manifold learning according to the present invention.

[0018] Figure 2 This is a comparative diagram of the NR-IQA contrastive learning paradigm.

[0019] Figure 3 This is a schematic diagram of the framework of the present invention.

[0020] Figure 4 This is a schematic diagram comparing the prediction results of the present invention with those of the benchmark model. Detailed Implementation

[0021] The overall approach of the technical solution in this application is as follows: Visual saliency cropping and a mutual learning mechanism between the teacher and student models are introduced. Compared to the traditional approach of treating randomly cropped images of the same image as positive classes, this innovatively uses the visual saliency region map of the training image as the positive class, while treating other random croppings of the training image as intra-class negative classes. This ensures the consistency of the local manifold without causing it to collapse. To distinguish the overall manifold, all other image croppings are marked as inter-class negative classes. A teacher-student cropping mechanism is also designed, where the teacher model performs visual saliency cropping, providing positive samples for comparative learning to the student model. The student model then uses the EMA algorithm to update the teacher model. This dual learning improves the accuracy and stability of visual saliency cropping, thereby enhancing the accuracy of image quality assessment.

[0022] Please refer to Figures 1 to 4 As shown, a preferred embodiment of the referenceless image quality assessment method based on local manifold learning of the present invention includes the following steps: Step S10: Create a teacher model and a student model for image quality assessment, and initialize the teacher model and the student model; both the teacher model and the student model are models based on knowledge distillation technology. Step S20: Obtain training images and crop the training images to obtain training sub-images; Step S30: Input each of the training images into the teacher model to obtain an attention map, and obtain a visual salient region map of each training image based on the attention map; Step S40: Input each of the training sub-images and the visual salient region map into the student model to obtain sub-image features and salient region features; Step S50: Perform local manifold learning on the sub-image features and salient region features and calculate the contrast loss to obtain the contrast loss value; input the training sub-image into the student model to calculate the label loss to obtain the label loss value; Step S60: Calculate the total loss value based on the contrast loss value and the label loss value, update the student model based on the total loss value, and update the teacher model based on the EMA algorithm (exponential moving average algorithm) until the image quality assessment accuracy of the student model is greater than the preset accuracy threshold. Step S70: Perform image quality assessment using the student model. This involves inputting the image to be assessed into the student model to obtain a predicted quality score.

[0023] The core idea of ​​contrastive learning is to reduce the distance within positive classes while increasing the distance between negative classes. However, the preservation of local manifolds has been neglected in related research, which is crucial because hard samples often exhibit high similarity in feature distances, leading to disastrous consequences. To address this, this invention introduces a novel local manifold learning mechanism within the contrastive learning framework of No-Reference Image Quality Assessment (NR-IQA). Specifically, visual saliency is used to represent the entire image, as this region is the most attention-grabbing, and then the salient region is positive for all other random cropping within the image. To distinguish the quality of different images, all other image croppings are further defined as negative for the salient region, and then combined with the positive class, ensuring that random cropping within the image is sufficiently close to maintain consistent representativeness. Furthermore, a novel teacher-student pruning mechanism is introduced, where the EMA algorithm facilitates mutual learning between the teacher and student models. This dual learning paradigm further improves the accuracy of salient pruning and increases sensitivity to quality perception.

[0024] In step S10, the encoders of the teacher model and the student model use a pre-trained Transformer model, and the decoders use a Transformer model decoder.

[0025] Step S20 specifically involves: Acquire training images For the training images Ten random cropping operations of size 224*224 were performed to obtain training sub-images. ;in, , h represents the cropping number, n represents the training image number, and N represents the total number of training images.

[0026] Unlike classifying two randomly cropped patches in the same image directly under the same quality perception, this invention performs a total of 11 cropping operations on the training image: 10 random croppings and 1 visually salient cropping. Specifically, the teacher model is used to identify the region with the highest attention weight in the training image, which is designated as the visually salient region map. In contrastive learning, the visually salient region map is used as the positive class, while other croppings from the same training image represent intra-class negative classes, and croppings from different images constitute inter-class negative classes.

[0027] Step S30 specifically includes: Step S31: Transfer each of the training sub-images Cut into T image blocks, and then cut each image block into T image blocks. piece Inputting the projection layer of the teacher model yields T image patch tokens, and then adding an additional class token yields T+1 tokens; Each of the aforementioned tokens is input into two different transmission layers of the teacher model to obtain Q and K; Calculate the global attention matrix based on Q and K: ; Where Q represents the Query matrix for attention calculation; K represents the Key matrix for attention calculation; and t represents the transpose. , R represents a real number; D represents the feature dimension. Represents the global attention matrix; This represents the normalized exponential function; Step S32: Based on the global attention matrix Calculate the attention weight matrix of class tokens to image patch tokens: ; in, This represents the attention weight matrix of the class token to the image patch token, i.e., the degree of attention. ; In this context, "0" represents the class token index, and "1:T+1" represents the image block token index; The shape of the attention weight matrix of the aforementioned token class for image patch tokens is transformed from one dimension to two dimensions, i.e. Change to ; The attention weight matrix of class tokens for image patch tokens is extracted from the last O layers of the encoder of the teacher model, and its average value is calculated to obtain the average attention level. ; in, Indicates the average level of attention; The teacher model encoder is represented by the first... The attention weight matrix of the class tokens of the layer network for the image patch tokens; Indicates the first layer; Find the average level of attention. The region with the largest sum of values ​​and a size of M*M is used as the training image. Visual salient region map: ; in, q represents the visual saliency region map; p represents the starting row coordinate of the top left corner of the visual saliency region map; q represents the starting column coordinate of the top left corner of the visual saliency region map; i represents the row increment of the visual saliency region map; j represents the column increment of the visual saliency region map.

[0028] In step S40, the calculation formulas for the sub-image features and salient region features are as follows: ; ; in, Representing sub-image features; Indicates the features of visually salient regions; The encoder representing the student model; Represents the training sub-image; Represents the visually salient region map; || is the L1 norm. In step S50, the calculation process for the contrast loss value is as follows: Based on the sub-image features Calculate the negative class within the class: ; Based on the sub-image features Compute the negative class between classes: ; The salient region features As a positive class, the comparative loss value is calculated by combining the negative classes within the class and the negative classes between the classes: ; in, Indicates a negative class within a class; Indicates a negative class between classes; This represents the contrast loss value; H represents the number of times an image is cropped. Indicates the cropping index of the image; This represents the nth image. Image features of a cropped image; The temperature hyperparameter represents the contrast loss; n represents the image index; Indicates the image index; Indicates the cropping index of the image; Represents the nth image. A cropped image feature.

[0029] In step S50, the calculation process for the tag loss value is as follows: The training sub-image The student model is input, and its encoder, decoder, and multilayer perceptron process are used to obtain the predicted score. ; The label loss value of the training sub-image is calculated based on the predicted score: ; in, Indicates the predicted score; Indicates the label loss value; This represents a multilayer perceptron; The decoder representing the student model; The encoder represents the student model; H represents the number of times an image is cropped; h represents the cropping number. This represents the prediction score for the h-th crop of the n-th image; This represents L1 loss.

[0030] In step S60, the formula for calculating the total loss value is: ; in, This represents the total loss value; Indicates the label loss value; Indicates the comparative loss value; represents the balance factor; N represents the total number of training images; n represents the training image number.

[0031] In step S60, updating the teacher model based on the EMA algorithm specifically involves: ; in, This represents the updated teacher model parameters; This represents the teacher model parameters before the update; Indicates the parameters of the student model; This represents a balancing factor. In the early training phase, the teacher model prioritizes salient objects, then shifts its focus to regions that significantly affect the perception of overall image quality.

[0032] Table 1 compares the performance of SRCC and PLCC average values ​​in the synthetic database:

[0033] As can be seen from Table 1, the performance of this invention on the synthetic database surpasses that of all other inventions. This demonstrates the effectiveness of the local manifold learning-based approach proposed in this invention.

[0034] Table 2 shows the performance comparison of SRCC and PLCC averages in a real database:

[0035] As shown in Table 2, this invention outperformed all other inventions in three out of four real-world databases, and also demonstrated good performance on LiveFB. This further proves the effectiveness and versatility of this invention.

[0036] Figure 2(a) is the traditional IQA method based on contrastive learning: it makes the cropping of the same image converge and cannot preserve the local manifold; (b) is the present invention: it preserves the local manifold and maintains the diversity of the feature space.

[0037] Figure 3 This is a framework diagram of the present invention; the input training image is randomly cropped 10 times and fed into the teacher model to crop the visual saliency region of the training image. All cropped images and visual saliency region maps are fed into the student model to calculate the contrast loss and label loss respectively. Then the student model is updated, and finally the teacher model is updated using the EMA algorithm.

[0038] Figure 4 As can be seen, the model of this invention pays more attention to image distortion-related features, and the predicted image quality score is closer to the true value; the number under each row of images represents the model's predicted value, and the number in parentheses represents the distance from the true value.

[0039] In summary, the advantages of this invention are as follows: The process begins by initializing the created teacher and student models. Next, training images are acquired, cropped to obtain sub-images, and input into the teacher model to generate attention maps. Based on these attention maps, visual salient region maps are obtained for each training image. These sub-images and their corresponding visual salient region maps are then input into the student model to obtain sub-image features and salient region features. Local manifold learning is then performed on these sub-image and salient region features, and contrastive loss is calculated to obtain contrastive loss values. The training sub-images are then input into the student model to calculate label loss values. Finally, the total loss value is calculated based on the contrastive and label loss values. The student model is updated based on the total loss value, and the teacher model is updated using the EMA algorithm until the image quality assessment accuracy of the student model exceeds a preset accuracy threshold. Finally, the student model is used to perform image quality assessment. To mitigate the local manifold collapse problem, quality assessment employs visual saliency cropping and a teacher-student model mutual learning mechanism. Compared to the traditional approach of treating randomly cropped images of the same image as positive classes, this innovative approach uses visually salient regions of the training image as positive classes, while treating other random cropped images of the training image as intra-class negative classes. This ensures the consistency of the local manifold without causing collapse. Furthermore, to distinguish the overall manifold, all other image croppings are labeled as inter-class negative classes. A teacher-student cropping mechanism is also designed, where the teacher model performs visual saliency cropping, providing positive samples for comparative learning to the student model. The student model then updates the teacher model using the EMA algorithm. This dual learning improves the accuracy and stability of visual saliency cropping, aids subsequent comparative learning, and ultimately greatly enhances the accuracy of image quality assessment.

[0040] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A no-reference image quality assessment method based on local manifold learning, characterized in that: Includes the following steps: Step S10: Create a teacher model and a student model for image quality assessment, and initialize the teacher model and the student model; Step S20: Obtain training images and crop the training images to obtain training sub-images; Step S30: Input each of the training images into the teacher model to obtain an attention map, and obtain a visual salient region map of each training image based on the attention map; Step S40: Input each of the training sub-images and the visual salient region map into the student model to obtain sub-image features and salient region features; Step S50: Perform local manifold learning on the sub-image features and salient region features and calculate the contrast loss to obtain the contrast loss value; input the training sub-image into the student model to calculate the label loss to obtain the label loss value; Step S60: Calculate the total loss value based on the contrast loss value and the label loss value, update the student model based on the total loss value, and update the teacher model based on the EMA algorithm until the image quality assessment accuracy of the student model is greater than the preset accuracy threshold. Step S70: Use the student model to perform image quality assessment; In step S40, the calculation formulas for the sub-image features and salient region features are as follows: ; ; in, Representing sub-image features; Indicates the characteristics of visually salient regions; The encoder representing the student model; Represents the training sub-image; Represents the visually salient region map; || is the L1 norm; In step S50, the calculation process of the contrast loss value is as follows: Based on the sub-image features Calculate the negative class within the class: ; Based on the sub-image features Compute the negative class between classes: ; The salient region features As a positive class, the comparative loss value is calculated by combining the negative classes within the class and the negative classes between the classes: ; in, Indicates a negative class within a class; Indicates a negative class between classes; This represents the contrast loss value; H represents the number of times an image is cropped. Indicates the cropping index of the image; This represents the nth image. Image features of a cropped image; The temperature hyperparameter representing the contrast loss; Indicates the image index; Indicates the cropping index of the image; Represents the nth image. There are 1 cropped image features, where N represents the total number of training images.

2. The no-reference image quality assessment method based on local manifold learning as described in claim 1, characterized in that: In step S10, the encoders of the teacher model and the student model use a pre-trained Transformer model, and the decoders use a Transformer model decoder.

3. The no-reference image quality assessment method based on local manifold learning as described in claim 1, characterized in that: Step S20 specifically involves: Acquire training images For the training images Ten random cropping operations of size 224*224 were performed to obtain training sub-images. ;in, , h represents the cropping number, n represents the training image number, and N represents the total number of training images.

4. The no-reference image quality assessment method based on local manifold learning as described in claim 1, characterized in that: Step S30 specifically includes: Step S31: Transfer each of the training sub-images Cut into T image blocks, and then cut each image block into T image blocks. piece Inputting the projection layer of the teacher model yields T image patch tokens, and then adding an additional class token yields T+1 tokens; Each of the aforementioned tokens is input into two different transmission layers of the teacher model to obtain Q and K; Calculate the global attention matrix based on Q and K: ; Where Q represents the Query matrix for attention calculation; K represents the Key matrix for attention calculation; and t represents the transpose. , R represents a real number; D represents the feature dimension. Represents the global attention matrix; This represents the normalized exponential function; Step S32: Based on the global attention matrix Calculate the attention weight matrix of class tokens to image patch tokens: ; in, This represents the attention weight matrix of the class token to the image patch token, i.e., the degree of attention. ; In the text, "0" represents the class token index, and "1:T+1" represents the image patch token index; The shape of the attention weight matrix of the aforementioned token class for image patch tokens is transformed from one dimension to two dimensions, i.e. Change to ; The attention weight matrix of class tokens for image patch tokens is extracted from the last O layers of the encoder of the teacher model, and its average value is calculated to obtain the average attention level. ; in, Indicates the average level of attention; The teacher model encoder is represented by the first... The attention weight matrix of the class tokens of the layer network for the image patch tokens; Indicates the first layer; Find the average level of attention. The region with the largest sum of values ​​and a size of M*M is used as the training image. Visual salient region map: ; in, q represents the visual saliency region map; p represents the starting row coordinate of the top left corner of the visual saliency region map; q represents the starting column coordinate of the top left corner of the visual saliency region map; i represents the row increment of the visual saliency region map; j represents the column increment of the visual saliency region map.

5. The no-reference image quality assessment method based on local manifold learning as described in claim 1, characterized in that: In step S50, the calculation process for the tag loss value is as follows: The training sub-image The student model is input, and its encoder, decoder, and multilayer perceptron process are used to obtain the predicted score. ; The label loss value of the training sub-image is calculated based on the predicted score: ; in, Indicates the predicted score; Indicates the label loss value; This represents a multilayer perceptron; The decoder representing the student model; The encoder represents the student model; H represents the number of times an image is cropped; h represents the cropping number. This represents the prediction score for the h-th crop of the n-th image; This represents L1 loss.

6. The no-reference image quality assessment method based on local manifold learning as described in claim 1, characterized in that: In step S60, the formula for calculating the total loss value is: ; in, This represents the total loss value; Indicates the label loss value; Indicates the comparative loss value; represents the balance factor; N represents the total number of training images; n represents the training image number.

7. The no-reference image quality assessment method based on local manifold learning as described in claim 1, characterized in that: In step S60, updating the teacher model based on the EMA algorithm specifically involves: ; in, This represents the updated teacher model parameters; This represents the teacher model parameters before the update; Indicates the parameters of the student model; This represents the balance factor.