A no-reference image quality assessment method based on information transfer VIT
By introducing the information transfer module into the VIT network, shallow features are integrated into deep features layer by layer, which solves the problem of global features ignoring local features in the existing image quality evaluation methods and achieves more consistent image quality prediction.
Patent Information
- Application Number
- CN202411760781.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-03
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2044-12-03
AI Technical Summary
Existing VIT-based image quality evaluation methods ignore the interaction between shallow image information and global information, resulting in inconsistency between model prediction quality and human visual perception quality.
By constructing an information transfer VIT model, the information transfer module is used to fuse shallow features into deep features layer by layer in the VIT network to enhance feature diversity, and the image quality score is predicted through the quality regression network.
The effective interactive fusion of global and local features of the image is achieved, which improves the consistency between the model prediction quality and the human visual perception quality.
Smart Images

Figure CN119672001B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image quality evaluation, and in particular relates to a reference-free image quality evaluation method based on information transfer (VIT). Background Art
[0002] In image quality assessment research, using the Vision Transformer (VIT) to extract global image features for quality evaluation is an increasingly mature approach. Its self-attention mechanism can model extensive dependencies between tokens in a data sequence, overcoming the limitations of convolutional neural networks in capturing global context. The VIT model is characterized by its deep architecture, comprising multiple modular layers consisting of self-attention layers, multi-layer perceptrons, and residual connections. However, in the standard VIT model, the generated attention map becomes increasingly global as image information passes deeper through the network. However, aggregating features across patches at a global scale causes the network to forget low-level features and emphasize only global features, resulting in a loss of feature diversity.
[0003] Currently, image quality assessment methods based on the VIT self-attention mechanism primarily focus on improving their ability to aggregate patches at a global scale. Local features are often extracted through convolutional neural networks, or even ignored. However, the human visual system does not simply focus on global image features; rather, it analyzes and understands them, combining them with information such as color and shape to make judgments. In this process, global features are not the direct determining factor; shallower features can also capture information related to image quality. Global features, as an important factor, interact with shallower information to jointly capture information related to image quality. In summary, due to a certain degree of neglect of shallow image information, its interaction with global image information and its impact on overall image quality are not fully considered. This leads to a lack of consistency between model prediction quality and human visual perception, and existing methods do not conform to the perceptual characteristics of the human visual system.
[0004] The patent application with publication number CN118469930A, titled "Image quality evaluation method based on multi-scale regional self-attention fusion under meta-learning framework", ignores the integration of global information and local information in the VIT architecture and does not take into account the characteristics of human visual perception, resulting in inconsistency between model prediction quality and human visual perception quality.
[0005] The patent application with publication number CN117593296A, titled "A reference-free image quality evaluation method based on diffusion model", does not consider the integration of shallow visual information and deep visual information of the image in the Transformer model, and does not take into account the characteristics of human visual perception, resulting in inconsistency between the model prediction quality and human visual perception quality.
[0006] Most existing image quality evaluation methods that use the self-attention mechanism (VIT) model ignore the impact of shallow image information in the image on image quality to a certain extent, or simply extract shallow visual information through convolutional neural networks and then simply fuse it with global characteristics, which does not conform to the characteristics of human visual perception and leads to inconsistency between model prediction quality and human visual perception quality. Summary of the Invention
[0007] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a reference-free image quality assessment method based on information transfer (VIT). By transferring and fusing shallow image information to deep image features, the feature diversity of deeper networks is enhanced, thereby completing the interaction and integration of image feature information between layers, and predicting the image quality score on this basis, which is used to solve the problem of inconsistency between model prediction quality and human visual perception quality caused by the self-attention mechanism being too deep, the gradual disappearance of low-level image information and the loss of feature diversity in the prior art.
[0008] In order to achieve the above object, the technical solution adopted by the present invention is:
[0009] A no-reference image quality assessment method based on information transfer (VIT) comprises the following steps:
[0010] Step 1: Randomly select images from the public image quality assessment dataset and divide them into training sample set B and test sample set C;
[0011] Step 2: Construct a no-reference image quality assessment network model S based on information transfer (VIT), and use the image as input to predict the image quality score;
[0012] Step 3, using the training sample set B, iteratively training the no-reference image quality assessment model based on information transfer (VIT), and testing the training results using the test sample set C;
[0013] Step 4: Use the trained model to obtain the no-reference quality evaluation results of the image and obtain the image quality score.
[0014] The step 1 is specifically as follows:
[0015] A part of images is randomly selected from the public image quality assessment dataset as the training sample set B, and another part of images is selected as the test sample set C.
[0016] The step 2 is specifically as follows:
[0017] Step 2.1: Construct a standard VIT network, which includes a linear mapping network for processing 2D image patches and an encoder consisting of an attention module and an MLP stack. The linear mapping network for processing 2D image patches obtains a shallow feature sequence of the image, and the shallow feature sequence is passed through the encoder to obtain deep features.
[0018] Step 2.2: Construct the information transfer module P. In the attention module part of step 2.1, directly calculate the original attention score in the original scheme. The improvement is to the original attention score S of each layer starting from the second layer, which is combined with the attention score S of the previous layer through the coefficient α l-1 Performing weighted operations, the information transmission module P enables the current layer to learn the features of the previous layer, that is, to fuse the shallow features with the deep features;
[0019] Step 2.3: Construct a quality regression network Q. The image feature sequence output in step 2.1 is passed through the quality regression network Q to obtain the predicted image quality score.
[0020] The step 2.1 is specifically as follows:
[0021] The linear mapping network for processing 2D image patches transforms the images in the training sample set B through convolution and flattening operations. Output is a sequence And embed the position code to obtain shallow features;
[0022] VIT uses the Transformer encoder as the backbone network and adds a learnable token before embedding the patch sequence Summarizes the information changes of image features; the learnable token is learned independently by each layer of encoder, which is equivalent to the abstract sum of the image sequence of this layer;
[0023] The shallow feature sequence is the feature vector obtained by the linear mapping network;
[0024] First, the attention module is calculated and three vectors Q, K, and V are generated through a learnable linear projection layer. Where D is the projection dimension and N is the area of the image. Then, for a given attention layer Calculate its raw attention score Then the attention weight matrix is obtained through the Softmax function The scaled attention is then calculated by weighted multiplication. The implementation process of the attention module is as follows:
[0025]
[0026]
[0027]
[0028] is the scaled attention, d is the projection dimension, are three vectors generated by a learnable linear projection layer;
[0029] The input image feature sequence passes through a multi-layer stack of attention modules and MLP to obtain the final output sequence y;
[0030] will output With input After addition, the final output sequence y is obtained through MLP;
[0031] The overall implementation process is as follows:
[0032]
[0033] Where MHSA, LN, and MLP represent multi-head attention, layer normalization, and multi-layer perceptron, respectively. L represents the number of stacked layers of the multi-head attention layer. E pos is the position code, z0 is the input image feature sequence, y is the output image feature sequence, Z l ' is the image feature sequence after the attention mechanism of the lth layer, Z l is the image feature sequence output by the lth layer. The deep feature is the shallow feature vector after learning, and its essence is still a vector.
[0034] The implementation process of step 2.2 is as follows:
[0035]
[0036] S l is the attention score of layer l, d is the projection dimension, Q, K, V are three vectors generated by the learnable linear projection layer, and α is the weight coefficient;
[0037] The attention score of the previous layer is added to the attention score of the current layer through the weight coefficient for fusion;
[0038] The attention score is equivalent to the weight of the features in this layer. Adding the attention score of the shallow layer features to the deep layer is equivalent to adding the attention score of the shallow layer features to the weight of the deep layer features. The specific steps of step 2.3 are:
[0039] Returning to the final image quality score, the implementation process is as follows:
[0040] q=MLP(y)
[0041] Among them, q represents the final prediction quality score, MLP represents the multi-layer perceptron, and y represents the final output image feature sequence.
[0042] The step 3 is specifically as follows:
[0043] Step 3.1, initialize the number of iterations to t, the maximum number of iterations to T, and the current no-reference image quality assessment network model based on information transfer VIT is S t , and let t = 1, S t =S;
[0044] Step 3.2: randomly select b training samples from the training sample set B without replacement as the no-reference image quality evaluation network model S t As the number of network layers increases, the standard VIT network extracts deep features and shallow features of the image and fuses them. The shallow feature information of the image and the feature information of the current layer are transmitted layer by layer. The quality regression network regresses to obtain the quality prediction score corresponding to each training sample.
[0045] Step 3.3: First, calculate the mean square error between the quality prediction score corresponding to each training sample and the quality score label corresponding to the training sample as the model S t The loss value of the network model S is then optimized using the adaptive moment estimation algorithm. t Update the weight parameters of each layer;
[0046] The formula for calculating the mean square error between the quality prediction score corresponding to each training sample and the quality score label corresponding to the training sample is:
[0047]
[0048] Where M represents the total number of images in each batch of training data, q j and They represent the objective quality score and subjective rating label predicted by the model corresponding to the j-th image in the batch.
[0049] The adaptive moment estimation is used as the optimization algorithm, and the initial learning rate lr is set to 10 -4 , and adjust the learning rate every s generations during training to speed up convergence and improve learning accuracy. The adjustment formula is as follows:
[0050]
[0051] Where t represents the current iteration number, s represents the learning rate update step size, d represents the learning rate attenuation coefficient, lr tIndicates the learning rate when the model is iterated to t times. In addition, to alleviate the overfitting phenomenon of noise and abnormal samples in the training data during model training, weight decay weight_decay is added during training, which is generally set to 0.0001.
[0052] Step 3.4: Determine whether the training sample set B has been traversed. If so, execute step 3.5; otherwise, execute steps 3.2 and 3.3.
[0053] Step 3.5, determine whether t=T is established. If so, obtain the trained information transfer VIT-based no-reference image quality assessment network model S * =S t Otherwise, set t = t + 1 and execute steps 3.3 and 3.4;
[0054] The step 4 is specifically as follows:
[0055] The test sample set C is used as the trained no-reference image quality assessment network model S based on information transfer VIT * The input is used for forward reasoning to obtain the quality prediction score of each test sample to verify the model effect.
[0056] Beneficial effects of the present invention:
[0057] The present invention uses the information transfer module in step 2.2 to fuse the image features of the previous layer with the image features of the current layer through weighting of the attention score S. In a multi-layer encoder, this weighting allows the initial shallow-layer feature information to be integrated layer by layer into the features of the next layer. This method of gradually transferring the VIT shallow-layer network feature information to the deep-layer network feature information achieves the interactive fusion of the global image features and local features, and on this basis, realizes the reference-free quality evaluation of the image and generates the predicted image quality score.
[0058] Since the effective interactive fusion of global and local features of the image in the VIT architecture is fully considered, the shortcomings of the traditional VIT model that focuses on global features but ignores local features are improved. The present invention has the advantage of high consistency between model prediction quality and human visual perception quality. BRIEF DESCRIPTION OF THE DRAWINGS
[0059] Figure 1 It is a schematic flow chart of the present invention.
[0060] Figure 2 Schematic diagram of the information transmission module of the present invention. DETAILED DESCRIPTION
[0061] The present invention will be described in further detail below with reference to the accompanying drawings.
[0062] Refer to the attached Figure 1 , the present invention comprises the following steps:
[0063] Step 1: Get the training sample set B and the test sample set C:
[0064] Randomly select 80% of all images from the public image quality assessment dataset as training sample set B, and the remaining 20% of images as test sample set C;
[0065] In this embodiment, the KonIQ-10k dataset is used, 80% of the reference images are used as the training set, and the remaining 20% of the reference images are used as the test set. This operation ensures that there are no image data with the same image scene and content in the training set and the test set;
[0066] Step 2: Construct a no-reference image quality assessment network model S based on information transfer VIT:
[0067] Step 2.1: Build a standard VIT network consisting of a linear map that processes 2D image patches and an encoder consisting of an attention module and an MLP stack.
[0068] The linear mapping network for processing 2D image patches converts the image into Output is a sequence And embed position coding; obtain the shallow feature sequence of the image through the linear mapping network, and add position coding to make the model know the relative position of the front and back of the image sequence;
[0069] The encoder composed of the attention module and MLP stack uses the Transformer encoder as the backbone network and adds a learnable token before embedding the patch sequence Summarize the changes in the visual information of the image.
[0070] First, the attention module is calculated and three vectors Q, K, and V are generated through a learnable linear projection layer. Where D is the projection dimension. Then, for a given attention layer Calculate its raw attention score Then the attention weight matrix is obtained through the Softmax function The scaled attention is then calculated by weighted multiplication. The implementation process of the attention module is as follows:
[0071]
[0072] After passing through the attention module, the output With input After addition, the final output sequence y is obtained through MLP. The output sequence of the image is obtained through the standard VIT network as the input of the subsequent quality regression network.
[0073] The overall implementation process is as follows:
[0074]
[0075] Where MHSA, LN, and MLP represent multi-head attention, layer normalization, and multi-layer perceptron, respectively. L represents the number of stacked layers of the multi-head attention layer. In this paper, L is set to 12, and E pos is the position code, z0 is the input image feature sequence, and y is the output image feature sequence.
[0076] Step 2.2, such as Figure 2 As shown, we construct the information transfer module P. First, in the attention module in step 2.1, we improve the original scheme by directly calculating the raw attention score. Instead, we weight the raw attention score S of each layer starting from the second layer by the coefficient α and the attention score S of the previous layer. This module is added to the attention module of the standard VIT network to transfer shallow image features to deeper layers.
[0077] In the present invention, α is set to 0.6. The implementation process is as follows:
[0078]
[0079] In step 2.3, we construct a quality regression network Q and regress the image quality score to obtain the final prediction score.
[0080] The implementation process is as follows:
[0081] q=MLP(y)
[0082] Among them, q represents the final prediction quality score, MLP represents the multi-layer perceptron, and y represents the final output image feature sequence.
[0083] Step 3: Iteratively train the no-reference image quality assessment model based on information transfer (VIT):
[0084] Step 3.1, initialize the number of iterations to t, the maximum number of iterations to T, and the current no-reference image quality assessment network model based on information transfer VIT is S t , and let t = 1, S t =S;
[0085] In this embodiment, T=100;
[0086] Step 3.2: randomly select b training samples from the training sample set B without replacement as the no-reference image quality evaluation network model S tAs the number of network layers increases, the standard VIT network extracts global features and local features of the image and fuses them. It transfers the shallow feature information of the image and the feature information of the current layer layer by layer, and the quality regression network regresses to obtain the quality prediction score corresponding to each training sample.
[0087] In this embodiment, b=4;
[0088] Step 3.3: First, calculate the mean square error between the quality prediction score corresponding to each training sample and the quality score label corresponding to the training sample as the model S t The loss value of the network model S is then optimized using the adaptive moment estimation algorithm. t Update the weight parameters of each layer;
[0089] In this embodiment, the formula for calculating the mean square error between the quality prediction score corresponding to each training sample and the quality score label corresponding to the training sample is:
[0090]
[0091] Where M represents the total number of images in each batch of training data, q j and They represent the objective quality score and subjective rating label predicted by the model corresponding to the j-th image in the batch.
[0092] In this embodiment, adaptive moment estimation is used as the optimization algorithm, and the initial learning rate lr is set to 10 -4 , and adjust the learning rate every s generations during training to speed up convergence and improve learning accuracy. The adjustment formula is as follows:
[0093]
[0094] Where t represents the current iteration number, s represents the learning rate update step size, d represents the learning rate attenuation coefficient, lr t Indicates the learning rate when the model is iterated to t times. In addition, to alleviate the overfitting phenomenon of noise and abnormal samples in the training data during model training, weight decay weight_decay is added during the training process.
[0095] In this embodiment, s=10, d=0.9, weight_decay=10 -4 .
[0096] Step 3.4: Determine whether the training sample set B has been traversed. If so, execute step 3.5; otherwise, execute steps 3.2 and 3.3.
[0097] Step 3.5, determine whether t=T is established. If so, obtain the trained information transfer VIT-based no-reference image quality assessment network model S * =S t Otherwise, set t = t + 1 and execute steps 3.3 and 3.4;
[0098] Step 4: Get the image's no-reference quality evaluation results:
[0099] The test sample set C is used as the trained no-reference image quality assessment network model S based on information transfer VIT * The input is used for forward reasoning to obtain the quality prediction score of each test sample to verify the model effect.
[0100] The following is a description of the technical effects of the present invention in conjunction with simulation experiments:
[0101] 1. Simulation conditions and content:
[0102] The hardware platform of the simulation experiment of the present invention is as follows: the processor is Intel(R) Core(TM) i9-7900X CPU with a main frequency of 3.30GH, the memory is 32GB, and the graphics card is NVIDIA GeForce GTX 1080Ti.
[0103] The software platform for the simulation experiment of the present invention is: Ubuntu 16.04 operating system, Pytorch 1.6.0 framework, and Python 3.7.
[0104] 2. Simulation Experiment
[0105] The input images used in the simulation experiments of the present invention are derived from the image quality assessment database KonIQ-10k. The KonIQ-10k dataset is an image quality assessment database proposed by Hosu et al. in “KonIQ-10k: An Ecologically Valid Database for DeepLearning of Blind Image Quality Assessment. IEEE Transactions on ImageProcessing, Jan. 2020, pp. 4041-56.”
[0106] The simulation experiment of the present invention uses two indicators, the Spearman rank-order correlation coefficient SROCC (Spearman rank-order correlation coefficient) and the Pearson linear correlation coefficient PLCC (Pearson linear correlation coefficient), to evaluate the image quality evaluation effects of the present invention and the prior art, respectively. Specifically, the prior art and the present invention respectively select n samples from the test sample set C for image quality evaluation, output a quality prediction score, and calculate the values of SROCC and PLCC based on the quality prediction score of the sample and the quality label score corresponding to the sample. Among them, the two indicators are calculated according to the following formulas: SROCC∈[-1,1], the higher the value, the more accurately the evaluation result of the evaluated no-reference image quality evaluation method can reflect the quality of the image; PLCC∈[-1,1], the higher the value, the closer the evaluation result of the evaluated no-reference image quality evaluation method is to the subjective evaluation score of humans.
[0107]
[0108] Where n represents the total number of images, b e The difference between the order of the quality prediction score of the e-th distorted image in the quality prediction score ranking of n distorted images and the order of its quality label score, s r represents the quality prediction score of the rth distorted image, represents the average quality prediction score of all distorted images, p r represents the subjective evaluation score of the rth distorted image, It represents the average of the subjective evaluation scores of all distorted images.
[0109] The simulation results are shown in Table 1.
[0110] Table 1 Comparison of evaluation results of the present invention and prior art
[0111]
[0112] As shown in Table 1, compared to existing technologies, the present invention achieves higher Spearman rank correlation coefficients (SROCC) and Pearson linear correlation coefficients (PLCC) for evaluation results on the KonIQ-10k dataset, demonstrating the improved accuracy of the present invention's no-reference image quality assessment method. By gradually transferring VIT shallow network feature information to deep network feature information, the present invention achieves interactive fusion of global and local image features. Based on this, no-reference image quality assessment is implemented, generating a predicted image quality score.
Claims
1. A no-reference image quality assessment method based on information transfer (VIT), characterized in that: The following steps are included: Step 1: Randomly select images from the public image quality assessment dataset and divide them into training sample set B and test sample set C; Step 2: Construct a no-reference image quality assessment network model S based on information transfer (VIT), and use the image as input to predict the image quality score; Step 3, using the training sample set B, iteratively training the no-reference image quality assessment model S based on information transfer (VIT), and testing the training results using the test sample set C; Step 4: Use the trained model to obtain the image quality score. The step 2 is specifically as follows: Step 2.1: Construct a standard VIT network, which includes a linear mapping network for processing 2D image patches and an encoder consisting of an attention module and an MLP stack. The linear mapping network for processing 2D image patches obtains a shallow feature sequence of the image, and the shallow feature sequence is passed through the encoder to obtain deep features. Step 2.2: Construct the information transfer module P. In the attention module part of step 2.1, directly calculate the original attention score in the original scheme. The improvement is to the original attention score S of each layer starting from the second layer, which is combined with the attention score S of the previous layer through the coefficient α l-1 Performing weighted operations, the information transmission module P enables the current layer to learn the features of the previous layer, that is, to fuse the shallow features with the deep features; Step 2.3: Construct a quality regression network Q. The image feature sequence output in step 2.1 is passed through the quality regression network Q to obtain the predicted image quality score.
2. The method for no-reference image quality assessment based on information transfer (VIT) according to claim 1, characterized in that: The step 1 is specifically as follows: A part of images is randomly selected from the public image quality assessment dataset as the training sample set B, and another part of images is selected as the test sample set C.
3. The method for no-reference image quality assessment based on information transfer (VIT) according to claim 1, characterized in that: The step 2.1 is specifically as follows: The linear mapping network for processing 2D image patches transforms the images in the training sample set B through convolution and flattening operations. Output is a sequence And embed the position code to obtain shallow features; VIT uses the Transformer encoder as the backbone network and adds a learnable token before embedding the patch sequence Summarizes the information changes of image features; the learnable token is learned independently by each layer of encoder, which is equivalent to the abstract sum of the image sequence of this layer; First, the attention module is calculated and three vectors Q, K, and V are generated through a learnable linear projection layer. Where D is the projection dimension and N is the area of the image. Then, for a given attention layer Calculate its raw attention score Then the attention weight matrix is obtained through the Softmax function The scaled attention is then calculated by weighted multiplication. The implementation process of the attention module is as follows: Z t is the scaled attention, d is the projection dimension, Q t , K t 、V t are three vectors generated by a learnable linear projection layer; The input image feature sequence passes through a multi-layer stack of attention modules and MLP to obtain the final output sequence y; The overall implementation process is as follows: Where MHSA, LN, and MLP represent multi-head attention, layer normalization, and multi-layer perceptron, respectively. L represents the number of stacked layers of the multi-head attention layer. E pos is the position code, z0 is the input image feature sequence, y is the output image feature sequence, Z l ' is the image feature sequence after the attention mechanism of the lth layer, Z l is the image feature sequence output by the lth layer.
4. The method for no-reference image quality assessment based on information transfer (VIT) according to claim 1, characterized in that: The implementation process of step 2.2 is as follows: S l is the attention score of layer l, d is the projection dimension, Q, K, V are three vectors generated by the learnable linear projection layer, and α is the weight coefficient; The attention score of the previous layer is added to the attention score of the current layer through the weight coefficient for fusion.
5. The method for no-reference image quality assessment based on information transfer (VIT) according to claim 1, characterized in that: The step 2.3 is specifically as follows: Returning to the final image quality score, the implementation process is as follows: q=MLP(y) Among them, q represents the final prediction quality score, MLP represents the multi-layer perceptron, and y represents the final output image feature sequence.
6. The method for no-reference image quality assessment based on information transfer (VIT) according to claim 1, characterized in that: The step 3 is specifically as follows: Step 3.1, initialize the number of iterations to t, the maximum number of iterations to T, and the current no-reference image quality assessment network model based on information transfer VIT is S t , and let t = 1, S t =S; Step 3.2: randomly select b training samples from the training sample set B without replacement as the no-reference image quality evaluation network model S t As the number of network layers increases, the standard VIT network extracts deep features and shallow features of the image and fuses them. The shallow feature information of the image and the feature information of the current layer are transmitted layer by layer. The quality regression network regresses to obtain the quality prediction score corresponding to each training sample. Step 3.3: First, calculate the mean square error between the quality prediction score corresponding to each training sample and the quality score label corresponding to the training sample as the model S t The loss value of the network model S is then optimized using the adaptive moment estimation algorithm. t Update the weight parameters of each layer; The formula for calculating the mean square error between the quality prediction score corresponding to each training sample and the quality score label corresponding to the training sample is: Where M represents the total number of images in each batch of training data, q j and They represent the objective quality score and subjective rating label predicted by the model corresponding to the j-th image in the batch; Step 3.4: Determine whether the training sample set B has been traversed. If so, execute step 3.5; otherwise, execute steps 3.2 and 3.
3. Step 3.5, determine whether t=T is established. If so, obtain the trained information transfer VIT-based no-reference image quality assessment network model S * =S t , otherwise, let t = t + 1 and execute steps 3.3 and 3.
4.
7. The method for no-reference image quality assessment based on information transfer (VIT) according to claim 6, characterized in that: The adaptive moment estimation is used as the optimization algorithm, and the initial learning rate lr is set to 10 -4 , and adjust the learning rate every s generations during training to speed up convergence and improve learning accuracy. The adjustment formula is as follows: Where t represents the current iteration number, s represents the learning rate update step size, d represents the learning rate attenuation coefficient, lr t Indicates the learning rate when the model is iterated to t times. In addition, in order to alleviate the overfitting phenomenon of noise and abnormal samples in the training data during model training, weight decay weight_decay is added during the training process.
8. The method for no-reference image quality assessment based on information transfer (VIT) according to claim 1, characterized in that: The step 4 is specifically as follows: The test sample set C is used as the trained no-reference image quality assessment network model S based on information transfer VIT * The input is used for forward reasoning to obtain the quality prediction score of each test sample to verify the model effect.
Citation Information
Patent Citations
Non-reference image quality evaluation method based on diffusion model
CN117593296A
No-reference image quality evaluation method based on distortion sensitivity
CN118429308A
Image quality evaluation method based on multi-scale region self-attention fusion under meta-learning framework
CN118469930A