Image quality assessment method based on high frequency extraction and knowledge distillation

Abstract

This invention discloses an image quality assessment method based on high-frequency extraction and knowledge distillation. The method calculates the total loss function of the image quality assessment model based on the degraded image features extracted by the trained student degraded image feature extraction module, the original quality image features extracted by the teacher degraded image feature extraction module, the student high-frequency features extracted by the trained student high-frequency feature extraction module, the teacher high-frequency features extracted by the trained teacher high-frequency feature extraction module, the restored features extracted by the restored feature extraction network, and the first, second, and third quality scores output by the quality prediction module. The proposed image quality assessment model is optimized with the goal of minimizing the total loss function. Training is considered complete when the loss decreases and remains stable, resulting in a trained image quality assessment model. The restored image is input into the trained image quality assessment model, and the quality score of the restored image is output. This invention is beneficial for improving the prediction of perceived image quality.

Description

Technical Field

[0001] This invention relates to an image quality assessment method based on high-frequency extraction and knowledge distillation, belonging to the field of image quality assessment technology. Background Technology

[0002] One important scenario for image quality assessment (IQA) is evaluating the restored images generated by image restoration (IR) algorithms. Current state-of-the-art IQA methods employ a full-reference approach, comparing the restored image with the corresponding original quality image. However, in blind image restoration tasks and real-world scenarios, the original quality image is often unavailable.

[0003] The no-reference image quality assessment method (NR-IQA) is commonly used to assess the quality of images in real-world scenarios. No-reference image quality assessment solutions typically regress the restored image to a quality score. However, the lack of reference information makes the image quality assessment problem more challenging and leads to a sharp decline in performance.

[0004] Therefore, how to improve the evaluation level of the restored image without the original quality image as a reference image has become an urgent problem to be solved in this field. Summary of the Invention

[0005] Objective: To overcome the shortcomings of existing technologies, this invention provides an image quality assessment method based on high-frequency extraction and knowledge distillation. It utilizes octave convolution to extract effective high-frequency information from reference images and introduces a knowledge distillation mechanism to train a feature extraction network, thereby improving the performance of image quality assessment.

[0006] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] An image quality assessment method based on high-frequency extraction and knowledge distillation includes the following steps:

[0008] The total loss function of the image quality assessment model is calculated based on the degraded image features extracted by the trained student degraded image feature extraction module, the original quality image features extracted by the teacher degraded image feature extraction module, the high-frequency student features extracted by the trained student high-frequency feature extraction module, the high-frequency teacher features extracted by the teacher high-frequency feature extraction module, the restoration features extracted by the restoration feature extraction network, and the first, second, and third quality scores output by the quality prediction module.

[0009] The proposed image quality assessment model is optimized with the goal of minimizing the total loss function. Training is considered complete when the loss decreases and remains stable, resulting in a well-trained image quality assessment model.

[0010] The restored image is input into the trained image quality assessment model, which outputs the quality score of the restored image.

[0011] As a preferred embodiment, the image quality assessment model includes: a student degraded feature extraction module, a teacher degraded feature extraction module, a restoration feature extraction network, a student high-frequency feature extraction module, a teacher high-frequency feature extraction module, and a quality prediction module.

[0012] As a preferred embodiment, the method for obtaining the trained student degradation feature extraction module is as follows:

[0013] Input the degraded image and the corresponding original quality image into the student degraded feature extraction module and the teacher degraded feature extraction module respectively to extract the features of the degraded image and the original quality image.

[0014] The restored image corresponding to the degraded image is input into the restoration feature extraction network to extract restoration features.

[0015] The original high-quality image is input into the teacher high-frequency feature extraction module and then passed through three octave convolutional layers to extract the teacher's high-frequency features.

[0016] The extracted degraded image features are subtracted from the restored features to obtain the first difference feature. This first difference feature is then input together with the teacher's high-frequency features into the quality prediction module to predict the quality score, resulting in the first quality score. The extracted original quality image features are then subtracted from the restored features to obtain the second difference feature. This second difference feature is then input together with the teacher's high-frequency features into the quality prediction module to predict the quality score, resulting in the second quality score.

[0017] By employing the knowledge distillation mechanism, the first quality score, the second quality score, and CKD Loss are used to train the student degraded feature extraction module and the teacher degraded feature extraction module, resulting in a well-trained student degraded feature extraction module.

[0018] As a preferred embodiment, the step of inputting the original high-quality image into the teacher high-frequency feature extraction module and then passing it through three octave convolutional layers to extract the teacher's high-frequency features includes:

[0019] The original quality image is input into the teacher's high-frequency feature extraction module to obtain features. The extracted features are divided into two parts, high-frequency features and low-frequency features, according to scale space theory. The two parts are further fed into an octave convolutional layer for feature extraction to obtain the high-frequency features and low-frequency features of the first stage.

[0020] The high-frequency and low-frequency features from the first stage are further fed into a second octave convolutional layer to obtain the high-frequency and low-frequency features from the second stage.

[0021] The high-frequency and low-frequency features from the second stage are further fed into the third octave convolutional layer to obtain the high-frequency and low-frequency features from the third stage.

[0022] Discard the low-frequency features of the first to third stages and use the high-frequency features of the first to third stages as the high-frequency features of teachers.

[0023] As a preferred embodiment, the step of subtracting the extracted degraded image features from the restored features to obtain a first difference feature, and inputting the first difference feature together with the teacher's high-frequency features into the quality prediction module to predict the quality score, thereby obtaining a first quality score, includes:

[0024] The first difference feature is obtained by subtracting the extracted degraded image features from the restored features. The first difference feature The high-frequency features of teachers are input into the quality prediction module to predict quality scores.

[0025] First difference characteristic First, the image passes through a convolutional layer in the quality prediction module and is concatenated with the high-frequency teacher features from the first stage. The concatenated result then passes through a second convolutional layer and is concatenated with the high-frequency teacher features from the second stage. The concatenated result then passes through a third convolutional layer and is concatenated with the high-frequency teacher features from the third stage. Finally, the concatenated result passes through a fourth convolutional layer and a fully connected layer to obtain the first quality score. As a preferred embodiment, the extraction of the original quality image features and the restored features yields a second difference feature. This second difference feature, along with the high-frequency teacher features, is input into the quality prediction module for quality score prediction to obtain the second quality score. This includes:

[0026] The second difference feature is obtained by subtracting the extracted original quality image features from the restored features. Second difference feature The high-frequency features of teachers are input into the quality prediction module to predict quality scores.

[0027] Second difference feature First, after passing through a convolutional layer of the quality prediction module, the result is concatenated with the high-frequency teacher features from the first stage. The concatenated result then passes through a second convolutional layer and is concatenated with the high-frequency teacher features from the second stage. The concatenated result then passes through a third convolutional layer and is concatenated with the high-frequency teacher features from the third stage. Finally, the concatenated result passes through a fourth convolutional layer and a fully connected layer to obtain the second quality score. As a preferred embodiment, the student quality degradation feature extraction module and the teacher quality degradation feature extraction module are trained using the first quality score, the second quality score, and CKD Loss through a knowledge distillation mechanism, resulting in a well-trained student quality degradation feature extraction module, including:

[0028] A knowledge distillation mechanism is introduced, using CKD Loss for training. The teacher's degraded feature extraction module guides the student's degraded feature extraction module in learning. The training result is that the output of the student's degraded feature extraction module approximates the output of the teacher's degraded feature extraction module, thus obtaining a well-trained student degraded feature extraction module. The CKD Loss calculation formula is as follows:

[0029]

[0030] in,

[0031]

[0032]

[0033] L CKD L represents the first conditional knowledge distillation loss. a This indicates the loss of the student quality degradation feature extraction module. λ represents the loss of the teacher quality degradation feature extraction module. CKD Let D represent the loss weights, D represent the degraded image, and H represent the original quality image. This represents the module for extracting student deterioration characteristics. This indicates the module for extracting teacher quality degradation features, s i Let S be the true quality score of the i-th image, where i represents the image index, N represents the total number of images, and S is the true quality score of the i-th image. DH S represents the first mass fraction. HH This indicates the second mass fraction.

[0034] As a preferred embodiment, the method for obtaining the trained student high-frequency feature extraction module is as follows:

[0035] The original quality image corresponding to the degraded image is input into the student high-frequency feature extraction module and then passed through three octave convolutional layers to extract the student high-frequency features.

[0036] The second difference feature The high-frequency features of students are input into the quality prediction module to predict the quality score and obtain the third quality score.

[0037] The second difference feature The high-frequency features of teachers are input into the quality prediction module to predict the quality score and obtain the second quality score.

[0038] By employing the knowledge distillation mechanism, the high-frequency feature extraction modules for students and teachers are trained using the second quality score, the third quality score, and LF Loss, resulting in a well-trained high-frequency feature extraction module for students.

[0039] As a preferred approach, the original quality image corresponding to the degraded image is input into the student high-frequency feature extraction module and then passed through three octave convolutional layers to extract student high-frequency features, including:

[0040] The restored image corresponding to the degraded image is input into the student high-frequency feature extraction module to obtain features. The extracted features are divided into two parts, high-frequency features and low-frequency features, according to scale space theory. The two parts are further fed into an octave convolutional layer for feature extraction to obtain the high-frequency features and low-frequency features of the first stage.

[0041] The high-frequency and low-frequency features from the first stage are further fed into a second octave convolutional layer to obtain the high-frequency and low-frequency features from the second stage.

[0042] The high-frequency and low-frequency features from the second stage are further fed into the third octave convolutional layer to obtain the high-frequency and low-frequency features of the students in the third stage.

[0043] Discard the low-frequency features of the first to third stages and use the high-frequency features of the first to third stages as the high-frequency features of the students.

[0044] As a preferred embodiment, the second difference feature The student's high-frequency features are input into the quality prediction module to predict the quality score, resulting in a third quality score, which includes:

[0045] The second difference feature The student's high-frequency characteristics are input into the quality prediction module to predict the quality score.

[0046] Second difference feature First, after passing through a convolutional layer of the quality prediction module, the result is concatenated with the high-frequency student features from the first stage. The concatenated result is then passed through a second convolutional layer and concatenated with the high-frequency student features from the second stage. This concatenated result is then passed through a third convolutional layer and concatenated with the high-frequency student features from the third stage. Finally, the concatenated result is passed through a fourth convolutional layer and a fully connected layer to obtain the third quality score. As a preferred embodiment, the high-frequency student feature extraction module and the teacher high-frequency feature extraction module are trained using the second quality score, the third quality score, and LF Loss through a knowledge distillation mechanism to obtain a trained high-frequency student feature extraction module, including:

[0047] A knowledge distillation mechanism is introduced, and training is performed using HF Loss. The teacher's high-frequency feature extraction module guides the learning of the student's high-frequency feature extraction module. The training result is that the output of the student's high-frequency feature extraction module approximates the output of the teacher's high-frequency feature extraction module, thus obtaining a well-trained student high-frequency feature extraction module.

[0048] The formula for calculating HF Loss is as follows:

[0049]

[0050] in,

[0051]

[0052]

[0053] L HF The second conditional knowledge distillation loss is represented by λ1, λ2, and λ3, which represent the weight parameters of the first, second, and third stages of the student high-frequency feature extraction module, respectively. i Let i be the true quality score of the i-th image. L represents the loss of the teacher's high-frequency feature extraction module. b Y represents the loss of the high-frequency feature extraction module for students, where i represents the image index, N represents the total number of images, and Y represents the loss of the high-frequency feature extraction module for students. j R Y represents the high-frequency features extracted in the j-th stage of the student high-frequency feature extraction module. j H S represents the high-frequency features extracted in the j-th stage of the teacher high-frequency feature extraction module. HR S represents the third mass fraction. HH Let j represent the second mass fraction, where j takes values ​​of 1, 2, or 3.

[0054] As a preferred embodiment, the formula for calculating the total loss function L is as follows:

[0055] L = L HF +L ckd

[0056] Among them, L CKD L represents the first conditional knowledge distillation loss. HF This represents the loss due to knowledge distillation under the second condition. Beneficial effects: The image quality assessment method based on high-frequency extraction and knowledge distillation provided by this invention is beneficial for improving the prediction of perceived image quality. Furthermore, the method proposed in this invention is an image quality assessment method that uses a degraded image as a reference, which has a wider range of applications compared to a full-reference image quality assessment method that uses the original image as a reference.

[0057] This invention employs a solution that uses a degraded image as a reference to evaluate the quality of the restored image. In natural images, information is transmitted at different frequencies; high frequencies are typically encoded using detail, while low frequencies are typically encoded using global structure. Furthermore, the output feature map of a convolutional layer can also be viewed as a mixture of information at different frequencies. This invention extracts knowledge from the original high-quality image and reference information from the degraded image, employing frequency-based decomposition of the mixed feature map to extract the high-frequency components from the degraded image and apply them to image quality assessment, thereby improving the effectiveness of image quality evaluation. Attached Figure Description

[0058] Figure 1 This is a structural framework diagram of the image quality assessment model of the present invention.

[0059] Figure 2 This is a flowchart of an image quality assessment method based on high-frequency extraction and knowledge distillation according to the present invention. Detailed Implementation

[0060] The present invention will be further described below with reference to specific embodiments.

[0061] like Figure 1 As shown, the present invention provides an image quality assessment method based on high-frequency extraction and knowledge distillation, including an image quality assessment model, which includes: a student degraded feature extraction module, a teacher degraded feature extraction module, a restoration feature extraction network, a student high-frequency feature extraction module, a teacher high-frequency feature extraction module, and a quality prediction module.

[0062] like Figure 2 As shown, the image quality assessment model is trained by following these steps:

[0063] Step 1: Input the degraded image and the corresponding original quality image into the student degraded feature extraction module and the teacher degraded feature extraction module, respectively, to extract the features of the degraded image and the original quality image. Input the restored image corresponding to the degraded image into the restoration feature extraction network to extract the restoration features. Input the original quality image into the teacher high-frequency feature extraction module and then through three octave convolutional layers to extract the teacher high-frequency features. Subtract the extracted degraded image features from the restored features to obtain the first difference feature. Input the first difference feature and the teacher high-frequency features together into the quality prediction module to predict the quality score and obtain the first quality score. Subtract the extracted original quality image features from the restored features to obtain the second difference feature. Input the second difference feature and the teacher high-frequency features together into the quality prediction module to predict the quality score and obtain the second quality score. Using the knowledge distillation mechanism, the first quality score, the second quality score, and CKD Loss are used to train the student degraded feature extraction module and the teacher degraded feature extraction module to obtain a trained student degraded feature extraction module.

[0064] Furthermore, the method of step 1 is as follows:

[0065] Step 1.1: Input the degraded image into the student degraded feature extraction module to extract the degraded image features, and input the original quality image into the teacher degraded feature extraction module to extract the original quality image features.

[0066] Step 1.2: Input the restored image corresponding to the degraded image into the restoration feature extraction network to extract restoration features.

[0067] Step 1.3: Input the original quality image into the teacher high-frequency feature extraction module to obtain features. The extracted features are divided into two parts, high-frequency features and low-frequency features, according to scale space theory. The two parts are further fed into an octave convolutional layer for feature extraction to obtain the first stage of high-frequency and low-frequency features. The first stage of high-frequency and low-frequency features are further fed into a second octave convolutional layer to obtain the second stage of high-frequency and low-frequency features. The second stage of high-frequency and low-frequency features are further fed into a third octave convolutional layer to obtain the third stage of high-frequency and low-frequency features. The low-frequency feature parts of the first to third stages are discarded, and the high-frequency features of the first to third stages are taken as the teacher's high-frequency features.

[0068] Step 1.4: Subtract the extracted degraded image features from the restored features to obtain the first difference feature. The first difference feature The high-frequency features of teachers are input into the quality prediction module to predict quality scores; the first difference feature First, the data passes through a convolutional layer in the quality prediction module and is then concatenated with the high-frequency features of teachers from the first stage. The concatenated result then passes through a second convolutional layer and is concatenated with the high-frequency features of teachers from the second stage. The concatenated result then passes through a third convolutional layer and is concatenated with the high-frequency features of teachers from the third stage. Finally, the concatenated result passes through a fourth convolutional layer and a fully connected layer to obtain the first quality score.

[0069]

[0070]

[0071]

[0072] Where D represents the degraded image and R represents the restored image; This represents the difference between the features extracted by the student quality degradation feature extraction module and the features extracted by the restoration feature extraction network; E2 represents the student quality degradation feature extraction module; E2 represents the restoration feature extraction network; Y j H This represents the high-frequency features extracted by the teacher's high-frequency feature extraction module and the j-th stage of processing by j octave convolutional layers. This indicates that the quality prediction module is spliced ​​together. With Y j H The concatenation features; ⊕ represents the concat operation, conv represents a convolutional layer, FC represents a fully connected layer, S DH This represents the first quality score predicted using the student quality reduction extraction module and the teacher high-frequency feature extraction module.

[0073] Step 1.5: Subtract the extracted original quality image features from the restored features to obtain the second difference feature. Second difference feature The high-frequency features of teachers are input into the quality prediction module to predict quality scores; the second difference feature First, the data passes through a convolutional layer in the quality prediction module and is then concatenated with the high-frequency teacher features from the first stage. The concatenated result then passes through a second convolutional layer and is concatenated with the high-frequency teacher features from the second stage. The concatenated result then passes through a third convolutional layer and is concatenated with the high-frequency teacher features from the third stage. Finally, the concatenated result passes through a fourth convolutional layer and a fully connected layer to obtain the second quality score.

[0074]

[0075]

[0076]

[0077] Where H represents the original quality image and R represents the restored image; This represents the difference between the features extracted by the teacher degradation feature extraction module and the features extracted by the restoration feature extraction network; E1 represents the teacher quality degradation feature extraction module; E2 represents the restoration feature extraction network; This represents the high-frequency features extracted by the teacher's high-frequency feature extraction module and the j-th stage of processing by j octave convolutional layers. This indicates that the quality prediction module is spliced ​​together. and The concatenation features; ⊕ represents the concat operation, conv represents a convolutional layer, FC represents a fully connected layer, S HH This represents the second quality score predicted using the teacher quality reduction extraction module and the teacher high-frequency feature extraction module.

[0078] Step 1.6 introduces a knowledge distillation mechanism, using CKD Loss for training. The teacher degraded feature extraction module guides the student degraded feature extraction module in learning. The training result is that the output of the student degraded feature extraction module approximates the output of the teacher degraded feature extraction module, so that the trained student degraded feature extraction module can efficiently extract features of degraded images.

[0079] CKD Loss is defined as:

[0080]

[0081] in,

[0082]

[0083]

[0084] L CKD L represents the first conditional knowledge distillation loss. a This indicates the loss of the student quality degradation feature extraction module. λ represents the loss of the teacher quality degradation feature extraction module. CKD Let D represent the loss weights, D represent the degraded image, and H represent the original quality image. This represents the module for extracting student deterioration characteristics. This indicates the module for extracting teacher quality degradation features, s i Let S be the true quality score of the i-th image, where i represents the image index, N represents the total number of images, and S is the true quality score of the i-th image. DH S represents the first mass fraction. HH This indicates the second mass fraction.

[0085] Step 2: Input the original quality image corresponding to the degraded image into the student high-frequency feature extraction module, and then pass it through three octave convolutional layers to extract the student high-frequency features; then input the second difference feature... The student's high-frequency features are input into the quality prediction module to predict the quality score, resulting in a third quality score; the second difference features are then used... The high-frequency features of teachers are input into the quality prediction module to predict the quality score, resulting in a second quality score. By referencing the knowledge distillation mechanism, the second quality score, the third quality score, and LF Loss are used to train the high-frequency feature extraction modules for students and teachers, resulting in a well-trained high-frequency feature extraction module for students.

[0086] Furthermore, the method for step 2 is as follows:

[0087] Step 2.1: Input the restored image corresponding to the degraded image into the student high-frequency feature extraction module to obtain features. The extracted features are divided into two parts, high-frequency features and low-frequency features, according to scale space theory. The two parts are further fed into an octave convolutional layer for feature extraction to obtain the high-frequency and low-frequency features of the first stage. The high-frequency and low-frequency features of the first stage are further fed into a second octave convolutional layer to obtain the high-frequency and low-frequency features of the second stage. The high-frequency and low-frequency features of the second stage are further fed into a third octave convolutional layer to obtain the high-frequency and low-frequency features of the student in the third stage. Discard the low-frequency feature parts of the first to third stages and use the high-frequency features of the first to third stages as the high-frequency features of the student.

[0088] Step 2.2, the second difference feature The high-frequency features of students are input into the quality prediction module to predict quality scores; the second difference feature First, the data passes through a convolutional layer in the quality prediction module and is then concatenated with the high-frequency features of students from the first stage. The concatenated result then passes through a second convolutional layer and is concatenated with the high-frequency features of students from the second stage. The concatenated result then passes through a third convolutional layer and is concatenated with the high-frequency features of students from the third stage. Finally, the concatenated result passes through a fourth convolutional layer and a fully connected layer to obtain the third quality score.

[0089]

[0090]

[0091]

[0092]

[0093] Where H represents the original quality image and R represents the restored image; This represents the difference between the features extracted by the teacher degradation feature extraction module and the features extracted by the restoration feature extraction network; E1 represents the teacher quality degradation feature extraction module; E2 represents the restoration feature extraction network; Y represents the teacher quality degradation feature extraction module. j R This represents the high-frequency features extracted by the student high-frequency feature extraction module and the j-th stage of the j-th octave convolutional layer. This indicates that the quality prediction module is spliced ​​together. With Y j R The concatenation features; ⊕ represents the concat operation, conv represents a convolutional layer, FC represents a fully connected layer, S HR This represents the third quality score predicted using the teacher quality reduction extraction module and the student high-frequency feature extraction module.

[0094] Step 2.3 introduces a knowledge distillation mechanism, using HF Loss for training. The teacher's high-frequency feature extraction module guides the student's high-frequency feature extraction module in learning. The training result is that the output of the student's high-frequency feature extraction module approximates the output of the teacher's high-frequency feature extraction module, enabling the trained student high-frequency feature extraction module to efficiently extract the high-frequency features of the restored image.

[0095] HF Loss is defined as:

[0096]

[0097] in,

[0098]

[0099]

[0100] L HF The second conditional knowledge distillation loss is represented by λ1, λ2, and λ3, which represent the weight parameters of the first, second, and third stages of the student high-frequency feature extraction module, respectively. i Let i be the true quality score of the i-th image. L represents the loss of the teacher's high-frequency feature extraction module. b Y represents the loss of the high-frequency feature extraction module for students, where i represents the image index, N represents the total number of images, and Y represents the loss of the high-frequency feature extraction module for students. j R Y represents the high-frequency features extracted in the j-th stage of the student high-frequency feature extraction module. j H S represents the high-frequency features extracted in the j-th stage of the teacher high-frequency feature extraction module. HR S represents the third quality score predicted using the teacher quality reduction extraction module and the student high-frequency feature extraction module. HHLet j represent the second quality score predicted using the teacher quality reduction extraction module and the teacher high-frequency feature extraction module, where j takes values ​​of 1, 2, and 3.

[0101] Step 3: Based on the features extracted by the trained student de-quality feature extraction module, the features extracted by the teacher de-quality feature extraction module, the student high-frequency features extracted by the trained student high-frequency feature extraction module, the teacher high-frequency features extracted by the trained teacher high-frequency feature extraction module, the restoration features extracted by the restoration feature extraction network, and the first, second, and third quality scores output by the quality prediction module, calculate the total loss function of the image quality assessment model. Optimize the image quality assessment model with the goal of minimizing the total loss function. Training is considered complete when the loss decreases and remains stable, resulting in a trained image quality assessment model.

[0102] Furthermore, the method for step 3 is as follows:

[0103] Step 3.1: Based on the degraded image features extracted by the trained student degraded image feature extraction module, the original quality image features extracted by the teacher degraded image feature extraction module, the high-frequency student features extracted by the trained student high-frequency feature extraction module, the high-frequency teacher features extracted by the teacher high-frequency feature extraction module, the restored features extracted by the restored feature extraction network, and the first, second, and third quality scores output by the quality prediction module, calculate the total loss function of the image quality assessment network.

[0104] Step 3.2: Optimize the image quality assessment model using the ADAM method. The goal is to minimize the total loss function. Training is considered complete when the loss decreases and remains stable.

[0105] The total loss function is defined as:

[0106] L = L HF +L ckd

[0107] Step 4: Input the restored image into the trained image quality assessment model and output the quality score of the restored image. The above description is merely a preferred embodiment of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An image quality assessment method based on high-frequency extraction and knowledge distillation, characterized in that: The method comprises the following steps: According to the degraded image features extracted by the trained student degraded feature extraction module, the original quality image features extracted by the teacher degraded feature extraction module, the student high-frequency features extracted by the trained student high-frequency feature extraction module, the teacher high-frequency features extracted by the teacher high-frequency feature extraction module, the restored features extracted by the restored feature extraction network, and the first, second and third quality scores output by the quality prediction module, the total loss function of the image quality evaluation model is calculated; The proposed image quality evaluation model is optimized with the minimum total loss function as the target, and the training is considered to be completed when the loss decreases and remains stable, and a trained image quality evaluation model is obtained; The restored image is input into the trained image quality evaluation model to output the quality score of the restored image; The degraded image and the corresponding original quality image are input into the student degraded feature extraction module and the teacher degraded feature extraction module respectively to extract the degraded image features and the original quality image features; The restored image corresponding to the degraded image is input into the restored feature extraction network to extract the restored features; The original quality image is input into the teacher high-frequency feature extraction module and then passes through three octave convolution layers to extract the teacher high-frequency features; The extracted degraded image features and the restored features are subtracted to obtain first difference features, and the first difference features and the teacher high-frequency features are input into the quality prediction module for quality score prediction to obtain a first quality score; The extracted original quality image features and the restored features are subtracted to obtain second difference features, and the second difference features and the teacher high-frequency features are input into the quality prediction module for quality score prediction to obtain a second quality score; Through the mechanism of knowledge distillation, the first quality score, the second quality score and the CKD Loss are used to train the student degraded feature extraction module and the teacher degraded feature extraction module, and trained student degraded feature extraction module and trained teacher degraded feature extraction module are obtained; The CKD Loss loss calculation formula is: ； wherein, ； ； represents a first condition knowledge distillation loss, represents a student degraded feature extraction module loss, represents a teacher degraded feature extraction module loss, represents a loss weight, D represents a degraded image, and H represents an original quality image, represents a student degraded feature extraction module, represents a teacher degraded feature extraction module, is the first image quality score real value, represents the serial number of the image, represents the total number of images, represents a first quality score, represents a second quality score; The original quality image corresponding to the degraded image is input into the student high-frequency feature extraction module and then passes through three octave convolution layers to extract the student high-frequency features; The second difference features and the student high-frequency features are input into the quality prediction module for quality score prediction to obtain a third quality score; Through the mechanism of knowledge distillation, the second quality score, the third quality score and the LF Loss are used to train the student high-frequency feature extraction module and the teacher high-frequency feature extraction module, and trained student high-frequency feature extraction module and trained teacher high-frequency feature extraction module are obtained; The LF Loss loss calculation formula is as follows: ； wherein, ； ； represents the second conditional knowledge distillation loss, , , respectively represent the weight parameters of the first, second and third stages of the student high-frequency feature extraction module, is the first is the real value of the image quality score, represents the loss of the teacher high-frequency feature extraction module, represents the loss of the student high-frequency feature extraction module, represents the image sequence number, represents the total number of images, represents the high-frequency feature extracted by the jth stage of the student high-frequency feature extraction module, represents the high-frequency feature extracted by the jth stage of the teacher high-frequency feature extraction module, represents the third quality score, represents the second quality score, j takes 1, 2, 3; The calculation formula of the total loss function L is as follows: 。 2. The image quality assessment method based on high-frequency extraction and knowledge distillation according to claim 1, characterized in that: The original quality image is input into the teacher high-frequency feature extraction module to extract the teacher high-frequency features, comprising: The original quality image is input into the teacher high-frequency feature extraction module to obtain features, and the extracted features are divided into high-frequency features and low-frequency features according to the scale space theory, and the two parts are further sent to octave convolution layers for feature extraction to obtain high-frequency features and low-frequency features in the first stage; The high-frequency and low-frequency features of the first stage are further input into a second octave convolution layer to obtain high-frequency features and low-frequency features of the second stage; The high-frequency and low-frequency features of the second stage are further input into a third octave convolution layer to obtain high-frequency features and low-frequency features of the third stage; The low-frequency feature parts of the first to third stages are discarded, and the high-frequency features of the first to third stages are taken as the teacher high-frequency features.

3. The method of claim 1, wherein the method is based on high-frequency extraction and knowledge distillation. The original quality image corresponding to the degraded image is input into the student high-frequency feature extraction module and then passes through three octave convolution layers to extract student high-frequency features, including: The restored image corresponding to the degraded image is input into the student high-frequency feature extraction module to obtain features, and the extracted features are divided into high-frequency features and low-frequency features according to the scale space theory, and the two parts are further input into an octave convolution layer for feature extraction to obtain high-frequency features and low-frequency features of the first stage; The high-frequency and low-frequency features of the first stage are further input into a second octave convolution layer to obtain high-frequency features and low-frequency features of the second stage; The high-frequency and low-frequency features of the second stage are further input into a third octave convolution layer to obtain high-frequency features and low-frequency features of the third stage; The low-frequency feature parts of the first to third stages are discarded, and the high-frequency features of the first to third stages are taken as the student high-frequency features.

Images