No-reference image quality assessment method based on causal-pair intervention

Abstract

The application provides a no-reference image quality evaluation method based on causal pairing intervention. The steps are as follows: a training set and a test set are generated; the samples in the training set and the test set are linearly coded; a feature extraction subnetwork is constructed; a causal inference subnetwork is constructed; a no-reference image quality evaluation network based on causal pairing intervention is constructed; the image quality evaluation network is trained; an image to be evaluated is input to obtain an image quality evaluation result; the application constructs a no-reference image quality evaluation network based on causal pairing intervention, learns the mapping relationship between image distortion and content and image quality by using the trained network, and evaluates the no-reference image quality according to the learned mapping relationship, so that the problem of low accuracy of no-reference image quality evaluation can be effectively solved, and the application has the advantages of strong generalization ability and high image evaluation accuracy.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and more specifically relates to a referenceless image quality assessment method based on causal pairing intervention within the field of image quality assessment methods. This invention can be used to assess image quality loss during the acquisition, transmission, and processing of images in audio-visual entertainment, medical imaging, and aerial remote sensing scenarios. Background Technology

[0002] Blind Image Quality Assessment (BIQA) is an essential method for automatically evaluating the perceived quality of visual signals under no-reference conditions. It is widely used in the design, optimization, and evaluation of image processing, computer vision, and computer graphics applications. Since various types of distortion can be introduced at each stage of these image processing systems, leading to image quality degradation, developing image quality assessment methods that can automatically predict human observer perception is particularly important.

[0003] Currently, no-reference image quality assessment methods are mainly divided into traditional methods based on handcrafted features and methods based on deep learning. Image quality assessment models based on handcrafted features typically consist of a feature extraction unit and a quality regression model. Depending on the feature extraction method, these models can be further divided into methods based on the statistical properties of natural scenes and methods guided by the human visual system. No-reference image quality assessment based on the statistical properties of natural scenes relies on the premise that high-fidelity images adhere to specific statistical properties. However, these statistical properties can change due to image quality degradation. The human visual system is the ultimate receiver of visual signals, and utilizing the perceptual characteristics of the human visual system is crucial in the design of image quality indicators. Currently, the two most prominent methods guided by the human visual system are those based on the free energy principle and those based on visual sensitivity. However, image quality assessment methods based on handcrafted features have some significant drawbacks, including reliance on specialized knowledge, limited generalization ability, and difficulty in adapting to new image types and degradation patterns. These limitations have prompted image quality assessment to shift towards machine learning-based methods to achieve more accurate, flexible, and automated image quality assessment.

[0004] Deep learning-based image quality assessment methods represent a significant leap forward in terms of consistency between subjective and objective assessment methods compared to traditional approaches. Researchers have found a high correlation between abstract features learned from image classification tasks during pre-training and no-reference image quality assessment. The transfer from image classification to no-reference image quality assessment primarily involves supervised constraints on abstract features by image quality labels, enabling the model to effectively build a quality-aware feature manifold and achieve strong generalization capabilities. Currently, many no-reference image quality assessment methods have borrowed strategies from pre-trained language models and their variants for downstream tasks, achieving significant success. However, the projection distance between images with different distortions in the manifold space does not represent the quality-aware distance. In other words, the regression relationship between the quality label MOS score and the image itself is affected by the non-uniformity of the quality-aware feature manifold. This semantically aware pre-trained model encourages similar representations for images of the same category, while ignoring variations in perceived image quality.

[0005] Xi'an University of Electronic Science and Technology proposed a no-reference image quality assessment method based on causal distillation in its patent application "A Causal Distillation-Based Method for Image Quality Assessment" (Patent Application No.: CN 202310830307.3, Publication No.: CN 116843658 A). This method constructs a causal distillation sub-network based on a causal distillation strategy, adjusting feature weights during training. This effectively avoids the catastrophic forgetting problem caused by distillation loss in transfer learning, as seen in existing techniques. This results in good prediction accuracy across multiple datasets, significantly improving the accuracy of no-reference image quality assessment. However, this method still has shortcomings. Because it uses transfer learning, it heavily relies on the empirical selection of weights and similarity metrics. The data distributions in the source and target domains are not entirely identical, and it lacks continuous learning for new image datasets. It cannot constrain the synchronous updating of distillation information in the distillation sub-network, making the causal feature weights unable to adapt to the image quality label distribution. Furthermore, since it still uses transfer learning, it cannot completely avoid the prominent forgetting problem, ultimately leading to low generalization ability in multi-domain image quality assessment and inaccurate image quality evaluation, thus reducing accuracy. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of the prior art by proposing a no-reference image quality assessment method based on causal pairing intervention. This method addresses the issues that existing deep learning methods are only robust to image distortion but sensitive to image content, and that the lack of continuous learning on new image datasets leads to biases in image quality assessment across different domains.

[0007] The technical approach to achieving the objective of this invention is as follows: In constructing a no-reference image quality assessment network with causal pairing intervention, the causal inference subnetwork of the quality assessment network is composed of hierarchical pooling units and convolutional layers connected in series. The hierarchical pooling units include channel-level pooling layers, which effectively prevents excessive loss of information during the pooling process due to average pooling in the regressor. This results in rich and effective image feature information and image quality score label distribution information, effectively improving the accuracy of the quality assessment network. Because this invention uses a causal pairing loss function and an absolute value loss function to train the causal inference subnetwork of the quality assessment network—where the causal pairing loss function is an improvement on the general framework of traditional prior Bayesian methods—it uses causal feature intervention to establish the causal relationship between image content, image distortion, and image quality. Causal inference methods typically rely on extensive prior knowledge about the causal properties of the underlying system, thus solving the problem that existing deep learning methods are only robust to image distortion but sensitive to image content. Because this invention uses the distribution characteristics of image quality labels to update the samples that affect the mapping weights when training the causal inference subnetwork of the quality evaluation network, it solves the problem of low accuracy caused by the lack of continuous learning for new image datasets and the inability to continuously learn and update the mapping weights based on sample data in the existing technology.

[0008] The technical solution adopted in this invention includes the following steps:

[0009] Step 1: Linearly encode the samples in the training set;

[0010] Step 2: Construct a feature extraction subnetwork consisting of ResNet50 units, dimension transformation units, and fully connected layers connected in series; set the number of output nodes of the fully connected layers to 16.

[0011] Step 3: Construct a causal reasoning subnetwork consisting of hierarchical pooling units and convolutional layers connected in series;

[0012] Step 4: Connect the feature extraction subnetwork and the causal inference subnetwork in parallel to obtain a no-reference image quality assessment network based on causal pairing intervention;

[0013] Step 5: Train a no-reference image quality assessment network based on causal paired intervention using a loss function constraint.

[0014] Step 5.1: Sample according to the distribution characteristics of image quality score labels to obtain a specially selected sample set, and input the specially selected sample set into the causal inference subnetwork of the quality assessment network;

[0015] Step 5.2: Input the encoded training set into the feature extraction subnetwork of the quality evaluation network;

[0016] Step 5.3: Using the Adam optimizer and the cosine annealing learning rate adjustment algorithm, the parameters of the quality assessment network are iteratively updated until the loss function of the quality assessment network, which consists of the absolute value loss function and the causal pairing loss function, converges, and the trained quality assessment network is obtained.

[0017] Step 6: Input the image to be evaluated to obtain the image quality evaluation result:

[0018] Using the same method as in step 1, the image to be evaluated is processed, and the processed image is input into the trained no-reference image quality assessment network based on causal pairing intervention, and the image quality assessment score is output.

[0019] Compared with the prior art, the present invention has the following advantages:

[0020] First, because this invention constructs a causal inference subnetwork composed of hierarchical pooling units and convolutional layers connected in series, the network can extract image feature information and image quality distribution information in layers, overcoming the problem that average pooling in the regressor causes excessive loss of information during the pooling process. This allows the invention to learn the mapping of image feature information to image quality scores from rich information, thereby improving the accuracy of image quality assessment.

[0021] Second, because this invention uses a causal pairing loss function to train the causal inference subnetwork of the quality evaluation network, the network can accurately learn the causal mapping relationship between quality score and image feature information. This overcomes the shortcomings of existing deep learning methods that are only robust to image distortion but sensitive to image content. This invention can evaluate image quality based on the distortion and content of the image to be evaluated, thus improving the accuracy of image quality evaluation.

[0022] Third, because the present invention uses the distribution characteristics of image quality labels to update the samples affecting the mapping weights when training the causal inference subnetwork of the quality evaluation network, and uses the method of dynamically updating the samples affecting the mapping weights according to the Gaussian distribution of image quality score labels, it overcomes the shortcomings of existing technologies that lack continuous learning for new image datasets. This allows the present invention to improve the generalization ability of image quality evaluation by dynamically updating the causal feature weights of the collected samples. Attached Figure Description

[0023] Figure 1 This is a flowchart of an embodiment of the present invention;

[0024] Figure 2 This is a schematic diagram of the no-reference image quality assessment network structure based on causal pairing intervention constructed in this invention. Detailed Implementation

[0025] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0026] Reference Figure 1 The implementation steps of the embodiments of the present invention will be described in further detail.

[0027] Step 1: Generate training and test sets.

[0028] In the embodiments of the present invention, the KonIQ-10k known database refers to the image quality assessment database proposed by Hosu V et al. in “Koniq-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment[J].IEEE Transactions on Image Processing, 2020, 29: 4041-4056.”, abbreviated as KonIQ-10k known database.

[0029] A training set of 3000 W×H×C RGB images was selected from the KonIQ-10k public database, and a test set of 1000 W×H×C RGB images was selected. Here, W, H, and C represent the width, height, and number of channels of the RGB image, respectively, with W≥224, H≥224, and C=3.

[0030] Each image in both the training and test sets is cropped into a 224×224×3 sub-image, and each cropped sub-image is then randomly flipped and pixel normalized sequentially.

[0031] Step 2: Linearly encode the samples in the training and test sets.

[0032] Each cropped sub-image is divided into non-overlapping image blocks, resulting in 14 local blocks of size 16×16×3. These blocks are then processed through a linear embedding coding layer h. i =LP(x i ):R 16×16×3 →R 512 Each image patch is mapped to a 512-dimensional feature vector h. i .

[0033] Step 3: Construct a feature extraction subnetwork consisting of ResNet50 units, dimension transformation units, and fully connected layers connected in series.

[0034] The ResNet50 unit is a commonly used deep residual network.

[0035] The dimension transformation unit consists of a convolutional layer, a normalization layer, and an activation layer connected in series. The kernel size of the convolutional layer is set to 3, the stride to 1, and the padding to 1. The function of the dimension transformation unit is to match the dimension of the extracted feature vector with the dimension of the inference result of the causal inference network. The activation layer is implemented using the ReLU function. The number of output nodes of the fully connected layer is set to 16.

[0036] Step 4: Construct a causal reasoning subnetwork consisting of hierarchical pooling units and convolutional layers connected in series.

[0037] The hierarchical pooling unit includes a branch consisting of a convolutional pooling layer group, a first residual layer, a second residual layer, a third residual layer, and a fourth residual layer connected in series. Each residual layer is connected in series with a channel-level hierarchical pooling layer before being connected to the output. The convolutional pooling layer group consists of a convolutional layer, a normalization layer, an activation layer, and a pooling layer connected in series. The convolutional layer consists of 64 convolutional kernels, each with a size of 7 and a stride of 2. The activation layer is implemented using the ReLU function. The pooling layer is max pooling with a kernel size of 3 and a stride of 2.

[0038] The first residual layer comprises nine convolutional layers connected in series, with a skip connection every three convolutional layers; the kernel sizes are set sequentially to 1, 3, 1, 1, 3, 1, 1, 3, 1, and the number of kernels is sequentially 64, 64, 256, 64, 64, 512, 64, 64, 512, with a stride of 1 for each kernel; the activation layer is implemented using the ReLU function.

[0039] The second residual layer comprises 12 convolutional layers connected in series, with a skip connection every three convolutional layers; the kernel sizes are set sequentially to 1, 3, 1, 1, 3, 1, 1, 3, 1, 1, 3, 1, and the number of kernels is sequentially 128, 128, 512, 128, 128, 512, 128, 128, 512, 128, 128, 512, and the stride is set to 1; the activation layer is implemented using the ReLU function.

[0040] The third residual layer comprises 18 convolutional layers connected in series, with a skip connection every three convolutional layers. The kernel sizes are set sequentially to 1, 3, 1, 1, 3, 1, 1, 3, 1, 1, 3, 1, 1, 3, 1, 1, 3, 1, and the number of kernels is 256, 256, 1024, 256, 256, 1024, 256, 256, 1024, 256, 256, 1024, 256, 256, 1024, 256, 256, 1024, 256, 256, 1024, with a stride of 1 for each kernel. The activation layer is implemented using a ReLU function.

[0041] The fourth residual layer comprises nine convolutional layers connected in series, with a skip connection every three convolutional layers; the kernel sizes are set sequentially to 1, 3, 1, 1, 3, 1, 1, 3, 1, and the number of kernels is 512, 512, 2048, 512, 512, 2048, 512, 512, 2048, and the stride is set to 1; the activation layer is implemented using the ReLU function.

[0042] The channel-level pooling layer includes a convolutional layer and a pooling layer. The convolutional layer is composed of 16 convolutional kernels with a kernel size of 3 and a stride of 1, connected in series. The pooling layer is an average pooling layer with a kernel size of 7.

[0043] The convolutional layer group is composed of a first convolutional layer, a second convolutional layer, a first pooling layer, a third convolutional layer, a fourth convolutional layer, a second pooling layer, a fifth convolutional layer, a sixth convolutional layer, a third pooling layer, a seventh convolutional layer, an eighth convolutional layer, a fourth pooling layer, a ninth convolutional layer, a tenth convolutional layer, a fifth pooling layer, and a fully connected layer connected in series. The kernel size of the first to tenth convolutional layers is set to 3, the stride is set to 1, and the padding is set to 1. The number of convolutional kernels is 16, 64, 64, 256, 64, 64, 256, 64, 64, 256, respectively. The first to fifth pooling layers are all set to max pooling with a kernel size of 3. The number of output nodes of the fully connected layer is set to 16.

[0044] Step 5: Connect the feature extraction subnetwork and the causal inference subnetwork in parallel to obtain a referenceless image quality assessment network based on causal pairing intervention. Pair the image feature information extracted by the feature extraction subnetwork with the mapping weights output by the causal inference subnetwork through the intervention set, and output the pairing result to the network.

[0045] Step 6: Use the loss function to constrain the training of the no-reference image quality assessment network based on causal paired intervention.

[0046] Step 6.1: Sampling is performed based on the distribution characteristics of the image quality score labels. A Gaussian distribution is used to sample the image quality score labels of the encoded training set samples. All sampled image feature vectors and corresponding quality score labels are combined to form a specially selected sample set. The specially selected sample set is then input into the causal inference subnetwork of the quality evaluation network.

[0047] Step 6.2: Input the encoded training set into the feature extraction subnetwork of the quality assessment network; Step 6.3: Use the Adam optimizer combined with the cosine annealing learning rate adjustment algorithm to iteratively update the parameters of the quality assessment network. The number of training epochs is set to 30, the training batch size is 32, and the initial learning rate is 2×10⁻⁶. -7 The warm-start learning rate is 2×10. -7After 4 epochs of model warm-up, the learning rate reaches its maximum value of 2×10⁻⁶. -5 Then, perform a cosine roll-off to 2×10. -7 The training continues until the loss function of the quality assessment network, which consists of the absolute value loss function and the causal pairing loss function, converges, thus obtaining the trained quality assessment network for no-reference images based on causal pairing intervention.

[0048] The absolute value loss function is as follows:

[0049]

[0050] Where L1 represents the absolute value loss function, N represents the total number of samples in the training set, P represents the entire training set, x(p) represents the estimated quality score label of the p-th sample in the training set, and y(p) represents the true quality score label of the p-th sample in the training set.

[0051] The causal pairing loss function is as follows:

[0052]

[0053] Among them, L c Let E(x) represent the causal pairing loss function, where N represents the total number of samples in the training set. i ) represents the feature encoding of the i-th sample image information, w i，j The table shows the mapping weights for causal inference based on the information of the i-th sample image, where 1 ≤ j ≤ |B|, and |B| represents the total number of specially selected sample sets.

[0054] In the embodiments of the present invention, |B| is set to 16, G(E(x) i )) represents the auxiliary sample data output by the feature extraction subnetwork for the i-th sample.

[0055] The loss function of the quality evaluation network is as follows:

[0056] L=ε1L1+ε2L c

[0057] Where L represents the loss function of the no-reference image quality assessment network, ε1 and ε2 represent the weight factors for relative importance trade-offs in the loss term, ε1 and ε2 are set to 0.99 and 0.01 respectively, and L1 and L2 represent the absolute value loss function and the causal pairing loss function respectively.

[0058] Step 7: Input the image to be evaluated to obtain the image quality evaluation result.

[0059] The encoded test set is input into a trained image quality assessment network based on causal pairing intervention for no-reference images, and the network outputs image quality assessment scores.

[0060] The effects of this invention will be further illustrated below with simulation experiments:

[0061] 1. Simulation experimental conditions:

[0062] The hardware platform for the simulation experiment of this invention is: the CPU and graphics card are Intel(R) Core(TM) i9-7900X CPU 3.30GHz and NVIDIA Geforce RTX 3090 GPU, respectively.

[0063] The software platform for the simulation experiment of this invention is the Ubuntu 20.04.6LTS operating system. The software environment is mainly based on the NVIDIA GeForce RTX 3090 GPU with CUDA version 11.3, using the PyTorch deep learning framework with version 11.8 for training and testing.

[0064] The input images used in the simulation experiments of this invention are derived from the well-known image quality assessment databases TID2013, KonIQ-10k, and LIVEC.

[0065] The TID2013 database includes 25 reference images and 3,000 distorted images.

[0066] The KonIQ-10k database contains 10,073 distorted images.

[0067] The LIVEC database contains 1,162 distorted images.

[0068] 2. Simulation content and result analysis:

[0069] The simulation experiment of this invention uses this invention and two existing technologies (P2P-BM and MetaIQA) to perform no-reference image quality evaluation on distorted images in three known image quality evaluation databases: TID2013, KonIQ-10k, and LIVEC.

[0070] The evaluation index is obtained by calculating the consistency between the quality prediction value of the no-reference image and the image label. The evaluation index is used to measure the no-reference image evaluation effect of the present invention and two existing technologies (P2P-BM method and MetaIQA method) on the distorted images in the above three known image quality evaluation databases.

[0071] The two existing technologies used in the simulation experiment are:

[0072] The P2P-BM method refers to the referenceless image quality assessment method proposed by Ying Z et al. in "From Patches to Pictures (Paq-2-Piq): Mapping the Perceptual Space of Picture Quality [C]. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2020:3575-3585.", abbreviated as P2P-BM method.

[0073] MetaIQA refers to the no-reference image quality assessment method proposed by Zhu H et al. in "MetaIQA: Deep meta-learning for no-reference image quality assessment[C] / / Proceedings of the IEEE / CVF conferenceon computer vision and pattern recognition.2020:14143-14152.", abbreviated as MetaIQA method.

[0074] In the simulation experiment, the three known image quality assessment databases used are:

[0075] The TID2013 public database refers to the image quality assessment database proposed by Ponomarenko N et al. in "Color Image Database TID2013: Peculiarities and Preliminary Results [C]. In European Workshop on Visual Information Processing. IEEE, 2013: 106-111.", abbreviated as TID2013 public database.

[0076] The KonIQ-10k public database refers to the image quality assessment database proposed by Hosu V et al. in "Koniq-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment[J].IEEE Transactions on Image Processing,2020,29:4041-4056.", abbreviated as KonIQ-10k public database.

[0077] The LIVEC database refers to the image quality assessment database proposed by Ghadiyaram D et al. in “Massive Online Crowdsourced Study of Subjective and Objective Picture Quality[J].IEEE Transactions on Image Processing,2015,25(1):372-387.”, abbreviated as LIVEC database.

[0078] To evaluate the effectiveness of this invention and two other prior art methods in evaluating the quality of no-reference images, this simulation experiment uses two indicators: Spearman Rank-Order Correlation Coefficient (SROCC) and Pearson Linear Correlation Coefficient (PLCC), to objectively assess the effectiveness of this invention and the two prior art methods in evaluating the quality of no-reference images.

[0079] The Spearman Rank Order Correlation Coefficient (SROCC) determines the strength and direction of the monotonic relationship between two variables, measuring the monotonicity of the algorithm's predictions. Its expression is:

[0080]

[0081] Where, r xi r represents the subjective quality assessment result of the i-th image to be tested. yi This represents the objective quality evaluation result, (r xi -r yi ) 2 This indicates the magnitude of the difference between the two, which is calculated by sorting the difference set.

[0082] Pearson Linear Correlation Coefficient (PLCC), x i and y i These represent the subjective quality assessment score and the objective score of the i-th tested image, respectively. Their expressions are:

[0083]

[0084] Where n represents the total number of images, and These are the average of the subjective evaluation scores given by the human eye to the database and the evaluation scores automatically calculated by the objective evaluation algorithm. The linear correlation coefficient describes the correlation between the algorithm's evaluation value and the subjective human score, and also measures the accuracy of the algorithm's predictions.

[0085] The simulation experiment of this invention uses the method of this invention and two existing technologies to evaluate images in three different known databases, and calculates two consistency indices for the evaluation results of each method. The calculation results are shown in Table 1.

[0086] Table 1. Comparison of Evaluation Results of the Three Methods

[0087]

[0088] As shown in Table 1, the Spearman rank correlation coefficient (SROCC) and Pearson linear correlation coefficient (PLCC) of the evaluation results of the present invention on three known image quality assessment databases are both higher than those of the two existing technologies, proving that the present invention has a better effect on no-reference image quality assessment.

Claims

1. A no-reference image quality assessment method based on causal-pair intervention, characterized in that, The causal inference subnetwork is constructed by connecting a hierarchical pooling unit and a convolutional layer group in series, and the causal inference subnetwork of the quality evaluation network is trained using a causal pairing loss function and samples obtained by sampling according to the distribution characteristics of the image quality score label; the steps of the no-reference image quality evaluation method are as follows: Step 1, linearly encoding the samples in the training set; Step 2, constructing a feature extraction subnetwork composed of a ResNet50 unit, a dimension conversion unit, and a full connection layer connected in series; The number of output nodes of the full connection layer is set to 16; Step 3, constructing a causal inference subnetwork composed of a hierarchical pooling unit and a convolutional layer group connected in series; Step 4, connecting the feature extraction subnetwork and the causal inference subnetwork in parallel to obtain a no-reference image quality evaluation network based on causal pairing intervention; Step 5, using a loss function to constrain the training of the no-reference image quality evaluation network based on causal pairing intervention: Step 5.1, according to the distribution characteristics of the image quality score label, sampling to obtain a specially selected sample set, and inputting the specially selected sample set into the causal inference subnetwork of the quality evaluation network; Step 5.2, inputting the encoded training set into the feature extraction subnetwork of the quality evaluation network; Step 5.3, using the Adam optimizer combined with the cosine annealing learning rate adjustment algorithm to iteratively update the parameters of the quality evaluation network until the loss function of the quality evaluation network composed of the absolute value loss function and the causal pairing loss function converges, obtaining the trained quality evaluation network; Step 6, inputting the image to be evaluated to obtain the image quality evaluation result: The same method as step 1 is used to process the image to be evaluated, and the processed image is input into the trained no-reference image quality evaluation network based on causal pairing intervention, and the image quality evaluation score is output.

2. The no-reference image quality assessment method based on causal pairing intervention according to claim 1, characterized in that, The training set in step 1 is generated by the following steps: First, L images of WxHxC are grouped into a sample set, where L≥1000, W, H, and C represent the width, height, and channel number of the RGB image respectively, W≥224, H≥224, and C≥3; Second, each image in the sample set is cropped into a w x h x c sub-image, and each cropped sub-image is sequentially processed by random flipping and pixel normalization, where w, h, and c represent the width, height, and channel number of the input image of the no-reference image quality evaluation network based on causal pairing intervention; Third, a quality score label is added to each processed sub-image; Fourth, all processed samples and their labels are grouped into a sample set.

3. The method of claim 1, wherein the method is characterized by, The step of linearly encoding the samples in the training set in step 1 is as follows: First, each cropped sub-image is divided into non-overlapping image blocks to obtain M local blocks of P x P x 3, where M represents the number of non-overlapping image blocks into which each cropped sub-image is divided, P, P, and 3 represent the width, height, and channel number of the image block respectively; Second step, M local blocks with size P x P x 3 are obtained, and then linear embedding coding layer h i = LP(x i ): R P×P×3 → R D is used to map each image block to a D-dimensional feature vector h i , where D is the linear coding dimension of the image block, and P is the dimension of the image block.

4. The method of claim 1, wherein the method is characterized by, The dimension conversion unit in step 2 is composed of a convolution layer, a normalization layer and an activation layer in sequence, the convolution kernel size of the convolution layer is set to 3, the step is set to 1, and the padding is set to 1; the activation layer is realized by a Relu function.

5. The method of claim 1, wherein the method is characterized by, The hierarchical pooling unit in step 3 comprises a convolution pooling layer group, a first residual layer, a second residual layer and a third residual layer connected in sequence, wherein each residual layer is connected with an output end after a channel hierarchical pooling layer. The convolution pooling layer group is composed of a convolution layer, a normalization layer, an activation layer and a pooling layer in sequence; the convolution layer is composed of 64 convolution kernels, each of which has a size of 7 and a step of 2; the activation layer is realized by a Relu function; and the pooling layer is a maximum pooling layer with a kernel size of 3 and a step of 2. The first residual layer comprises 9 convolution layers connected in sequence, wherein every three convolution layers are connected by a jump connection; the convolution kernel size is set to 1, 3, 1, 1, 3, 1, 1, 3, 1 in sequence, the number of convolution kernels is 64, 64, 256, 64, 64, 512, 64, 64, 512 in sequence, and the step is set to 1; and the activation layer is realized by a Relu function. The second residual layer comprises 12 convolution layers connected in sequence, wherein every three convolution layers are connected by a jump connection; the convolution kernel size is set to 1, 3, 1, 1, 3, 1, 1, 3, 1, 1, 3, 1 in sequence, the number of convolution kernels is 128, 128, 512, 128, 128, 512, 128, 128, 512, 128, 128, 512 in sequence, and the step is set to 1; and the activation layer is realized by a Relu function. The third residual layer comprises 18 convolution layers connected in sequence, wherein every three convolution layers are connected by a jump connection; the convolution kernel size is set to 1, 3, 1, 1, 3, 1, 1, 3, 1, 1, 3, 1, 1, 3, 1, 1, 3, 1 in sequence, the number of convolution kernels is 256, 256, 1024, 256, 256, 1024, 256, 256, 1024, 256, 256, 1024, 256, 256, 1024, 256, 256, 1024 in sequence, and the step is set to 1; and the activation layer is realized by a Relu function. The fourth residual layer comprises 9 convolution layers connected in sequence, wherein every three convolution layers are connected by a jump connection; the convolution kernel size is set to 1, 3, 1, 1, 3, 1, 1, 3, 1 in sequence, the number of convolution kernels is 512, 512, 2048, 512, 512, 2048, 512, 512, 2048 in sequence, and the step is set to 1; and the activation layer is realized by a Relu function. The channel hierarchical pooling layer comprises a convolution layer and a pooling layer, wherein the convolution layer is composed of convolution kernels with a size of 3, a step of 1 and a number of 16; and the pooling layer is an average pooling layer with a kernel size of 7.

6. The method of claim 1, wherein the method is characterized by, The convolution layer group in step 3 is composed of a first convolution layer, a second convolution layer, a first pooling layer, a third convolution layer, a fourth convolution layer, a second pooling layer, a fifth convolution layer, a sixth convolution layer, a third pooling layer, a seventh convolution layer, an eighth convolution layer, a fourth pooling layer, a ninth convolution layer, a tenth convolution layer, a fifth pooling layer, and a full connection layer connected in sequence; the convolution kernel size of the first to tenth convolution layers is all set to 3, the step is all set to 1, the padding is all set to 1, and the number of convolution kernels is 16, 64, 64, 256, 64, 64, 256, 64, 64, and 256 in sequence; the first to fifth pooling layers are all set to maximum pooling with a kernel size of 3; and the output node number of the full connection layer is set to 16.

7. The method of claim 1, wherein the method is characterized by, The sampling according to the distribution characteristics of the image quality score labels in step 5.1 refers to sampling the image quality score labels of the encoded training set samples in a Gaussian distribution manner, and grouping all the image feature vectors and corresponding quality score labels after sampling into a specially selected sample set.

8. The method of claim 1, wherein the method is based on causal pairing intervention. The absolute value loss function in step 5.3 is as follows: Wherein, L1 represents the absolute value loss function, N represents the total number of training set samples, P represents the entire training set sample, x(p) represents the quality score label estimation value of the pth sample in the training set, and y(p) represents the quality score label true value of the pth sample in the training set.

9. The method of claim 1, wherein the method is based on causal pair-wise intervention. The causal pairing loss function in step 5.3 is as follows: wherein, L c represents the causal pair loss function, N represents the total number of samples in the training set, E(x i ) represents the feature encoding of the i-th sample image information, w i，j represents the mapping weight of the i-th sample image information for causal inference, 1≤j≤|B|, |B| represents the total number of the specially selected sample set, G(E(x i )) represents the auxiliary sample data output by the i-th sample through the feature extraction subnetwork.

10. The method of claim 1, wherein the method is based on causal pairing intervention. The loss function of the quality evaluation network in step 5.3 is as follows: L = ε1L1+ ε2L c Wherein, L represents the loss function of the no-reference image quality evaluation network, and ε1 and ε2 represent weight factors for relative importance weighting in the loss term, and are respectively set to 0.99 and 0.01; L1 and L2 represent the absolute value loss function and the causal pairing loss function, respectively.

Images