Image quality evaluation method and system based on combination of transformer and few-shot learning

Abstract

The application discloses an image quality evaluation method and system based on combination of a Transformer and few-shot learning, and relates to the technical field of image quality evaluation. An image quality evaluation model is proposed, which comprises an image restoration network branch and a score evaluation network based on the Transformer. The former is used for restoring a distorted image and extracting a feature map, and the latter extracts quality features and fuses the quality features with the feature map to output a quality score. First, the model is trained by using a general large-scale data set, and then the model is fine-tuned by using a self-built data set containing a small number of target scene samples and scores, so that the labeling cost is reduced and the generalization ability of the model is enhanced. The method can simultaneously capture local details and global structure information, improves the evaluation precision and robustness in multiple distortions and multiple scenes, and provides reliable data support for downstream tasks.

Description

Technical Field

[0001] This invention relates to the field of image quality assessment technology, specifically to an image quality assessment method and system based on a combination of Transformer and few-shot learning. Background Technology

[0002] In the intelligent construction of power transmission and transformation projects, image quality assessment is a crucial step in ensuring data reliability and the accuracy of subsequent analysis. High-quality images are not only essential for the accuracy of equipment status detection, construction quality monitoring, and safety violation identification, but also a prerequisite for the effective operation of intelligent annotation systems.

[0003] However, current technologies still have many shortcomings in image quality assessment. First, the environment at power transmission and transformation construction sites is complex. Factors such as changes in lighting, equipment aging, differences in sensor performance, and electromagnetic interference can easily lead to problems such as blurring, noise, and uneven exposure in images. Traditional assessment methods struggle to objectively and comprehensively quantify these quality defects. Second, existing image assessment models based on convolutional neural networks (CNNs) have limited feature extraction capabilities when dealing with complex images involving multiple scenes and various distortion types, especially in long-range dependency modeling, resulting in low stability and generalization ability of the assessment results. Furthermore, models typically rely on large-scale, high-quality labeled data for training, but high-quality sample collection in power transmission and transformation scenarios is costly and unevenly distributed, often resulting in small or scarce samples, making existing technologies insufficiently adaptable to such scenarios. Summary of the Invention

[0004] The purpose of this invention is to solve the problems mentioned in the background art, such as insufficient evaluation accuracy, poor generalization, and difficulty in meeting the needs of complex scenarios with few samples, and to propose an image quality evaluation method and system based on the combination of Transformer and few-sample learning.

[0005] A first aspect of this invention provides an image quality assessment method based on a combination of Transformer and few-shot learning, the method comprising:

[0006] A first dataset is obtained, and an image quality assessment model with a preset architecture is trained using the first dataset to obtain a first assessment model. The first dataset is a large-scale dataset containing real images, distorted images, and quality scores. The network architecture of the image quality assessment model includes a first network branch and a second network branch. The first network branch is used to restore distorted images to real images and output feature maps during the distorted image restoration process. The second network branch is a score evaluation network built based on Transformer, used to extract quality features of distorted images and fuse the quality features with the feature maps output by the first network branch, and output a quality score through a fully connected network.

[0007] Obtain a second dataset and fine-tune the first evaluation model using the second dataset to obtain a second evaluation model; the second dataset is a self-built dataset containing a small number of target scene image samples and quality scores;

[0008] The second evaluation model is applied to the quality evaluation process of the target scene image.

[0009] Optionally, training the image quality assessment model with a preset architecture using the first dataset to obtain the first assessment model includes:

[0010] Step 1: Train the preset image restoration model using the first dataset; the image restoration model is built based on the U-Net model and includes encoder modules E1, E2, E3, E4, E5 and decoder modules D1, D2, D3, D4, D5 connected in sequence.

[0011] Step 2: Delete the last two modules D4 and D5 of the image restoration model to obtain the first network branch; denote the outputs of the decoder modules D1, D2, and D3 as the restored feature maps;

[0012] Step 3: Train the preset score evaluation model using the first dataset;

[0013] The score evaluation model includes a feature extraction network, a feature fusion network, a natural scene statistics network, and a score prediction network; among which...

[0014] The feature extraction network receives the distorted image as input and outputs feature maps of multiple scales through multiple Transformer layers, which are denoted as quality feature maps.

[0015] The feature fusion network fuses the recovery feature map and the quality feature map at the corresponding scale to obtain the first evaluation feature;

[0016] The natural scene statistical network performs local normalization and generalized Gaussian distribution fitting on the input distorted image to obtain natural statistical features, and then performs deep learning on the natural statistical features through a fully connected layer to obtain the second evaluation features.

[0017] The score prediction network uses a fully connected network to map the first evaluation feature and the second evaluation feature into quality scores.

[0018] Optionally, the feature fusion network includes multiple primary fusion modules (FFM) and one secondary fusion module (SFM); the multiple primary fusion modules have the same structure; wherein:

[0019] Any one of the fusion modules is used to fuse the restored feature map and the quality feature map to obtain the fused feature map;

[0020] The secondary fusion module is used to perform secondary fusion of fusion feature maps at multiple scales to obtain the first evaluation feature.

[0021] Optionally, any one-time fusion module FFM includes two structurally identical multi-scale feature extraction modules MFEM and attention enhancement modules CAM;

[0022] The operation expression for any one-time fusion module is:

[0023] ;

[0024] in, and These are the restored feature map and the quality feature map, respectively, which are the inputs to the FFM module; Y is the output of the FFM module; Concat represents the concatenation operation; represents global average pooling; FC represents a fully connected layer; the operational expression for the module MFEM is:

[0025] ;

[0026] Where x is the input of module MFEM, and so on. and X5 is the output of module MFEM; , , , These represent convolutional layers with kernel sizes of 1×1, 3×3, 5×5, and 7×7, respectively. and These represent max pooling and average pooling along the channel dimension, respectively.

[0027] Optionally, the operational expression of the secondary fusion module is:

[0028] ;

[0029] in, , and EF1 is the input of module SFM, which is provided by three primary fusion modules; EF1 is the output of module SFM. Represents adaptive L2 pooling, used to... and Spatial resolution adjusted to match same; This involves applying Softmax normalization to the feature map in each channel dimension to obtain the spatial weight distribution, and then performing weighted average pooling. These are learnable parameters.

[0030] A second aspect of this invention provides an image quality assessment system based on a combination of Transformer and few-shot learning, the system comprising:

[0031] A pre-training module is used to acquire a first dataset and train a pre-defined image quality assessment model using the first dataset to obtain a first assessment model. The first dataset is a large-scale dataset containing real images, distorted images, and quality scores. The network architecture of the image quality assessment model includes a first network branch and a second network branch. The first network branch is used to restore distorted images to real images and output feature maps during the distorted image restoration process. The second network branch is a score evaluation network built based on Transformer, used to extract quality features of distorted images and fuse the quality features with the feature maps output by the first network branch, outputting a quality score through a fully connected network.

[0032] The fine-tuning module is used to acquire a second dataset and fine-tune the first evaluation model using the second dataset to obtain a second evaluation model; the second dataset is a self-built dataset containing a small number of target scene image samples and quality scores.

[0033] The scheduling module is used to apply the second evaluation model to the quality evaluation process of the target scene image.

[0034] Optionally, the pre-training module includes:

[0035] The reconstruction model training module is used to train a preset image restoration model using the first dataset; the image restoration model is built based on the U-Net model and includes encoder modules E1, E2, E3, E4, E5 and decoder modules D1, D2, D3, D4, D5 connected in sequence.

[0036] The branch determination module is used to delete the last two modules D4 and D5 of the image restoration model to obtain the first network branch; and to denote the outputs of the decoder modules D1, D2 and D3 as the restored feature maps.

[0037] The evaluation model training module is used to train a preset score evaluation model using the first dataset;

[0038] The score evaluation model includes a feature extraction network, a feature fusion network, a natural scene statistics network, and a score prediction network; among which...

[0039] The feature extraction network receives the distorted image as input and outputs feature maps of multiple scales through multiple Transformer layers, which are denoted as quality feature maps.

[0040] The feature fusion network fuses the recovery feature map and the quality feature map at the corresponding scale to obtain the first evaluation feature;

[0041] The natural scene statistical network performs local normalization and generalized Gaussian distribution fitting on the input distorted image to obtain natural statistical features, and then performs deep learning on the natural statistical features through a fully connected layer to obtain the second evaluation features.

[0042] The score prediction network uses a fully connected network to map the first evaluation feature and the second evaluation feature into quality scores.

[0043] Optionally, the feature fusion network includes multiple primary fusion modules (FFM) and one secondary fusion module (SFM); the multiple primary fusion modules have the same structure; wherein:

[0044] Any one of the fusion modules is used to fuse the restored feature map and the quality feature map to obtain the fused feature map;

[0045] The secondary fusion module is used to perform secondary fusion of fusion feature maps at multiple scales to obtain the first evaluation feature.

[0046] Optionally, any one-time fusion module FFM includes two structurally identical multi-scale feature extraction modules MFEM and attention enhancement modules CAM;

[0047] The operation expression for any one-time fusion module is:

[0048] ;

[0049] in, and These are the restored feature map and the quality feature map, respectively, which are the inputs to the FFM module; Y is the output of the FFM module; Concat represents the concatenation operation; represents global average pooling; FC represents a fully connected layer; the operational expression for the module MFEM is:

[0050] ;

[0051] Where x is the input of module MFEM, and so on. and X5 is the output of module MFEM; , , , These represent convolutional layers with kernel sizes of 1×1, 3×3, 5×5, and 7×7, respectively. and These represent max pooling and average pooling along the channel dimension, respectively.

[0052] Optionally, the operational expression of the secondary fusion module is:

[0053] ;

[0054] in, , and EF1 is the input of module SFM, which is provided by three primary fusion modules; EF1 is the output of module SFM. Represents adaptive L2 pooling, used to... and Spatial resolution adjusted to match same; This involves applying Softmax normalization to the feature map in each channel dimension to obtain the spatial weight distribution, and then performing weighted average pooling. These are learnable parameters.

[0055] The beneficial effects of this invention are:

[0056] By combining an image restoration network with a Transformer-based score evaluation network, it can capture both local image details and model global structural information, solving the problem of insufficient feature extraction in traditional CNNs under multiple distortions and scenarios. At the same time, it introduces a few-sample fine-tuning strategy, relying on a small number of target scene samples to complete fine-tuning, reducing annotation costs and enhancing the model's generalization ability, effectively improving the accuracy and robustness of image quality evaluation in complex scenes, and providing reliable data support for downstream tasks. Attached Figure Description

[0057] Figure 1 A flowchart of an image quality assessment method based on a combination of Transformer and few-shot learning provided in this embodiment of the invention;

[0058] Figure 2 A network architecture diagram of an image quality assessment model provided in an embodiment of the present invention;

[0059] Figure 3 This is a network architecture diagram of an image restoration model provided in an embodiment of the present invention;

[0060] Figure 4 This is a schematic diagram of the structure of a primary fusion module provided in an embodiment of the present invention;

[0061] Figure 5 This is a schematic diagram of the structure of a secondary fusion module provided in an embodiment of the present invention;

[0062] Figure 6This is an architecture diagram of an image quality assessment system based on a combination of Transformer and few-shot learning, provided for an embodiment of the present invention. Detailed Implementation

[0063] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0064] This invention provides an image quality assessment method based on a combination of Transformer and few-shot learning. See also... Figure 1 , Figure 1 A flowchart illustrating an image quality assessment method based on a combination of Transformer and few-shot learning, provided as an embodiment of the present invention. The method includes the following steps:

[0065] S101, Obtain the first dataset and use the first dataset to train the image quality evaluation model with the preset architecture to obtain the first evaluation model.

[0066] S102, obtain the second dataset, and fine-tune the first evaluation model using the second dataset to obtain the second evaluation model.

[0067] S103, apply the second evaluation model to the quality evaluation process of the target scene image.

[0068] The first dataset is a large-scale dataset containing real images, distorted images, and quality scores. The second dataset is a self-built dataset containing a small number of target scene image samples and quality scores, where the quality scores reflect the subjective perceived quality of the distorted images in the target scene.

[0069] See Figure 2 , Figure 2 This is a network architecture diagram of an image quality assessment model provided in an embodiment of the present invention. The network architecture of the image quality assessment model includes a first network branch and a second network branch: the first network branch is used to restore the distorted image to a real image and outputs a feature map in the process of distorted image restoration; the second network branch is a score evaluation network based on Transformer, used to extract the quality features of the distorted image and fuse the quality features with the feature map output by the first network branch, and output a quality score through a fully connected network.

[0070] This invention provides an image quality assessment method based on Transformer and few-shot learning. By combining an image restoration network with a Transformer-based score evaluation network, it can capture local image details and model global structural information, solving the problem of insufficient feature extraction in traditional CNNs under multiple distortions and scenarios. At the same time, it introduces a few-shot fine-tuning strategy, relying on a small number of target scene samples to complete fine-tuning, reducing annotation costs and enhancing the model's generalization ability. This effectively improves the accuracy and robustness of image quality assessment in complex scenes, providing reliable data support for downstream tasks.

[0071] In this implementation, the first dataset can be a mix of mainstream IQA datasets such as LIVE, KADID-10k, and KonIQ-10k, selecting corresponding feature data at different training stages. The acquisition process for the second dataset includes: raw image acquisition: obtaining high-quality and low-quality raw images from historical production data to ensure diversity; subjective quality assessment: inviting numerous observers to subjectively rate the processed images; associating the subjective ratings with image annotations; introducing quality degradation: performing degradation processing such as noise addition, blurring, compression, and color adjustment on the high-quality raw images, followed by scoring and annotation.

[0072] In this embodiment, step S101 includes:

[0073] Step 1: Train the pre-defined image restoration model using the first dataset. Loss function: Loss res We employ root mean square error loss and structural similarity loss:

[0074] ;

[0075] Where RMSE is the root mean square error; SSIM is the structural similarity; I res and I gt These are the restored image and the original image, respectively. and These are weighting coefficients, which can be used to... Set to 1, dynamically adjust during training. .

[0076] The image restoration model is a U-Net model, consisting of sequentially connected encoding modules E1, E2, E3, E4, and E5, and decoding modules D1, D2, D3, D4, and D5. For details, see [link to documentation]. Figure 3 , Figure 3 This is a network architecture diagram of an image restoration model provided in an embodiment of the present invention. Each encoding and decoding module contains two 3×3 convolutional layers. Each downsampling step reduces the width and height of the feature map by half.

[0077] Step two: Delete the last two modules D4 and D5 of the image restoration model to obtain the first network branch.

[0078] The decoding modules D1, D2, and D3 output the recovered feature maps Fr1, Fr2, and Fr3. If the input image size is H×W, then the sizes of Fr1, Fr2, and Fr3 are H / 8×W / 8, H / 4×W / 4, and H / 2×W / 2, respectively.

[0079] Step 3: Train the pre-defined score evaluation model using the first dataset. At this point, freeze the parameters of the first network branch and use the output recovered feature map for training. Loss function: Loss score Mean absolute error loss is used:

[0080] ;

[0081] Where MAE is the mean absolute error; S pre and S gt These are the predicted quality score and the actual quality score, respectively.

[0082] like Figure 2 As shown, the score evaluation model includes a feature extraction network, a feature fusion network, a natural scene statistics network, and a score prediction network. Among them:

[0083] The feature extraction network receives distorted images as input and outputs quality feature maps Fq1, Fq2, and Fq3 at multiple scales through multiple Transformer layers.

[0084] The feature fusion network fuses the restored feature maps and quality feature maps at corresponding scales (Fr1 and Fq3, Fr2 and Fq2, Fr3 and Fq1) to obtain the first evaluation feature EF1. Specifically, the feature fusion network includes multiple primary fusion modules (FFM) and one secondary fusion module (SFM). The primary fusion modules fuse the restored feature maps and quality feature maps to obtain a fused feature map; the secondary fusion module performs secondary fusion of the fused feature maps from multiple scales to obtain the first evaluation feature.

[0085] The Natural Scene Statistical Network performs local normalization and generalized Gaussian distribution fitting on the input distorted image to obtain natural statistical features. These features are then subjected to deep learning through a fully connected layer to obtain the second evaluation feature, EF2. Specifically, the image is processed using the BRISQUE method to obtain the natural statistical features.

[0086] The score prediction network uses a fully connected network to map the first and second evaluation features into quality scores.

[0087] This embodiment realizes a multi-source information fusion image quality assessment method by combining U-Net image restoration features, Transformer-based multi-scale quality features, and natural scene statistical features. This method not only utilizes restoration features to reflect the structural differences between distorted and real images, but also captures global quality features through Transformer and introduces natural statistical regularities to enhance the stability and interpretability of the assessment. The overall scheme can significantly improve assessment accuracy, generalization ability, and adaptability to complex distortion patterns.

[0088] In this embodiment, multiple primary fusion modules have the same structure. See [link / reference] Figure 4 , Figure 4 This is a schematic diagram of the structure of a primary fusion module provided in an embodiment of the present invention. Figure 4 (a) shows the overall structure of the primary fusion module FFM, which includes two structurally identical multi-scale feature extraction modules MFEM and attention enhancement modules CAM. Figure 4 (b) shows the specific structure of the multi-scale feature extraction module.

[0089] The operation expression for any one-time fusion module is:

[0090] ;

[0091] in, and These are the restored feature map and the quality feature map, respectively, which are the inputs to the FFM module; Y is the output of the FFM module; Concat represents the concatenation operation; represents global average pooling; FC represents a fully connected layer; the operational expression for the module MFEM is:

[0092] ;

[0093] Where x is the input of module MFEM, and so on. and X5 is the output of module MFEM; , , , These represent convolutional layers with kernel sizes of 1×1, 3×3, 5×5, and 7×7, respectively. and These represent max pooling and average pooling along the channel dimension, respectively.

[0094] This fusion module extracts rich structural and semantic information from restored features and quality features at different scales, and achieves efficient fusion through splicing and attention mechanisms. It highlights key regions and suppresses redundant information, thereby significantly enhancing feature representation capabilities and improving the accuracy, stability, and adaptability to various distortion types in image quality assessment.

[0095] In the implementation of this embodiment, see [link to relevant documentation]. Figure 5 , Figure 5 This is a schematic diagram of a secondary fusion module provided in an embodiment of the present invention. The operational expression of the secondary fusion module is:

[0096] ;

[0097] in, , and EF1 is the input of module SFM, which is provided by three primary fusion modules; EF1 is the output of module SFM. Represents adaptive L2 pooling, used to... and Spatial resolution adjusted to match same; This involves applying Softmax normalization to the feature map in each channel dimension to obtain the spatial weight distribution, and then performing weighted average pooling. These are learnable parameters.

[0098] In this embodiment, if the spatial size of feature map x is H*W, then its i-th channel The result of weighted average pooling is: We perform weighted average pooling on each channel to obtain the feature vector.

[0099] This fusion module redistributes channel weights on the stitched feature maps using channel attention to obtain optimized feature maps. Weighted spatial pooling is employed to enhance the emphasis on larger feature values, making the model more focused on salient regions in the image. Traditional global average pooling is used to calculate global statistical information of the feature maps, capturing the overall contextual information of the image. The coefficients are adjusted accordingly. By controlling the influence weights of spatial information, the final fusion yields feature EF1. This fusion method preserves global image information while enhancing features of local salient regions, making the model's image quality assessment more accurate.

[0100] This invention provides an image quality assessment system based on a combination of Transformer and few-shot learning. See also... Figure 6 , Figure 6This is an architecture diagram of an image quality assessment system based on a combination of Transformer and few-shot learning, provided as an embodiment of the present invention. The system includes:

[0101] The pre-training module is used to acquire the first dataset and train the image quality assessment model with a pre-defined architecture using the first dataset to obtain the first assessment model.

[0102] The fine-tuning module is used to obtain a second dataset and fine-tune the first evaluation model using the second dataset to obtain a second evaluation model.

[0103] The scheduling module is used to apply the second evaluation model as the final image quality evaluation model to the quality evaluation process of the target scene image.

[0104] The image quality assessment system based on the combination of Transformer and few-shot learning provided by this invention combines an image restoration network with a Transformer-based score evaluation network. This system can capture local image details and model global structural information, solving the problem of insufficient feature extraction in traditional CNNs under multiple distortions and scenarios. At the same time, it introduces a few-shot fine-tuning strategy, relying on a small number of target scene samples to complete fine-tuning, reducing annotation costs and enhancing the model's generalization ability. This effectively improves the accuracy and robustness of image quality assessment in complex scenes, providing reliable data support for downstream tasks.

[0105] In this embodiment, the pre-training module includes:

[0106] The reconstruction model training module is used to train a preset image restoration model using the first dataset.

[0107] The branch determination module is used to delete the last two modules of the image restoration model to obtain the first network branch.

[0108] The evaluation model training module is used to train a pre-defined score evaluation model using the first dataset.

[0109] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall fall within the scope of the present invention.

Claims

1. An image quality assessment method based on a combination of Transformer and few-shot learning, characterized in that, The method includes: A first dataset is obtained, and an image quality assessment model with a preset architecture is trained using the first dataset to obtain a first assessment model. The first dataset is a large-scale dataset containing real images, distorted images, and quality scores. The network architecture of the image quality assessment model includes a first network branch and a second network branch. The first network branch is used to restore distorted images to real images and output feature maps during the distorted image restoration process. The second network branch is a score evaluation network built based on Transformer, used to extract quality features of distorted images and fuse the quality features with the feature maps output by the first network branch, and output a quality score through a fully connected network. Obtain a second dataset and fine-tune the first evaluation model using the second dataset to obtain a second evaluation model; the second dataset is a self-built dataset containing a small number of target scene image samples and quality scores; The second evaluation model is applied to the quality evaluation process of the target scene image; The process of training the image quality assessment model with a preset architecture using the first dataset to obtain the first assessment model includes: Step 1: Train the preset image restoration model using the first dataset; the image restoration model is built based on the U-Net model and includes encoder modules E1, E2, E3, E4, E5 and decoder modules D1, D2, D3, D4, D5 connected in sequence. Step 2: Delete the last two modules D4 and D5 of the image restoration model to obtain the first network branch; denote the outputs of the decoder modules D1, D2, and D3 as the restored feature maps; Step 3: Train the preset score evaluation model using the first dataset; The score evaluation model includes a feature extraction network, a feature fusion network, a natural scene statistics network, and a score prediction network; among which... The feature extraction network receives the distorted image as input and outputs feature maps of multiple scales through multiple Transformer layers, which are denoted as quality feature maps. The feature fusion network fuses the recovery feature map and the quality feature map at the corresponding scale to obtain the first evaluation feature; The natural scene statistical network performs local normalization and generalized Gaussian distribution fitting on the input distorted image to obtain natural statistical features, and then performs deep learning on the natural statistical features through a fully connected layer to obtain the second evaluation features. The score prediction network uses a fully connected network to map the first evaluation feature and the second evaluation feature into a quality score; The feature fusion network includes multiple primary fusion modules (FFM) and one secondary fusion module (SFM); the multiple primary fusion modules have the same structure; wherein: Any one of the fusion modules is used to fuse the restored feature map and the quality feature map to obtain the fused feature map; The secondary fusion module is used to perform secondary fusion of fusion feature maps at multiple scales to obtain the first evaluation feature.

2. The image quality assessment method based on a combination of Transformer and few-shot learning according to claim 1, characterized in that, Any single-stage fusion module FFM consists of two structurally identical multi-scale feature extraction modules MFEM and attention enhancement modules CAM; The operation expression for any one-time fusion module is: ； in, and These are the restored feature map and the quality feature map, respectively, which are the inputs to the FFM module; Y is the output of the FFM module; Concat represents the concatenation operation. represents global average pooling; FC represents a fully connected layer; the operation expression for the module MFEM is: ； Where x is the input of module MFEM, and so on. and X5 is the output of module MFEM; , , , These represent convolutional layers with kernel sizes of 1×1, 3×3, 5×5, and 7×7, respectively. and These represent max pooling and average pooling along the channel dimension, respectively.

3. The image quality assessment method based on a combination of Transformer and few-shot learning according to claim 2, characterized in that, The operational expression for the secondary fusion module is: ； in, , and EF1 is the input of module SFM, which is provided by three primary fusion modules; EF1 is the output of module SFM. Represents adaptive L2 pooling, used to... and Spatial resolution adjusted to match same; This involves applying Softmax normalization to the feature map in each channel dimension to obtain the spatial weight distribution, and then performing weighted average pooling. These are learnable parameters.

4. An image quality assessment system based on a combination of Transformer and few-shot learning, characterized in that, The system includes: A pre-training module is used to acquire a first dataset and train a pre-defined image quality assessment model using the first dataset to obtain a first assessment model. The first dataset is a large-scale dataset containing real images, distorted images, and quality scores. The network architecture of the image quality assessment model includes a first network branch and a second network branch. The first network branch is used to restore distorted images to real images and output feature maps during the distorted image restoration process. The second network branch is a score evaluation network built based on Transformer, used to extract quality features of distorted images and fuse the quality features with the feature maps output by the first network branch, outputting a quality score through a fully connected network. The fine-tuning module is used to acquire a second dataset and fine-tune the first evaluation model using the second dataset to obtain a second evaluation model; the second dataset is a self-built dataset containing a small number of target scene image samples and quality scores. The scheduling module is used to apply the second evaluation model to the quality evaluation process of the target scene image; The pre-training module includes: The reconstruction model training module is used to train a preset image restoration model using the first dataset; the image restoration model is built based on the U-Net model and includes encoder modules E1, E2, E3, E4, E5 and decoder modules D1, D2, D3, D4, D5 connected in sequence. The branch determination module is used to delete the last two modules D4 and D5 of the image restoration model to obtain the first network branch; and to denote the outputs of the decoder modules D1, D2 and D3 as the restored feature maps. The evaluation model training module is used to train a preset score evaluation model using the first dataset; The score evaluation model includes a feature extraction network, a feature fusion network, a natural scene statistics network, and a score prediction network; among which... The feature extraction network receives the distorted image as input and outputs feature maps of multiple scales through multiple Transformer layers, which are denoted as quality feature maps. The feature fusion network fuses the recovery feature map and the quality feature map at the corresponding scale to obtain the first evaluation feature; The natural scene statistical network performs local normalization and generalized Gaussian distribution fitting on the input distorted image to obtain natural statistical features, and then performs deep learning on the natural statistical features through a fully connected layer to obtain the second evaluation features. The score prediction network uses a fully connected network to map the first evaluation feature and the second evaluation feature into a quality score; The feature fusion network includes multiple primary fusion modules (FFM) and one secondary fusion module (SFM); the multiple primary fusion modules have the same structure; wherein: Any one of the fusion modules is used to fuse the restored feature map and the quality feature map to obtain the fused feature map; The secondary fusion module is used to perform secondary fusion of fusion feature maps at multiple scales to obtain the first evaluation feature.

5. The image quality assessment system based on a combination of Transformer and few-shot learning according to claim 4, characterized in that, Any single-stage fusion module FFM consists of two structurally identical multi-scale feature extraction modules MFEM and attention enhancement modules CAM; The operation expression for any one-time fusion module is: ； in, and These are the restored feature map and the quality feature map, respectively, which are the inputs to the FFM module; Y is the output of the FFM module; Concat represents the concatenation operation. represents global average pooling; FC represents a fully connected layer; the operation expression for the module MFEM is: ； Where x is the input of module MFEM, and so on. and X5 is the output of module MFEM; , , , These represent convolutional layers with kernel sizes of 1×1, 3×3, 5×5, and 7×7, respectively. and These represent max pooling and average pooling along the channel dimension, respectively.

6. The image quality assessment system based on a combination of Transformer and few-shot learning according to claim 5, characterized in that, The operational expression for the secondary fusion module is: ； in, , and EF1 is the input of module SFM, which is provided by three primary fusion modules; EF1 is the output of module SFM. Represents adaptive L2 pooling, used to... and Spatial resolution adjusted to match same; This involves applying Softmax normalization to the feature map in each channel dimension to obtain the spatial weight distribution, and then performing weighted average pooling. These are learnable parameters.

Images