Construction and application method of image quality evaluation model, model, equipment and medium

Abstract

The invention relates to the field of computer vision, and the method can dynamically adjust and selectively emphasize or suppress the specific features in an image through the gating mechanism operation of the content perception features and distortion perception features of a distorted image. Therefore, the model can flexibly respond to different types of distortion information in the image and the influence of the distortion information on the image content information, so that the complex nonlinear interaction between the two is captured. The mechanism not only enhances the sensitivity of the model to local and global distortion, but also ensures that the subsequent feature fusion operation on distortion features and content features is closer to the real visual quality feeling, thereby improving the accuracy and generalization of the image quality evaluation model in image quality evaluation. Meanwhile, according to the model construction method provided by the invention, the image quality evaluation model with more competitive power and even more excellent performance can be trained by using a small amount of training data, so that the dependence of the image quality evaluation model on the labeled training sample is effectively reduced.

Description

Technical Field

[0001] The present application relates to the field of computer vision, and in particular to a method, model, device, and medium for constructing and applying an image quality assessment model. Background Art

[0002] Image quality assessment aims to develop objective quality indicators that are consistent with human visual perception. Accurate and comprehensive image quality assessment technology is crucial to many fields, including digital photography, video surveillance, video compression, streaming media services, etc.

[0003] Traditional no-reference image quality assessment methods are mainly used to deal with synthetic distortion. These methods use manually designed feature descriptors to extract image features and adopt simple regression models for quality prediction. However, manually designed feature descriptors can only represent a small part of the distortion types of the image and cannot capture the content information of the image. Therefore, traditional no-reference image quality assessment methods often have difficulty in dealing with the complexity of distortion in natural scenes, and therefore cannot accurately assess the quality of images. Recently, advances in deep learning have enabled no-reference image quality assessment methods to evolve from methods based on manually designed features to methods based on deep learning (i.e., image quality assessment models are constructed based on deep learning models), and have achieved significant improvements in quality prediction accuracy. However, image quality assessment models constructed by no-reference image quality assessment methods combined with deep learning are more inclined to extract features that are sensitive to global content information in the image, and the extracted features lack sensitivity to local distortion of the image. Therefore, image quality assessment models constructed by image quality assessment methods combined with deep learning still cannot comprehensively and accurately assess the quality of images. The latest image quality assessment methods mainly use image quality assessment models pre-trained on large-scale datasets for feature extraction, and then fine-tune the backbone and quality predictor on the quality assessment dataset. However, the representations learned by these pre-trained image quality assessment models tend to emphasize the content information of the image and perform poorly in the image quality assessment results. There are also some other image quality assessment methods that construct image quality assessment models that mainly use supervised learning and self-supervised learning to learn the distortion manifold of the image, but ignore the content information of the image.

[0004] The inventors have found through research that the content and distortion information in an image are often closely intertwined, and different regions usually present different visual qualities. Considering that image quality is a comprehensive judgment derived from the interaction between distortion information estimation and content information recognition, understanding these interactions is crucial for image quality assessment. The image quality assessment models constructed in existing image quality assessment methods either cannot obtain the content information of the image or lack sensitivity to local distortion, and fail to effectively process the interaction between the content and distortion of the image. In addition, the existing reference-free image quality assessment methods have constructed relevant image quality assessment models for certain aspects of factors such as content and distortion, but the constructed image quality assessment model simply fuses the content information and distortion information of the image, and fails to explore the complex interaction between content and distortion in the image (i.e., the degree of influence of different distortion types on the image content is different), resulting in low accuracy and generalization of image quality assessment by the image quality assessment model.

[0005] Therefore, there is an urgent need to build an image quality assessment model that can realize complex interaction between image content information and distortion information. Summary of the invention

[0006] The present application provides a method, device, and medium for constructing and applying an image quality assessment model to solve the problem that the existing image quality assessment model has low accuracy and generalization in image quality assessment.

[0007] A first aspect of the present application provides a method for constructing an image quality assessment model, where the image quality assessment model is used to assess the quality of an image. The method includes:

[0008] Acquire a distorted image set, wherein the distorted image set includes a plurality of distorted images;

[0009] An initial image quality assessment model is constructed, wherein the initial image quality assessment model includes:

[0010] a content-aware encoder module, configured to perform a content-aware feature extraction operation on each distorted image to obtain a content-aware feature of each distorted image;

[0011] A distortion-aware encoder module, configured to perform a distortion-aware feature extraction operation on each distorted image to obtain a distortion-aware feature of each distorted image;

[0012] a progressive perceptual interaction module, for respectively performing convolution operations on the content-perceptual features and the distortion-perceptual features of each distorted image, respectively performing gating mechanism operations on the content-perceptual features and the distortion-perceptual features of each distorted image, and obtaining content-perceptual weighted features of each distorted image based on the results of performing convolution operations and gating mechanism operations on the content-perceptual features of each distorted image, and obtaining distortion-perceptual weighted features of each distorted image based on the results of performing convolution operations and gating mechanism operations on the distortion-perceptual features of each distorted image, and performing feature fusion operations on the content-perceptual weighted features and the distortion-perceptual weighted features of each distorted image to obtain interaction features of each distorted image;

[0013] A quality prediction module, used for obtaining a predicted quality evaluation result of each distorted image based on the interactive features of each distorted image;

[0014] The initial image quality assessment model is iteratively trained multiple times using the distorted image set until convergence, and during each iterative training process, the parameters of the image quality assessment model are updated according to a preset loss function.

[0015] In some embodiments of the present application, the content-aware features of each distorted image include content-aware features of multiple levels of each distorted image, the distortion-aware features of each distorted image include distortion-aware features of multiple levels of each distorted image, and the initial image quality assessment model includes:

[0016] Multiple progressive perceptual interaction modules are used to obtain interactive features of multiple levels of each distorted image, wherein each progressive perceptual interaction module is used to perform convolution operations on content-perceptual features and distortion-perceptual features at the same level of each distorted image, respectively, and perform gating mechanism operations on content-perceptual features and distortion-perceptual features at the same level of each distorted image, respectively, and obtain content-perceptual weighted features at the same level of each distorted image based on the results of convolution operations and gating mechanism operations on content-perceptual features at the same level of each distorted image, and obtain distortion-perceptual weighted features at the same level of each distorted image based on the results of convolution operations and gating mechanism operations on distortion-perceptual features at the same level of each distorted image, and perform feature fusion operations on content-perceptual weighted features and distortion-perceptual weighted features at the same level of each distorted image to obtain interactive features at the same level of each distorted image.

[0017] In some embodiments of the present application, each distorted image contains distortion perception features of multiple channels in each level, each distorted image contains distortion perception features of multiple channels in each level, and each progressive perception interaction module includes:

[0018] a feature alignment submodule, configured to perform a channel dimensionality reduction operation on the content-aware features of multiple channels in the same level of each distorted image to obtain content-aware features of a first preset number of channels in the same level of each distorted image, and to perform a channel dimensionality reduction operation on the distortion-aware features of multiple channels in the same level of each distorted image to obtain distortion-aware features of a first preset number of channels in the same level of each distorted image;

[0019] The feature interaction submodule is used to perform convolution operations on the content-aware features and distortion-aware features of all channels in the same level of each distorted image obtained by the feature alignment submodule, and to perform gating mechanism operations on the content-aware features and distortion-aware features of all channels in the same level of each distorted image, and to obtain content-aware weighted features at the same level of each distorted image based on the results of the convolution operations and gating mechanism operations on the content-aware features of all channels in the same level of each distorted image, and to obtain distortion-aware weighted features at the same level of each distorted image based on the results of the convolution operations and gating mechanism operations on the distortion-aware features of all channels in the same level of each distorted image, and to perform feature fusion operations on the content-aware weighted features and distortion-aware weighted features at the same level of each distorted image to obtain interaction features at the same level of each distorted image.

[0020] In some embodiments of the present application, the feature fusion operation includes a feature coarse fusion operation and a feature fine fusion operation, and the feature interaction submodule further includes:

[0021] A convolution unit, used to perform convolution operations on the content-aware features and distortion-aware features of all channels in the same level of each distorted image obtained by the feature alignment submodule;

[0022] A gating unit, for performing a gating mechanism operation on the content-aware features and the distortion-aware features of all channels in the same level of each distorted image respectively;

[0023] a feature weighting unit, for obtaining content-aware weighted features at the same level of each distorted image based on a result of performing a convolution operation and a gating mechanism operation on content-aware features of all channels at the same level of each distorted image, and for obtaining distortion-aware weighted features at the same level of each distorted image based on a result of performing a convolution operation and a gating mechanism operation on distortion-aware features of all channels at the same level of each distorted image;

[0024] A feature coarse interaction unit, used for performing a feature coarse fusion operation on the content-aware weighted features and the distortion-aware weighted features at the same level of each distorted image, so as to obtain a coarse interaction feature at the same level of each distorted image;

[0025] A feature fine interaction unit, used for performing a feature fine fusion operation on the content-aware weighted features and the distortion-aware weighted features at the same level of each distorted image, so as to obtain a fine interaction feature at the same level of each distorted image;

[0026] The feature deep interaction unit is used to obtain the interaction features at the same level of each distorted image based on the coarse interaction features and fine interaction features at the same level of each distorted image.

[0027] In some embodiments of the present application, the content-aware weighted features of multiple channels in the same level of each distorted image are evenly divided into two parts according to the number of channels, and the feature fine interaction unit includes:

[0028] a first depthwise separable convolution subunit, configured to perform a first depthwise separable convolution operation on the content-aware weighted features of a portion of the channels at the same level of each distorted image, so as to obtain content-aware first intermediate features of a portion of the channels at the same level of each distorted image;

[0029] A second depthwise separable convolution subunit is used to perform a second depthwise separable convolution operation on the distortion-perceived weighted features at the same level of each distorted image to obtain the distortion-perceived offset features at the same level of each distorted image;

[0030] a depthwise separable deformable convolution subunit, configured to perform a depthwise separable deformable convolution operation on the content-aware weighted features of another portion of channels in the same level of each distorted image based on the distortion-aware offset features of another portion of channels in the same level of each distorted image, so as to obtain content-aware second intermediate features of another portion of channels in the same level of each distorted image;

[0031] The channel stitching and shuffling subunit is used to perform a channel stitching operation on the content-aware first intermediate features of a part of the channels at the same level of each distorted image and the content-aware second intermediate features of another part of the channels, and to perform a channel shuffling operation on the result of the channel stitching operation to obtain the detailed interactive features at the same level of each distorted image.

[0032] In some embodiments of the present application, the quality prediction module is configured to:

[0033] Based on the interactive features of multiple levels of each distorted image, the prediction quality evaluation result of each distorted image is obtained.

[0034] A second aspect of the present application provides an image quality assessment model, the image quality assessment model comprising:

[0035] A content-aware encoder module, used for performing a content-aware feature extraction operation on the acquired distorted image to obtain content-aware features of the distorted image;

[0036] A distortion perception encoder module, used for performing a distortion perception feature extraction operation on the distorted image to obtain the distortion perception features of the distorted image;

[0037] a progressive perceptual interaction module, for respectively performing convolution operations on content-perceptual features and distortion-perceptual features of the distorted image, respectively performing gating mechanism operations on the content-perceptual features and distortion-perceptual features of the distorted image, and obtaining content-perceptual weighted features of the distorted image based on the results of performing convolution operations and gating mechanism operations on the content-perceptual features of the distorted image, and obtaining distortion-perceptual weighted features of the distorted image based on the results of performing convolution operations and gating mechanism operations on the distortion-perceptual features of the distorted image, and performing feature fusion operations on the content-perceptual weighted features and distortion-perceptual weighted features of the distorted image to obtain interactive features of the distorted image;

[0038] The quality prediction module is used to obtain a predicted quality evaluation result of the distorted image based on the interactive features of the distorted image.

[0039] A third aspect of the present application provides an application method of an image quality assessment model, the method comprising:

[0040] Obtaining a distorted image;

[0041] The image quality assessment model described in the above embodiment is used to perform quality assessment on the distorted image.

[0042] The fourth aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the first and third aspects of the above-mentioned embodiments when executing the computer program.

[0043] The fifth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, and when the program instructions are executed by a processor, the method described in any one of the first aspect and the third aspect of the above embodiments is implemented.

[0044] This application has the following beneficial effects:

[0045] The construction scheme of the image quality assessment model provided by the present application first obtains the content-aware features and distortion-aware features of the distorted image through the content-aware encoder module and the distortion-aware encoder module; then, the content-aware features and the distortion-aware features of the distorted image are respectively convolved through the progressive perceptual interaction module, and the content-aware features and the distortion-aware features of the distorted image are respectively gated, and based on the results of the convolution operation and the gated operation on the content-aware features of the distorted image, the content-aware weighted features of the distorted image are obtained, and based on the results of the convolution operation and the gated operation on the distortion-aware features of the distorted image, the distortion-aware weighted features of the distorted image are obtained, and feature fusion operations are performed on the content-aware weighted features and the distortion-aware weighted features of the distorted image to obtain the interactive features of the distorted image; finally, the quality prediction module is used to obtain the predicted quality assessment result of the distorted image based on the interactive features of the distorted image. The present application can dynamically adjust and selectively emphasize or suppress specific features in the image by operating the gating mechanism on the content-aware features and distortion-aware features of the distorted image, so that the model can flexibly respond to different types of distortion information in the image and its impact on the image content information, thereby capturing the complex nonlinear interaction between the two. This mechanism not only enhances the model's sensitivity to local and global distortions, but also ensures that the subsequent feature fusion operations on distortion features and content features are closer to the real visual quality perception, thereby improving the accuracy and generalization of image quality assessment by the image quality assessment model. At the same time, the model can show good data-efficient learning capabilities in different distortion scenarios (that is, the model can flexibly respond to different types of distortion information in the image and its impact on the image content information), so that the image quality model construction method of the present application can use only a small amount of training data to train a more competitive or even superior image quality assessment model, thereby effectively reducing the image quality assessment model's dependence on labeled training samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0047] Figure 1 It is a flowchart of an embodiment of a method for constructing an image quality assessment model provided by the present application;

[0048] Figure 2 It is a schematic diagram of the framework of the first embodiment of the image quality assessment model provided by the present application;

[0049] Figure 3 is a schematic diagram of the framework of the second embodiment of the image quality assessment model provided by the present application;

[0050] Figure 4 is a schematic diagram of the framework of the third embodiment of the image quality assessment model provided by the present application;

[0051] Figure 5 is a schematic diagram of the framework of the fourth embodiment of the image quality assessment model provided by the present application;

[0052] Figure 6 is a schematic diagram of the framework of the fifth embodiment of the image quality assessment model provided by the present application;

[0053] Figure 7 is a schematic diagram of the framework of the sixth embodiment of the image quality assessment model provided by the present application;

[0054] Figure 8 is a schematic diagram of the framework of the seventh embodiment of the image quality assessment model provided by the present application;

[0055] Fig. 9 is a schematic diagram of the framework of the eighth embodiment of the image quality assessment model provided by the present application;

[0056] Fig.10 is a schematic diagram of the framework of the ninth embodiment of the image quality assessment model provided by the present application;

[0057] Fig.11 is a schematic diagram of the framework of the tenth embodiment of the image quality assessment model provided by the present application;

[0058] Fig.12 It is a schematic diagram of the framework of another embodiment of the method for building an image quality assessment model provided by the present application;

[0059] Fig.13 It is a schematic diagram of experimental results of the image quality assessment model construction scheme provided by this application in terms of assessment accuracy;

[0060] Fig.14 It is a schematic diagram of experimental results of the image quality assessment model construction scheme provided by this application in terms of data efficiency;

[0061] Fig.15 It is a schematic diagram of a framework of an embodiment of an electronic device provided by the present application;

[0062] Fig.16 It is a schematic diagram of a framework of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0063] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0064] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0065] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0066] As described in the background technology, the image quality assessment models constructed in the existing image quality assessment methods either cannot obtain the content information of the image or lack sensitivity to local distortion, and fail to effectively process the interaction between the content and distortion of the image. In addition, the existing no-reference image quality assessment methods have constructed relevant image quality assessment models for certain aspects of factors such as content and distortion, but the constructed image quality assessment model simply fuses the content information and distortion information of the image, and fails to explore the complex interaction between the content and distortion in the image (that is, different types of distortion have different degrees of influence on the image content), resulting in low accuracy and generalization of image quality assessment by the image quality assessment model.

[0067] In order to solve the above problems, the present application proposes a new image quality assessment model construction scheme. The image quality assessment model constructed by the scheme can dynamically adjust and selectively emphasize or suppress specific features in the image by operating the gating mechanism on the content-aware features and distortion-aware features of the distorted image, so that the model can flexibly respond to different types of distortion information in the image and its impact on the image content information, thereby capturing the complex nonlinear interaction between the two. This mechanism not only enhances the sensitivity of the model to local and global distortions, but also ensures that the subsequent feature fusion operation of the distortion features and content features is closer to the real visual quality perception, thereby improving the accuracy and generalization of the image quality assessment model for image quality assessment. At the same time, the model can show good data-efficient learning ability in different distortion scenarios (that is, the model can flexibly respond to different types of distortion information in the image and its impact on the image content information), so that the image quality model construction method of the present application can use only a small amount of training data to train a more competitive or even superior image quality assessment model, thereby effectively reducing the image quality assessment model's dependence on labeled training samples.

[0068] Before a detailed introduction to the embodiments of the present application, a preliminary introduction to some basic concepts and related contents involved in the present application is first given to facilitate a more intuitive understanding of the technical solutions of the embodiments of the present application.

[0069] The No-Reference Image Quality Assessment (NR-IQA) method refers to a technology that quantitatively evaluates the quality of an image without the original undistorted image as a comparison benchmark.

[0070] Image distortion can manifest as a variety of visual phenomena that generally reduce the quality and perception of image content. The following are some common manifestations of image distortion:

[0071] 1. Blurring: Details in the image become unclear and edges are no longer sharp. This may be caused by camera shake, object motion, or focus issues.

[0072] 2. Noise: It appears as random point-like interference in the image, such as granular spots under high ISO settings. Common types include salt and pepper noise, Gaussian noise, etc.

[0073] 3. Geometric Distortion: The shape of the image is deformed, such as barrel distortion (the edges of the image expand outward) and pincushion distortion (the edges of the image shrink inward), which is usually caused by lens characteristics.

[0074] 4. Chromatic Aberration: Color fringes appear at high-contrast edges due to the failure of light of different wavelengths to focus on the same plane.

[0075] 5. Moire Pattern: An interference pattern produced when periodic structures in the image interact with the sensor pixel array, appearing as wavy or spiral lines.

[0076] 6. Compression Artifacts: For example, the block effect in the JPEG format, that is, the image is divided into obvious square areas, and the color or brightness changes in each area are unnatural.

[0077] 7. Exposure Issues: including overexposure (the image is too bright and details are lost) and underexposure (the image is too dark and details are difficult to discern), usually caused by improper lighting conditions or camera settings.

[0078] 8. Color Cast: The entire image presents some abnormal color tone tendency, such as blue or red, which may be due to incorrect white balance settings or the influence of the color temperature of the light source.

[0079] 9. Quantization Error: During the digitization process, some information may be lost when continuous tones are represented by discrete values, especially at low bit depths, and banding may be seen.

[0080] 10. Ghosting: The phenomenon of repeated or double images in certain parts of the image, usually caused by multiple reflections or a shutter speed that is not fast enough.

[0081] 11. Diffraction Limitation: When shooting with a very small aperture, the diffraction of light will cause the overall image to become blurred, especially on high-resolution sensors.

[0082] 12. Perspective Distortion: Due to the shooting angle, the proportion of objects in the image is distorted, for example, a building looks tilted.

[0083] Understanding the distortion in the image helps to select appropriate technical means for image restoration or preventive measures to improve the quality of the final image.

[0084] The present application is described in detail below with reference to the accompanying drawings and specific embodiments.

[0085] According to one embodiment of the present application, Figure 1 As shown, the present application provides a method for constructing an image quality assessment model, the method comprising: S1, obtaining a distorted image set, wherein the distorted image set includes a plurality of distorted images; S2, constructing an initial image quality assessment model, wherein, as Figure 2As shown, the initial image quality assessment model includes: a content-aware encoder module, which is used to perform a content-aware feature extraction operation on each distorted image to obtain a content-aware feature of each distorted image; a distortion-aware encoder module, which is used to perform a distortion-aware feature extraction operation on each distorted image to obtain a distortion-aware feature of each distorted image; a progressive perceptual interaction module, which is used to perform a convolution operation on the content-aware feature and the distortion-aware feature of each distorted image, respectively, and perform a gating mechanism operation on the content-aware feature and the distortion-aware feature of each distorted image, respectively, and obtain each based on the results of the convolution operation and the gating mechanism operation on the content-aware feature of each distorted image. Content-aware weighted features of distorted images, and distortion-aware weighted features of each distorted image obtained by convolution and gating mechanism operations on the distortion-aware features of each distorted image, and feature fusion operations on the content-aware weighted features and distortion-aware weighted features of each distorted image to obtain interactive features of each distorted image; a quality prediction module, used to obtain a predicted quality assessment result of each distorted image based on the interactive features of each distorted image; S3, using the distorted image set to iteratively train the initial image quality assessment model for multiple times until convergence, and in each iterative training process, updating the parameters of the image quality assessment model according to a preset loss function.

[0086] It can be seen from the above embodiments that, compared with the existing image quality assessment model, the image quality assessment model constructed by the construction scheme of the above embodiments of the present application can explore the complex interaction between content and distortion in the image, thereby improving the accuracy and generalization of the image quality assessment model for image quality assessment.

[0087] Each module of the image quality assessment model is described below.

[0088] 1. Content-Aware Encoder Module

[0089] The content-aware encoder module is used to perform a content-aware feature extraction operation on each distorted image to obtain content-aware features of each distorted image.

[0090] As can be seen from the above embodiments, the content-aware features of an image can reduce the impact of image content changes on the image quality assessment model. These features need to have a higher sensitivity to the content so as to accurately understand the correspondence between the image content and its underlying semantics.

[0091] Among them, according to one embodiment of the present application, a pre-trained first deep learning model is configured in the content-aware encoder module, and the first deep learning model is used to perform content-aware feature extraction operations on each distorted image to obtain content-aware features of each distorted image, and the first deep learning model is designed using ResNet50 as the backbone network. In addition, in order to meet the challenges brought by the diversity of image content in the real world, the present application uses the ImageNet dataset to pre-train ResNet50 to enhance the content perception capability of the content-aware encoder module. It should be noted that with the development of deep learning models, the pre-trained first deep learning model configured in the content-aware encoder module may not be limited to being designed with ResNet50 as the backbone network, and may be adaptively designed with other network structures with better performance, which is not specifically limited here.

[0092] It can be seen from the above embodiments that using the pre-trained ResNet50 as the backbone network of the content-aware encoder module can make full use of its powerful feature extraction capabilities and transfer learning advantages, which not only speeds up the model development and reduces the demand for computing resources, but also significantly improves the performance and generalization ability of the model on the target task, effectively reduces the risk of overfitting, and can be flexibly adjusted according to specific application scenarios to ensure the efficiency and accuracy of the model when processing complex tasks.

[0093] The inventors have found through research that based on the perception process of the human visual system, visual perception is a hierarchical structure, that is, the input visual image is processed hierarchically from simple low-level information to complex high-level information. Inspired by this, the present application uses the content-aware encoder module to extract multi-level representations of different scales to capture content-aware information of different scales, while laying the foundation for the feature interaction process in the subsequent stages. Among them, the image representations at different levels are representations of the features of the image obtained using different convolutional layers in the neural network.

[0094] According to an embodiment of the present application, the content-aware features of each distorted image include content-aware features of multiple levels of each distorted image.

[0095] In order to better understand the above embodiments of the present application, the inventor takes 5 levels of image representation to capture image content perception information of different scales as an example. Therefore, after the content-aware encoder module performs content-aware feature extraction on the image, the multi-level feature extraction process is expressed as follows:

[0096]

[0097] in, represents the content-aware encoder module, I drepresents the dth distorted image in the distorted image set, Represents the content-aware features of the distorted image at level i.

[0098] It can be seen from the above embodiments that using image features at different levels for quality assessment can more comprehensively and accurately capture the content information of the image from the global structure to the local details, improve the robustness, generalization ability and consistency of the assessment, and promote the interpretability of the model and support for complex tasks.

[0099] 2. Distortion-aware encoder module

[0100] The distortion-aware encoder module is used to perform a distortion-aware feature extraction operation on each distorted image to obtain a distortion-aware feature of each distorted image.

[0101] Among them, according to one embodiment of the present application, a pre-trained second deep learning model is configured in the distortion-aware encoder module, and the second deep learning model is used to perform a distortion-aware feature extraction operation on each distorted image to obtain the distortion-aware features of each distorted image, and the second deep learning model is designed using ResNet50 as the backbone network. In order to effectively learn distortion-aware representation, the present application introduces another ResNet50 pre-trained using contrast loss as the backbone network. By maximizing the representation similarity between image blocks that exhibit the same type of degradation but different content during pre-training, the encoder can identify various types and degrees of image distortion. It should be noted that with the development of deep learning models, the pre-trained second deep learning model configured in the distortion-aware encoder module may not be limited to being designed with ResNet50 as the backbone network, and may be adaptively designed with other network structures with better performance, which is not specifically limited here.

[0102] It can be seen from the above embodiments that the use of pre-trained ResNet50 as the backbone network of the distortion-aware encoder module can make full use of its powerful feature extraction capabilities and transfer learning advantages, which not only speeds up the model development and reduces the demand for computing resources, but also significantly improves the performance and generalization ability of the model on the target task, effectively reduces the risk of overfitting, and can be flexibly adjusted according to specific application scenarios to ensure the efficiency and accuracy of the model when processing complex tasks.

[0103] According to an embodiment of the present application, the distortion perception feature of each distorted image includes distortion perception features of multiple levels of each distorted image.

[0104] Similarly, in order to better understand the above embodiments of the present application, the inventor takes 5 levels of image representation to capture image distortion perception information of different scales as an example. Therefore, after the distortion perception encoder module performs a distortion perception feature extraction operation on the image, the multi-level feature extraction process obtained is expressed as follows:

[0105]

[0106] in, represents the distortion-aware encoder module, Represents the distortion perception features of the i-th level of the distorted image.

[0107] It can be seen from the above embodiments that using image features at different levels for quality assessment can more comprehensively and accurately capture the distortion information of the image from the global structure to the local details, improve the robustness, generalization ability and consistency of the assessment, and promote the interpretability of the model and support for complex tasks.

[0108] 3. Progressive Perception Interaction Module

[0109] The progressive perceptual interaction module is used to perform convolution operations on the content-perceptual features and the distortion-perceptual features of each distorted image, respectively, and to perform gating mechanism operations on the content-perceptual features and the distortion-perceptual features of each distorted image, respectively, and to obtain content-perceptual weighted features of each distorted image based on the results of the convolution operations and the gating mechanism operations on the content-perceptual features of each distorted image, and to obtain distortion-perceptual weighted features of each distorted image based on the results of the convolution operations and the gating mechanism operations on the distortion-perceptual features of each distorted image, and to perform feature fusion operations on the content-perceptual weighted features and the distortion-perceptual weighted features of each distorted image to obtain interaction features of each distorted image.

[0110] According to one embodiment of the present application, Figure 3As shown, the initial image quality assessment model includes: multiple progressive perceptual interaction modules, which are used to obtain interactive features of multiple levels of each distorted image, wherein each progressive perceptual interaction module is used to perform convolution operations on content-perceptual features and distortion-perceptual features at the same level of each distorted image, and to perform gating mechanism operations on content-perceptual features and distortion-perceptual features at the same level of each distorted image, respectively; and based on the results of the convolution operations and gating mechanism operations on the content-perceptual features at the same level of each distorted image, content-perceptual weighted features at the same level of each distorted image are obtained; and based on the results of the convolution operations and gating mechanism operations on the distortion-perceptual features at the same level of each distorted image, distortion-perceptual weighted features at the same level of each distorted image are obtained; and feature fusion operations are performed on the content-perceptual weighted features and distortion-perceptual weighted features at the same level of each distorted image to obtain interactive features at the same level of each distorted image.

[0111] It can be seen from the above embodiments that by processing the content perception features and distortion perception features of different levels of the distorted image through different progressive perception interaction modules, the speed of the model processing the image is accelerated. At the same time, applying multiple progressive perception interaction modules to hierarchically fuse the multi-level features of the image can ensure sufficient interaction of features and further improve the accuracy of the image quality assessment results.

[0112] In order to allow the deep learning models configured in the content-aware encoder module and the distortion-aware encoder module to be arbitrarily selected, the present application needs to align the channels of the two features before interacting the content-aware features and the distortion-aware features of the image. Therefore, according to an embodiment of the present application, each distorted image contains distortion-aware features of multiple channels at each level, and each distorted image contains distortion-aware features of multiple channels at each level, and Figure 4 As shown, each progressive perception interaction module includes: a feature alignment submodule and a feature interaction submodule. The feature alignment submodule and the feature interaction submodule of the progressive perception interaction module are described in detail below.

[0113] (I) Feature Alignment Submodule

[0114] The feature alignment submodule is used to perform channel dimensionality reduction operation on the content-aware features of multiple channels in the same level of each distorted image to obtain the content-aware features of the first preset channels in the same level of each distorted image, and to perform channel dimensionality reduction operation on the distortion-aware features of multiple channels in the same level of each distorted image to obtain the distortion-aware features of the first preset channels in the same level of each distorted image.

[0115] According to one embodiment of the present application, Figure 5As shown in Figure 1, the feature alignment submodule consists of a 1×1 convolution. It should be noted that Figure 5 The structure of the feature alignment submodule is only one embodiment of the image quality assessment model of the present application, and the specific convolution size and convolution setting are not limited by this embodiment.

[0116] It can be seen from the above embodiment that the above embodiment reduces the number of channels of the content-aware features and the distortion-aware features of the image to a unified dimension. This process not only aligns the two types of features in the channels, but also reduces the computational complexity of the subsequent processing process.

[0117] 2. Feature Interaction Submodule

[0118] The feature interaction submodule is used to perform convolution operations on the content-aware features and distortion-aware features of all channels in the same level of each distorted image obtained by the feature alignment submodule, and to perform gating mechanism operations on the content-aware features and distortion-aware features of all channels in the same level of each distorted image, and to obtain content-aware weighted features at the same level of each distorted image based on the results of the convolution operations and gating mechanism operations on the content-aware features of all channels in the same level of each distorted image, and to obtain distortion-aware weighted features at the same level of each distorted image based on the results of the convolution operations and gating mechanism operations on the distortion-aware features of all channels in the same level of each distorted image, and to perform feature fusion operations on the content-aware weighted features and distortion-aware weighted features at the same level of each distorted image to obtain interaction features at the same level of each distorted image.

[0119] According to one embodiment of the present application, the feature fusion operation includes a feature coarse fusion operation and a feature fine fusion operation, and as Figure 6 As shown, the feature interaction submodule also includes: a convolution unit, a gating unit, a feature weighting unit, a feature coarse interaction unit, a feature fine interaction unit, and a feature deep interaction unit. Each unit of the feature interaction submodule is described below.

[0120] 1. Convolutional Unit

[0121] The convolution unit is used to perform convolution operations on the content-aware features and distortion-aware features of all channels in the same level of each distorted image obtained by the feature alignment submodule.

[0122] According to one embodiment of the present application, reference is still made to Figure 5 As shown in the figure, the convolution unit consists of a 3×3 depth convolution and two 1×1 convolutions. It should be noted that Figure 5 The structure of the convolution unit is only one embodiment of the image quality assessment model of the present application, and the specific convolution size and convolution setting are not limited by this embodiment.

[0123] The above-mentioned embodiment of the present application can efficiently extract and process the content-aware features and distortion-aware features of the distorted image by adopting a convolution unit composed of a 3×3 depth convolution and two 1×1 convolutions, thereby enhancing the model's understanding of local spatial information, while optimizing the feature expression and improving the ability to focus on the distorted area, thereby improving the image quality assessment accuracy of the model.

[0124] 2. Door control unit

[0125] The gating unit is used to perform gating mechanism operations on the content-aware features and distortion-aware features of all channels in the same level of each distorted image respectively.

[0126] According to one embodiment of the present application, reference is still made to Figure 5 As shown in Figure 1, the gating unit consists of a 1×1 convolution, two 3×3 convolutions and a Sigmoid function. Figure 5 The structure of the middle gate control unit is only one embodiment of the image quality assessment model of the present application, and the specific convolution size and convolution setting are not limited by this embodiment.

[0127] The above embodiment of the present application can accurately operate the gating mechanism for the content-aware features and distortion-aware features of all channels in the same level of each distorted image by using a gating unit composed of a 1×1 convolution, two 3×3 convolutions and a Sigmoid function. This structure not only enhances the selectivity of the model for key features, effectively suppresses irrelevant or noise information, but also increases the focus on distorted areas, thereby significantly improving the accuracy and robustness of the model in image quality assessment.

[0128] 3. Feature Weighting Unit

[0129] The feature weighting unit is used to obtain content-aware weighted features at the same level of each distorted image based on the results of convolution operations and gating mechanism operations on content-aware features of all channels at the same level of each distorted image, and to obtain distortion-aware weighted features at the same level of each distorted image based on the results of convolution operations and gating mechanism operations on distortion-aware features of all channels at the same level of each distorted image.

[0130] According to an embodiment of the present application, the feature weighting unit is configured to obtain the content-aware weighted features and the distortion-aware weighted features at the same level of each distorted image in the following manner:

[0131]

[0132] in, represents the feature alignment submodule of the i-th progressive perceptual interaction module, represents the convolutional unit of the i-th progressive perceptual interaction module, represents the gating unit of the i-th progressive perceptual interaction module, σ(·) is the normalization function, represents the content-aware weighted features of the d-th distorted image at level i, represents the distortion-perceptual weighted features of the ith level of the dth distorted image.

[0133] The above-mentioned embodiment of the present application performs a gating mechanism operation on the content-aware features and distortion-aware features of all channels in the same level of each distorted image, so as to assign corresponding weights to the corresponding content-aware features and distortion-aware features, thereby obtaining content-aware weighted features and distortion-aware weighted features. This method not only enhances the selectivity and sensitivity of the model to key information in the image, effectively suppresses irrelevant or noise information, but also increases the attention to the distorted area, ensures the accuracy and robustness of the feature representation, and significantly improves the accuracy and reliability of the model for image quality assessment.

[0134] 4. Feature coarse interaction unit

[0135] The feature coarse interaction unit is used to perform a feature coarse fusion operation on the content-aware weighted features and the distortion-aware weighted features at the same level of each distorted image to obtain the coarse interaction features at the same level of each distorted image.

[0136] According to an embodiment of the present application, the feature coarse interaction unit is configured to obtain the coarse interaction features of the same level of each distorted image in the following manner:

[0137]

[0138] in, represents the feature coarse interaction unit of the i-th progressive perceptual interaction module, W i represents the coarse interaction features of the ith level of the dth distorted image.

[0139] According to one embodiment of the present application, It consists of a 3×3 convolution.

[0140] The above-mentioned embodiment of the present application performs a coarse feature fusion operation on the content-aware weighted features and distortion-aware weighted features at the same level of each distorted image through a coarse feature interaction unit. This operation can effectively integrate two types of feature information, enhance the model's understanding of key areas in the image, and improve the comprehensiveness and robustness of the feature representation, thereby improving the overall performance and accuracy of the model in image quality assessment.

[0141] 5. Feature Detail Interaction Unit

[0142] The feature fine interaction unit is used to perform a feature fine fusion operation on the content-aware weighted features and the distortion-aware weighted features at the same level of each distorted image to obtain the fine interaction features at the same level of each distorted image.

[0143] The inventors also found through research that since distortion usually occurs in multiple local areas within an image, and humans tend to understand the overall content of an image before focusing on a specific area, this indicates the need for adaptive interaction within multiple distorted areas. Therefore, the present application uses deformable convolution to achieve adaptive interaction of features, also known as fine interaction, that is, using distortion-aware weighted features to learn sampling offsets, and performing deformable sampling operations (also known as deformable convolution operations) on content-aware weighted features based on the sampling offsets, so that the model focuses on distorted areas while maintaining semantic integrity.

[0144] According to one embodiment of the present application, Figure 7 As shown, the feature fine interaction unit includes: a first depth-separable convolution subunit, used to perform a first depth-separable convolution operation on the content-aware weighted features at the same level of each distorted image to obtain the content-aware first intermediate features at the same level of each distorted image; a second depth-separable convolution subunit, used to perform a second depth-separable convolution operation on the distortion-aware weighted features at the same level of each distorted image to obtain the distortion-aware offset features at the same level of each distorted image; a depth-separable deformable convolution subunit, used to perform a depth-separable deformable convolution operation on the content-aware weighted features at the same level of each distorted image based on the distortion-aware offset features at the same level of each distorted image to obtain the content-aware second intermediate features at the same level of each distorted image; a feature fusion subunit, used to perform a fusion operation on the content-aware first intermediate features and the content-aware second intermediate features at the same level of each distorted image to obtain the fine interaction features at the same level of each distorted image.

[0145] It can be seen from the above embodiments that the present application performs deep separable convolution operations on the content-aware weighted features and distortion-aware weighted features of the image respectively, and uses the obtained distortion-aware offset features to perform deep separable deformable convolution operations on the content-aware features, so that the model focuses on the distorted area while maintaining semantic integrity, thereby achieving fine processing and fusion of content-aware and distortion-aware features at each level of the distorted image. This method can not only effectively extract and enhance the key features of the image at different levels, but also dynamically adjust the convolution operation according to the distortion characteristics, thereby more accurately capturing the impact of image distortion. Ultimately, by integrating multi-level information through the feature fusion subunit, the accuracy, robustness and adaptability of image quality assessment are improved, ensuring that the evaluation results are closer to the real experience of human visual perception.

[0146] According to another embodiment of the present application, the content-aware weighted features of multiple channels in the same level of each distorted image are evenly divided into two parts according to the number of channels, and Figure 8 As shown, the feature fine interaction unit includes: a first depth-separable convolution subunit, which is used to perform a first depth-separable convolution operation on the content-aware weighted features of a part of the channels in the same level of each distorted image, so as to obtain the content-aware first intermediate features of a part of the channels in the same level of each distorted image; a second depth-separable convolution subunit, which is used to perform a second depth-separable convolution operation on the distortion-aware weighted features of the same level of each distorted image, so as to obtain the distortion-aware offset features of the same level of each distorted image; a depth-separable deformable convolution subunit, which is used to perform a second depth-separable convolution operation on the distortion-aware weighted features of the same level of each distorted image, so as to obtain the distortion-aware offset features of the same level of each distorted image; The distortion-aware offset features of another part of the channels in the same level of each distorted image are used to perform a depth-separable deformable convolution operation on the content-aware weighted features of another part of the channels in the same level of each distorted image to obtain the content-aware second intermediate features of another part of the channels in the same level of each distorted image; a channel splicing and shuffling subunit is used to perform a channel splicing operation on the content-aware first intermediate features of a part of the channels in the same level of each distorted image and the content-aware second intermediate features of another part of the channels, and perform a channel shuffling operation on the result of the channel splicing operation to obtain the fine interaction features at the same level of each distorted image.

[0147] It can be seen from the above embodiments that the content-aware weighted features of multiple channels in the same level of each distorted image are evenly divided into two parts according to the number of channels for subsequent processing, which can reduce the amount of calculation while avoiding the occurrence of overfitting. At the same time, the present application performs deep separable convolution operations on the content-aware weighted features and distortion-aware weighted features of the image respectively, and uses the obtained distortion-aware offset features to perform deep separable deformable convolution operations on the content-aware features, so that the model focuses on the distorted area while maintaining semantic integrity, thereby realizing fine processing and fusion of content-aware and distortion-aware features at each level of the distorted image; at the same time, the feature Channel shuffling operations can promote information exchange between different channels and enhance the diversity and representativeness of features (that is, when the present application divides the channel into two parts and performs different convolution operations respectively, the original information flow between the channels of each part is cut off, and then the channel shuffling operation can be used to re-establish this cross-channel information exchange without significantly reducing the computational cost, so that each channel can be exposed to the information of all input channels, thereby improving the learning ability of the model and improving the quality of the final fine interaction features. In addition, channel shuffling helps to break the homogeneity of features caused by channel division and ensure that the feature representation is richer and more robust). The above scheme can not only effectively extract and enhance the key features of the image at different levels, but also dynamically adjust the convolution operation according to the distortion characteristics, so as to more accurately capture the impact of image distortion. Finally, by integrating multi-level information through the feature fusion subunit, the accuracy, robustness and adaptability of image quality assessment are improved, ensuring that the evaluation results are closer to the real experience of human visual perception.

[0148] In order to alleviate the increased computational requirements and prevent overfitting, the present application uses channel dimensionality reduction techniques to project the features to be processed into smaller dimensions when distorting the fine interaction features of the image.

[0149] According to one embodiment of the present application, Fig. 9As shown, the feature fine interaction unit also includes: a channel dimension reduction subunit, which is used to perform a first channel dimension reduction operation on the content-aware weighted features of a part of the channels in the same level of each distorted image, and transmit the result after performing the first channel dimension reduction operation to the first depth-wise separable convolution subunit so that it performs a first depth-wise separable convolution operation on the result, and is used to perform a first channel dimension reduction operation on the content-aware weighted features of another part of the channels in the same level of each distorted image, and transmit the result after performing the first channel dimension reduction operation to the depth-wise separable and deformable convolution subunit so that it performs a depth-wise separable and deformable convolution operation on the result; a channel dimension upgrading subunit, which is used to perform a first channel dimension upgrading operation on the output result of the first depth-wise separable convolution subunit, and transmit the result of performing the first channel dimension upgrading operation to the channel splicing and shuffling subunit, and perform a first channel dimension upgrading operation on the output result of the depth-wise separable and deformable convolution subunit, and transmit the result of performing the first channel dimension upgrading operation to the channel splicing and shuffling subunit.

[0150] From the above description, it can be seen that the above-mentioned adaptive fine interaction (i.e., fine interaction) will be performed in a low-dimensional space, and then the channel dimension enhancement technique will be used to project the features back to the original dimension, and then the features of the two parts of the channel will be connected, and the channel shuffle operation will be used to promote information exchange between channels, and finally the coarse interaction features and the fine interaction features will be aggregated for coarse-to-fine interaction. This execution of fine interaction in a low-dimensional space can alleviate the increased computational requirements and prevent overfitting.

[0151] Combine the following Fig.10 The structure of an embodiment of a feature interaction unit and the process of feature interaction performed by the module are described.

[0152] from Fig.10 It can be seen that the execution process of the feature fine interaction unit is:

[0153] First, Input the feature alignment submodule respectively, and get

[0154] Second, The second depth-wise separable convolution subunit is input to obtain the distortion-aware offset feature of the i-th level of the distorted image.

[0155] According to an embodiment of the present application, the calculation formula corresponding to the distortion perception offset feature of the i-th level of the distorted image is:

[0156]

[0157] Among them, Δp i represents the distortion-perceived offset feature of the ith level of the dth distorted image, Represents the second depthwise separable convolutional subunit of the i-th progressive perceptual interaction module.

[0158] Third, and According to the number of channels, it is divided into and will Input the channel dimension reduction subunit separately.

[0159] Fourth, the descending channel Input the first deep learning separable convolution subunit to obtain the content-aware first intermediate feature of the dimension reduction channel in the i-th level of the distorted image, and The input is a depth-wise separable deformable convolution subunit, and the content-aware second intermediate features of the dimension-reduced channels in the i-th level of the distorted image are obtained.

[0160] Fifth, the content-aware first intermediate features of the dimension reduction channels in the i-th level of the distorted image and the content-aware second intermediate features of the dimension reduction channels in the i-th level of the distorted image are respectively input into the channel dimension increase sub-unit to obtain the content-aware first intermediate features of a part of the channels in the i-th level of the distorted image and the content-aware second intermediate features of another part of the channels in the i-th level of the distorted image.

[0161] According to one embodiment of the present application, the calculation formula corresponding to the content-aware first intermediate feature of a part of the channels in the i-th level of the distorted image is:

[0162]

[0163] in, represents the channel dimension reduction subunit of the i-th progressive perceptual interaction module, represents the first depth-wise separable convolutional subunit of the i-th progressive perceptual interaction module, represents the channel dimension-increasing subunit of the i-th progressive perceptual interaction module, represents the depthwise separable deformable convolutional subunit of the i-th progressive perceptual interaction module, Represents the content-aware first intermediate features of a portion of channels in the ith level of the dth distorted image.

[0164] According to one embodiment of the present application, the calculation formula corresponding to the content-aware second intermediate feature of another part of the channels in the i-th level of the distorted image is:

[0165]

[0166] in, represents the depthwise separable deformable convolutional subunit of the i-th progressive perceptual interaction module, Represents the content-aware second intermediate features of another part of the channels in the i-th level of the d-th distorted image.

[0167] Sixth, the content-aware first intermediate features of a part of the channels in the i-th level of the distorted image and the content-aware second intermediate features of another part of the channels in the i-th level of the distorted image are input into the channel stitching subunit for channel stitching operation.

[0168] Seventh, the result after the channel splicing operation is input into the channel shuffling subunit and then the channel shuffling operation is performed to obtain the fine interaction features of the i-th level of the distorted image.

[0169] According to an embodiment of the present application, the channel dimension reduction subunit is composed of a 1×1 convolution, the channel dimension increase subunit is composed of a 1×1 convolution, the first depth separable convolution subunit is composed of a 3×3 depth convolution and a 1×1 convolution, the second depth separable convolution subunit is composed of a 3×3 depth convolution and a 1×1 convolution, and the depth separable deformable convolution subunit is composed of a 3×3 deformable convolution and a 1×1 convolution. It should be noted that, Fig.10 The structure is only one embodiment of the image quality assessment model of the present application, and the specific convolution size and convolution setting are not limited by this embodiment.

[0170] It can be seen from the above embodiment that by using the first depth-separable convolutional word unit to retain The content information in is adjusted by using the depth-wise separable deformable convolution subunit This method not only effectively enhances the model's ability to understand and characterize image content, but also significantly improves the accuracy of adaptive processing of distorted areas in the image, thereby reducing the amount of computation while achieving complex interaction between image content information and distortion information, thereby improving the accuracy of image quality assessment.

[0171] 6. Feature Deep Interaction Unit

[0172] The feature deep interaction unit is used to obtain the interaction features at the same level of each distorted image based on the coarse interaction features and fine interaction features at the same level of each distorted image.

[0173] According to an embodiment of the present application, the feature deep interaction unit is configured to obtain the interaction features of the same level of each distorted image in the following manner:

[0174]

[0175] in, represents the channel shuffling subunit of the i-th progressive perceptual interaction module.

[0176] The above-mentioned embodiments of the present application adopt feature alignment and a coarse-to-fine interaction strategy, so that the extracted content and distortion features can be aligned and fused in a progressive perceptual interaction module, thereby more accurately representing the interaction between the content-perceived features and the distortion-perceived features of the image, and improving the accuracy and generalization of the image quality assessment model for image quality assessment.

[0177] 4. Quality Prediction Module

[0178] The quality prediction module is used to obtain a predicted quality evaluation result of each distorted image based on the interactive features of each distorted image.

[0179] According to an embodiment of the present application, the quality prediction module is configured to obtain a predicted quality evaluation result of each distorted image based on interactive features of multiple levels of each distorted image.

[0180] The following takes the example that the content-aware features of each distorted image include content-aware features of 5 levels of each distorted image, and the distortion-aware features of each distorted image include distortion-aware features of 5 levels of each distorted image as an example to introduce in detail the functions performed by the quality prediction module.

[0181] Among them, according to one embodiment of the present application, the quality prediction module is configured to: perform serial operations on the interaction features of multiple levels of each distorted image to obtain the target interaction features of each distorted image; based on the target interaction features of each distorted image, obtain the predicted quality evaluation result of each distorted image.

[0182] The quality prediction module is configured to obtain the target interaction features of each distorted image in the following manner:

[0183]

[0184] Among them, G 1 represents the first-level interactive features of each distorted image, G 1 represents the first-level interactive features of each distorted image, G 2 represents the interactive features of the second level of each distorted image, G 3 represents the interactive features of the third level of each distorted image, G 4 represents the fourth-level interactive features of each distorted image, G 5 Represents the interactive features of the fifth level for each distorted image.

[0185] Since the spatial resolution of these features is inconsistent (the feature resolutions obtained at different levels are different), this application uses average pooling to convert G 1 , G 2 , G 3 , G4 Reduced to the highest level feature G 5 The same resolution is used to perform the above-mentioned feature concatenation operation, and quality assessment is performed based on the result after the concatenation operation.

[0186] The above embodiments of the present application connect interactive features of different levels in series, and the model can integrate multi-level information from low-level to high-level. Low-level features usually contain detailed information such as edges and textures, while high-level features capture more abstract content and structural information. Fusing these different levels of information together can provide a more comprehensive and detailed image representation, thereby improving the sensitivity and ability of the image quality assessment (IQA) model to distinguish various types of distortion in the image.

[0187] According to one embodiment of the present application, Fig.11 As shown, the target interaction feature of each distorted image includes multiple pixels, and the quality prediction module includes: a feature concatenation submodule, which is used to perform concatenation operations on the interaction features of multiple levels of each distorted image to obtain the target interaction feature of each distorted image; a weighting submodule is used to obtain the weight matrix corresponding to the quality score of each pixel in each distorted image based on the target interaction feature of each distorted image; a prediction submodule is used to obtain the initial quality score corresponding to each pixel in each distorted image based on the target interaction feature of each distorted image; and a weighted summation submodule is used to obtain the predicted quality evaluation result of each distorted image based on the weight matrix corresponding to the quality score of each pixel in each distorted image and the initial quality score corresponding to each pixel in each distorted image. Among them, the weighting submodule and the prediction submodule are both implemented using independent MLP.

[0188] According to an embodiment of the present application, the weighted summation submodule is configured to obtain the prediction quality evaluation result of each distorted image in the following manner:

[0189]

[0190] Among them, w(G) represents the weight matrix corresponding to the quality score of each pixel in each distorted image, s(G) represents the initial quality score corresponding to each pixel in each distorted image, Q pred represents the prediction quality evaluation result of each distorted image, and * represents element-wise multiplication.

[0191] The above-mentioned embodiments of the present application achieve refined quality assessment of each pixel in the distorted image by introducing a weighted sub-module, a prediction sub-module and a weighted summation sub-module to ensure consistency between quality prediction and the interaction process, thereby reflecting the different contributions of different image areas to the overall perceived quality.

[0192] In addition, in order to more clearly understand the content of step S3 in the above embodiment of the present application, step S3 is described in detail below.

[0193] In step S3, the initial image quality assessment model is iteratively trained multiple times using the distorted image set until convergence, and during each iterative training process, the parameters of the image quality assessment model are updated according to a preset loss function.

[0194] According to one embodiment of the present application, the preset loss function is:

[0195] L=||Q pred -Q lable || 2

[0196] Among them, Q lable is the true quality score of each distorted image derived from subjective experiments.

[0197] The above-mentioned embodiment of the present application compares the predicted quality score with the actual quality score obtained from subjective experiments. This method ensures that the model can accurately capture the details of image distortion, thereby providing more reliable quality assessment results in practical applications.

[0198] In order to maintain the stability of the interaction, the present application adopts a specific training strategy. According to an embodiment of the present application, in each iterative training process, the parameters of the content-aware encoder module, the progressive perceptual interaction module and the quality prediction module are updated according to a preset loss function.

[0199] The above-mentioned embodiments of the present application freeze the parameters of the distortion-aware encoder module to ensure that the captured distortion information remains stable throughout the training process, which is crucial for generating consistent offsets that reflect the distorted area.

[0200] Combine the following Fig.12 The following describes in detail the process of an embodiment of a method for constructing an image quality assessment model of the present application.

[0201] from Fig.12 It can be seen that, first, the distorted image is input into the distortion-aware encoder module and the content-aware encoder module respectively to obtain the distortion-aware features and content-aware features of the distorted image, and the distortion-aware features and content-aware features of the distorted image are input into the progressive perceptual interaction module for feature interaction, including channel alignment, convolution and gating mechanism operations on the features, and the operation results are input into the feature coarse interaction unit and the feature fine interaction unit in the progressive perceptual interaction module, and finally the target interaction features of the distorted image are obtained, and the target interaction features of the distorted image are input into the prediction submodule and the weighted submodule respectively, and the quality prediction result of the distorted image is obtained based on the output results of the two submodules.

[0202] The above-mentioned embodiments of the present application can accurately identify the distortion position in the image while maintaining semantic integrity, thereby realizing a complex interaction between image content information and distortion information.

[0203] In order to verify the effectiveness of the scheme for constructing the image quality assessment model proposed in the above embodiment of the present application, the inventor conducted the following experiments:

[0204] In order to demonstrate the advantages of the above embodiments of the present application, the present application uses two evaluation indicators to measure the performance of different image quality assessment models. Among them, the Spearman rank correlation coefficient (SRCC) is used to evaluate the monotonicity of model quality prediction, and the Pearson linear correlation coefficient (PLCC) is used to measure the accuracy of model quality prediction. The range of these two indicators is [0,1]. The larger the value of the indicator, the better the performance of the model. The present application fully verifies the proposed method in terms of evaluation accuracy, data efficiency and generalization ability.

[0205] (1) Evaluation accuracy: In order to demonstrate the superiority of the above embodiments of the present application in terms of evaluation accuracy, the quality evaluation methods such as DEIQT and ARNIQA were compared on the KADID-10K and KonIQ-10K datasets. The experimental results are shown in Figure 2. Fig.13 As shown, from Fig.13 It can be seen that the performance of the above embodiments of the present application on the two data sets is significantly better than the comparison methods DEIQT and ARNIQA, which confirms the effectiveness and superiority of the above embodiments of the present application in accurately expressing image quality. In addition, the above embodiments of the present application further analyze the impact on the final evaluation accuracy when only a single progressive perceptual interaction module is used for feature interaction. It can be found that when only single-scale features are extracted (single-scale means that the model only extracts and processes one-level features of the distorted image, and multi-scale means that the model extracts and processes multiple-level features of the distorted image), the performance of the model constructed by the above embodiments of the present application can surpass the existing methods, which shows the effectiveness of the progressive perceptual interaction module in modeling the interaction between content and distortion.

[0206] (2) Data efficiency: Given the high cost of image annotation, data-efficient learning capabilities are critical for image quality assessment models. The above embodiments of the present application demonstrate excellent data-efficient learning capabilities. Compared with other advanced methods, the methods proposed in the above embodiments of the present application can achieve competitive or even superior performance using only a small amount of training data. In order to verify this feature, the performance changes of the above embodiments of the present application were observed by gradually increasing the amount of training data from 10% to 70% in intervals of 10%. The experimental results are as follows: Fig.14 As shown, from Fig.14It can be seen that when tested on the KADID-10K dataset, the above embodiment of the present application can achieve significantly better performance than ARNIQA even with only 10% of the training data. When tested on the KonIQ-10K dataset, the above embodiment of the present application can surpass DEIQT with 40% of the training data. These experimental results show that the above embodiment of the present application can easily model the interaction between the content and distortion of the image using less training data, thereby demonstrating excellent data-efficient learning capabilities.

[0207] (3) Generalization ability: The above embodiment of the present application evaluates the generalization ability of the image quality assessment model by training the image quality assessment model on one data set and testing it on other data sets. Specifically, the model is first trained using the KADID-10K data set and tested using the CSIQ data set. Then the KonIQ-10K data set is used for training and the CLIVE data set is used for testing. The SRCC results compared with DEIQT and ARNIQA are shown in Table 1. In addition, the above embodiment of the present application also explores two additional technical variants, namely, further verifying the effectiveness of the above embodiment of the present application by retaining the coarse interaction part or the fine interaction part in the progressive perceptual interaction module. As can be seen from Table 1, the generalization ability of the above embodiment of the present application is significantly better than the comparison method, indicating that the above embodiment of the present application can construct a universal quality perception representation, so that it can be well applied in different distortion scenarios. The comparison results of different technical variants also prove the effectiveness of the above embodiment of the present application.

[0208] Table 1

[0209]

[0210] In summary, compared with the image quality assessment models constructed in the existing image quality assessment methods, which have failed to effectively process the interaction between the content and distortion of the image, that is, they have failed to explore the complex interaction between the content and distortion in the image (that is, the degree of influence of different distortion types on the image content is different), the image quality assessment model constructed by the new image quality assessment model construction scheme proposed in this application first performs channel alignment on the extracted content-aware features and distortion-aware features to ensure channel dimension consistency; secondly, a gating mechanism is used to dynamically determine which features to retain or suppress, thereby achieving internal interaction; then, the aligned features are fused for coarse interaction, and in order to achieve adaptive interaction, a distortion-guided deformable convolution operation is applied to resample the aligned content-aware features to focus on the distorted area while maintaining semantic integrity; finally, the features obtained from the coarse interaction and the features resampled by the adaptive interaction are added to achieve coarse-to-fine interaction. The above processing method promotes high-order interaction between distortion-aware features and content-aware features using a progressive perceptual interaction module to capture the complex interaction between content and distortion features. In addition, the above-mentioned embodiment of the present application expands the progressive perceptual interaction module to hierarchically fuse content-aware features and distortion-aware features of different scales to ensure sufficient interaction between content-aware features and distortion-aware features, thereby improving the accuracy of the model in assessing image quality.

[0211] Based on the inventive concept of the above embodiments, the present application further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the above embodiments when executing the computer program. Fig.15 Provide detailed explanation.

[0212] like Fig.15 As shown, it shows the electronic device of the present application, which may specifically include a processor 110 and a memory 120. The memory 120 is coupled to the processor 110.

[0213] The processor 110 is used to control the operation of the electronic device. The processor 110 may also be referred to as a CPU (Central Processing Unit). The processor 110 may be an integrated circuit chip having the ability to process signals. The processor 110 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor 110 may also be any conventional processor, etc.

[0214] The memory 120 is used to store computer programs, and may be a RAM, a ROM, or other types of storage terminals. Specifically, the memory 120 may include one or more computer-readable storage media, which may be non-transitory or transient. The memory 120 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage terminals, flash memory storage terminals. In some embodiments, the non-transitory computer-readable storage medium in the memory 120 is used to store at least one program code.

[0215] The processor 110 is used to execute the computer program stored in the memory 120 to implement the methods described in the various method embodiments of the present application.

[0216] In some embodiments, the electronic device may further include: a peripheral terminal interface 130 and at least one peripheral terminal. The processor 110, the memory 120 and the peripheral terminal interface 130 may be connected via a bus or a signal line. Each peripheral terminal may be connected to the peripheral terminal interface 130 via a bus, a signal line or a circuit board. Specifically, the peripheral terminal includes: at least one of a radio frequency circuit 140, a display screen 150, an audio circuit 160 and a power supply 170.

[0217] The peripheral terminal interface 130 may be used to connect at least one peripheral terminal related to I / O (Input / output) to the processor 110 and the memory 120. In some embodiments, the processor 110, the memory 120, and the peripheral terminal interface 130 are integrated on the same chip or circuit board; in some other implementations, any one or two of the processor 110, the memory 120, and the peripheral terminal interface 130 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0218] The radio frequency circuit 140 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 140 communicates with the communication network and other IoT devices through electromagnetic signals, and the radio frequency circuit 140 is the communication circuit of the electronic device. The radio frequency circuit 140 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 140 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, an operator identity module card, and the like. The radio frequency circuit 140 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 140 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0219] The display screen 150 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 150 is a touch display screen, the display screen 150 also has the ability to collect touch signals on the surface or above the surface of the display screen 150. The touch signal can be input to the processor 110 as a control signal for processing. At this time, the display screen 150 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 150 can be one, set on the front panel of the electronic device; in other embodiments, the display screen 150 can be at least two, respectively set on different surfaces of the electronic device or in a folding design; in other embodiments, the display screen 150 can be a flexible display screen, set on a curved surface or a folding surface of the electronic device. Even, the display screen 150 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 150 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light□EmittingDiode, organic light-emitting diode).

[0220] The audio circuit 160 may include a microphone and a speaker. The microphone is used to collect sound waves from the operator and the environment, and convert the sound waves into electrical signals and input them into the processor 110 for processing, or input them into the radio frequency circuit 140 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there can be multiple microphones, which are respectively arranged in different parts of the electronic device. The microphone can also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 110 or the radio frequency circuit 140 into sound waves. The speaker can be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 160 may also include a headphone jack.

[0221] The power supply 170 is used to power various components in the electronic device. The power supply 170 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 170 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0222] For a detailed description of the functions and execution processes of each functional module or component in the electronic device embodiment of the present application, reference can be made to the description in the above-mentioned method embodiments of the present application, which will not be repeated here.

[0223] In the several embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the various embodiments of the electronic devices described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0224] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0225] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0226] Based on the inventive concept of the above embodiment, the present application also provides a computer-readable storage medium storing a computer program, which can be executed by a processor to implement the above projection area calculation method. Fig.16 The execution process of the above-mentioned embodiment in a computer-readable storage medium is described.

[0227] like Fig.16 As shown, it shows the computer-readable storage medium of the present application. If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium 200. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions / computer programs to enable an IoT device (which can be a personal computer, server, or network terminal, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and electronic terminals such as computers, mobile phones, laptops, tablet computers, cameras, etc. having the above-mentioned storage medium.

[0228] The description of the execution process of the program data in the computer-readable storage medium can refer to the description in the above-mentioned method embodiments of the present application, which will not be repeated here.

[0229] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

[0230] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

Claims

1. A method for constructing an image quality assessment model, wherein the image quality assessment model is used to assess the quality of an image, characterized in that: The method comprises: Acquire a distorted image set, wherein the distorted image set includes a plurality of distorted images; Constructing an initial image quality assessment model, wherein the initial image quality assessment model includes: a content-aware encoder module, configured to perform a content-aware feature extraction operation on each distorted image to obtain content-aware features of each distorted image; A distortion perception encoder module, configured to perform a distortion perception feature extraction operation on each distorted image to obtain a distortion perception feature of each distorted image; a progressive perceptual interaction module, configured to perform convolution operations on the content-perceptual features and the distortion-perceptual features of each distorted image, respectively, and perform gating mechanism operations on the content-perceptual features and the distortion-perceptual features of each distorted image, respectively, and obtain content-perceptual weighted features of each distorted image based on the results of the convolution operations and the gating mechanism operations on the content-perceptual features of each distorted image, and obtain distortion-perceptual weighted features of each distorted image based on the results of the convolution operations and the gating mechanism operations on the distortion-perceptual features of each distorted image, and perform feature fusion operations on the content-perceptual weighted features and the distortion-perceptual weighted features of each distorted image to obtain interaction features of each distorted image; A quality prediction module, configured to obtain a predicted quality evaluation result of each distorted image based on the interactive features of each distorted image; The initial image quality assessment model is iteratively trained multiple times using the distorted image set until convergence, and during each iterative training process, the parameters of the image quality assessment model are updated according to a preset loss function.

2. The method for constructing an image quality assessment model according to claim 1, characterized in that: The content-aware features of each distorted image include content-aware features of multiple levels of each distorted image, the distortion-aware features of each distorted image include distortion-aware features of multiple levels of each distorted image, and the initial image quality assessment model includes: The plurality of progressive perceptual interaction modules are used to obtain the interaction features of multiple levels of each distorted image, wherein each of the progressive perceptual interaction modules is used to perform convolution operations on the content-perceptual features and distortion-perceptual features of the same level of each distorted image, respectively, and perform gating mechanism operations on the content-perceptual features and distortion-perceptual features of the same level of each distorted image, respectively, and obtain the content-perceptual weighted features of the same level of each distorted image based on the results of the convolution operations and gating mechanism operations on the content-perceptual features of the same level of each distorted image, and obtain the distortion-perceptual weighted features of the same level of each distorted image based on the results of the convolution operations and gating mechanism operations on the distortion-perceptual features of the same level of each distorted image, and perform feature fusion operations on the content-perceptual weighted features and distortion-perceptual weighted features of the same level of each distorted image to obtain the interaction features of the same level of each distorted image.

3. The method for constructing an image quality assessment model according to claim 2, characterized in that: Each distorted image contains distortion perception features of multiple channels at each level, each distorted image contains distortion perception features of multiple channels at each level, and each of the progressive perception interaction modules includes: a feature alignment submodule, configured to perform a channel dimensionality reduction operation on the content-aware features of the multiple channels at the same level of each distorted image to obtain content-aware features of a first preset number of channels at the same level of each distorted image, and to perform a channel dimensionality reduction operation on the distortion-aware features of the multiple channels at the same level of each distorted image to obtain distortion-aware features of a first preset number of channels at the same level of each distorted image; A feature interaction submodule is used to perform convolution operations on the content-aware features and distortion-aware features of all channels in the same level of each distorted image obtained by the feature alignment submodule, and to perform gating mechanism operations on the content-aware features and distortion-aware features of all channels in the same level of each distorted image, and to obtain content-aware weighted features at the same level of each distorted image based on the results of the convolution operations and gating mechanism operations on the content-aware features of all channels in the same level of each distorted image, and to obtain distortion-aware weighted features at the same level of each distorted image based on the results of the convolution operations and gating mechanism operations on the distortion-aware features of all channels in the same level of each distorted image, and to perform feature fusion operations on the content-aware weighted features and distortion-aware weighted features at the same level of each distorted image to obtain interaction features at the same level of each distorted image.

4. The method for constructing an image quality assessment model according to claim 3, characterized in that: The feature fusion operation includes a feature coarse fusion operation and a feature fine fusion operation, and the feature interaction submodule also includes: A convolution unit, configured to perform convolution operations on the content-aware features and the distortion-aware features of all channels in the same level of each distorted image obtained by the feature alignment submodule; A gating unit, configured to perform a gating mechanism operation on the content-aware features and the distortion-aware features of all channels in the same level of each distorted image respectively; a feature weighting unit, configured to obtain content-aware weighted features of the same level of each distorted image based on a result of performing a convolution operation and a gating mechanism operation on the content-aware features of all channels in the same level of each distorted image, and to obtain distortion-aware weighted features of the same level of each distorted image based on a result of performing a convolution operation and a gating mechanism operation on the distortion-aware features of all channels in the same level of each distorted image; A feature coarse interaction unit, used for performing a feature coarse fusion operation on the content-aware weighted features and the distortion-aware weighted features at the same level of each distorted image, so as to obtain a coarse interaction feature at the same level of each distorted image; A feature fine interaction unit, configured to perform a feature fine fusion operation on the content-aware weighted features and the distortion-aware weighted features at the same level of each distorted image, so as to obtain a fine interaction feature at the same level of each distorted image; The feature deep interaction unit is used to obtain the interaction features at the same level of each distorted image based on the coarse interaction features and fine interaction features at the same level of each distorted image.

5. The method for constructing an image quality assessment model according to claim 4, characterized in that: The content-aware weighted features of the multiple channels in the same level of each distorted image are evenly divided into two parts according to the number of channels, and the feature fine interaction unit includes: a first depthwise separable convolution subunit, configured to perform a first depthwise separable convolution operation on the content-aware weighted features of a portion of the channels at the same level of each distorted image, so as to obtain content-aware first intermediate features of a portion of the channels at the same level of each distorted image; A second depthwise separable convolution subunit is configured to perform a second depthwise separable convolution operation on the distortion-perceived weighted features at the same level of each distorted image to obtain distortion-perceived offset features at the same level of each distorted image; a depthwise separable deformable convolution subunit, configured to perform a depthwise separable deformable convolution operation on the content-aware weighted features of another portion of the channels in the same level of each distorted image based on the distortion-aware offset features of another portion of the channels in the same level of each distorted image, so as to obtain content-aware second intermediate features of another portion of the channels in the same level of each distorted image; The channel stitching and shuffling subunit is used to perform a channel stitching operation on the content-aware first intermediate features of a part of the channels at the same level of each distorted image and the content-aware second intermediate features of another part of the channels, and to perform a channel shuffling operation on the result of the channel stitching operation to obtain the detailed interactive features at the same level of each distorted image.

6. The method for constructing an image quality assessment model according to claim 5, characterized in that: The quality prediction module is configured to: Based on the interactive features of multiple levels of each distorted image, a prediction quality evaluation result of each distorted image is obtained.

7. An image quality assessment model obtained based on the method for constructing an image quality assessment model according to any one of claims 1 to 6, characterized in that: The image quality assessment model includes: a content-aware encoder module, configured to perform a content-aware feature extraction operation on the acquired distorted image to obtain content-aware features of the distorted image; A distortion perception encoder module, used for performing a distortion perception feature extraction operation on the distorted image to obtain distortion perception features of the distorted image; a progressive perceptual interaction module, configured to perform convolution operations on the content-perceptual features and the distortion-perceptual features of the distorted image, respectively, and perform gating mechanism operations on the content-perceptual features and the distortion-perceptual features of the distorted image, respectively, and obtain content-perceptual weighted features of the distorted image based on the results of the convolution operations and the gating mechanism operations on the content-perceptual features of the distorted image, and obtain distortion-perceptual weighted features of the distorted image based on the results of the convolution operations and the gating mechanism operations on the distortion-perceptual features of the distorted image, and perform feature fusion operations on the content-perceptual weighted features and the distortion-perceptual weighted features of the distorted image to obtain interaction features of the distorted image; The quality prediction module is used to obtain a predicted quality evaluation result of the distorted image based on the interactive features of the distorted image.

8. An application method of an image quality assessment model, characterized in that: The method comprises: Obtaining a distorted image; The quality of the distorted image is assessed using the image quality assessment model as claimed in claim 7.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 6 and 8 when executing the computer program.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method described in any one of claims 1-6 and 8 is implemented.

Images