Method, device and computer equipment for evaluating image quality without reference

Abstract

The present application is applicable to the field of image quality assessment technology, and provides a reference-free image quality assessment method, apparatus, and computer equipment, including: performing multi-scale layer decomposition of a target image through discrete wavelet transform, and extracting statistical features for each subband; converting the multi-scale and multi-band subband statistical features into structured coded frequency features and coded scale features through scale-frequency dual-path feature coding; defining frequency nodes and scale nodes, connecting the frequency nodes and scale nodes to form a heterogeneous graph, and fusing the graph embedding vector according to the attention weight of each node; mapping the graph embedding vector to the Transformer input dimension, and generating a global feature vector representing the overall image quality based on the Transformer output result; and using a linear regression network to map the global feature vector to an image quality score, effectively improving the ability to capture and model global information and enhance the ability to recognize global structural distortion.

Description

Technical Field

[0001] The present application belongs to the technical field of image quality assessment, and in particular relates to a method, apparatus, and computer equipment for non-reference image quality assessment. Background Art

[0002] Image quality assessment (IQA) is an important research area in computer vision and image processing. Among traditional image quality assessment methods, no-reference image quality assessment (NR-IQA) has attracted widespread attention due to its high practical value due to its independence from original, undistorted images.

[0003] Traditional NR-IQA methods primarily rely on hand-crafted statistical features and perceptual models, such as those based on Natural Scene Statistics (NSS). These methods analyze the statistical distribution of images in the spatial or frequency domain to construct quality metrics (such as mean, standard deviation, entropy, and energy). These methods aim to assess image quality by detecting deviations in statistical properties caused by distortions such as compression, noise, and blur. However, traditional methods have obvious limitations: First, traditional methods rely on predefined manual features, which makes it difficult to fully and effectively characterize the complex distortion patterns of images at multiple scales and frequency bands. For example, simple statistics (mean, variance, etc.) often cannot fully capture the subtle changes in image details and texture; second, many methods focus too much on local statistical characteristics and lack the ability to model the global structure and cross-scale information of images, resulting in a large deviation between the evaluation results and the subjective perception of the human eye when processing high-resolution images or complex scenes; finally, models based on manual features are usually well adapted to specific distortion types, but have poor generalization capabilities when faced with the superposition of multiple distortion types or complex actual application scenarios. Feature engineering adjustments are often required for different scenarios, and there is a lack of a unified and robust evaluation framework.

[0004] To overcome these shortcomings, deep learning methods, especially convolutional neural networks (CNNs), have been introduced for automatic feature learning and have made progress in image processing tasks. However, due to their inherent local receptive field limitations, existing CNN-based methods are still unable to model global distortion structures in image quality assessment. Summary of the Invention

[0005] The embodiments of the present application provide a reference-free image quality assessment method, apparatus, and computer equipment, which can effectively improve the ability to capture and model global information and enhance the ability to identify global structural distortion.

[0006] In a first aspect, an embodiment of the present application provides a method for evaluating image quality without reference, comprising:

[0007] The target image is decomposed into multiple scale layers by discrete wavelet transform. Each scale layer includes low-frequency sub-bands and high-frequency sub-bands, and statistical features are extracted for each sub-band.

[0008] Through scale-frequency dual-path feature encoding, the statistical features of each sub-band are converted into structured coded frequency features and coded scale features. Frequency nodes and scale nodes are defined based on the coded frequency features and coded scale features, and the frequency nodes and scale nodes are connected to form a heterogeneous graph. The attention weight of each node is calculated using a graph attention network and fused to form a graph embedding vector.

[0009] The above image embedding vector is mapped to the Transformer input dimension to obtain the global feature vector of the Transformer output that represents the overall image quality; the position encoding of the above Transformer integrates frequency information and scale information, and the self-attention mechanism integrates the scale weight matrix;

[0010] The above global feature vector is mapped into an image quality score using a linear regression network.

[0011] Exemplarily, each scale layer includes one low-frequency sub-band and three high-frequency sub-bands; the low-frequency sub-band is the LL sub-band, which contains the overall structural information of the image; the high-frequency sub-band includes the LH sub-band, the HL sub-band and the HH sub-band, the LH sub-band reflects the edge features of the image in the horizontal direction, the HL sub-band reflects the edge features of the image in the vertical direction, and the HH sub-band contains the texture and detail information of the image in the diagonal direction.

[0012] Exemplarily, extracting statistical features for each sub-band includes:

[0013] For each subband, four statistical features are extracted: mean μ, standard deviation σ, energy E, and entropy H.

[0014] Exemplarily, the statistical features of each sub-band are converted into structured coding frequency features and coding scale features, including:

[0015] Construct a frequency feature vector of each sub-band based on all statistical features of the sub-band, stack the frequency feature vectors of all sub-bands to obtain a frequency feature sequence, and encode the frequency feature sequence to obtain an encoded frequency feature;

[0016] Global average pooling is performed on the low-frequency subband and high-frequency subband of each scale layer to obtain low-frequency features and high-frequency features, and the scale features of each scale layer are obtained by concatenating the above low-frequency features and the above high-frequency features. The scale features of each scale layer are encoded to obtain encoded scale features.

[0017] Exemplarily, defining frequency nodes and scale nodes according to the coding frequency feature and the coding scale feature includes:

[0018] The coding frequency characteristics of each high frequency sub-band are defined as frequency nodes;

[0019] The encoded scale features of each scale layer are defined as scale nodes.

[0020] Exemplarily, connecting the frequency nodes and the scale nodes to form a heterogeneous graph includes:

[0021] Connect scale nodes of adjacent scale layers to form scale-scale edges;

[0022] Connect frequency nodes and scale nodes in the same scale layer to form frequency-scale edges;

[0023] Nodes with different frequencies are not explicitly connected or are assigned weights less than a threshold.

[0024] Exemplarily, mapping the above graph embedding vector to the Transformer input dimension includes:

[0025] The above graph embedding vector is divided into multiple small patches, and each patch is converted into a node embedding vector of fixed dimension through linear mapping.

[0026] For example, the Transformer's positional encoding fuses frequency information and scale information, including:

[0027] Generate a position code based on the spatial position, frequency information, and scale information of each patch, and add the position code to the above node embedding vector to obtain the fused input vector;

[0028] The above self-attention mechanism integrates the scale weight matrix, including:

[0029] The scale matrix is ​​introduced into the self-attention calculation formula. The scale matrix is ​​calculated based on the scale information of each patch, and the attention weight is positively correlated with the similarity of the scale or the similarity of the frequency band.

[0030] In a second aspect, an embodiment of the present application provides a non-reference image quality assessment device, comprising:

[0031] The wavelet transform feature extraction module is used to decompose the target image into multiple scale layers through discrete wavelet transform. Each scale layer includes low-frequency sub-bands and high-frequency sub-bands, and statistical features are extracted for each sub-band.

[0032] The frequency-scale coding and graph structure fusion module is used to convert the statistical features of each subband into structured coded frequency features and coded scale features through scale-frequency dual-path feature coding; frequency nodes and scale nodes are defined based on the coded frequency features and the coded scale features, and the frequency nodes and scale nodes are connected to form a heterogeneous graph. The graph attention network is used to calculate the attention weight of each node and fuse them to form a graph embedding vector;

[0033] The Transformer global modeling module is used to map the above-mentioned graph embedding vector to the Transformer input dimension, and obtain the global feature vector of the Transformer output that represents the overall quality of the image. The position encoding of the above-mentioned Transformer integrates frequency information and scale information, and the self-attention mechanism integrates the scale weight matrix.

[0034] The quality assessment and regression module is used to map the above global feature vector into an image quality score using a linear regression network.

[0035] In a third aspect, an embodiment of the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method described in any one of the first aspects above is implemented.

[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement any method described in the first aspect above.

[0037] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer device, enables the computer device to execute any of the methods described in the first aspect above.

[0038] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0039] The present application provides a reference-free image quality assessment method based on discrete wavelet transform, frequency-scale dual-path coding and graph structure fusion, which is mainly used for automatic detection and evaluation of image quality. This method overcomes the problems of insufficient local feature expression, limited global information modeling ability and limited generalization ability of traditional perception models based on statistical characteristics and manual design, and at the same time solves the problem of insufficient global distortion capture caused by the local receptive field limitation of CNN in deep learning methods. By utilizing the frequency decomposition ability of discrete wavelet transform, the global modeling ability of Transformer and the perception ability of graph attention network for nonlinear interaction between frequency and scale, a frequency-scale dual-path structure is constructed, and a graph structure attention fusion mechanism is introduced to perform deep joint modeling of the statistical features of the image at different frequency bands and scales, effectively improving the perception accuracy and robustness of complex distortion patterns of the image, thereby achieving efficient and accurate reference-free image quality assessment.

[0040] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0042] Figure 1 1 is a flow chart of a method for evaluating image quality without reference provided by an embodiment of the present application;

[0043] Figure 2 This is a flowchart of step 102 provided in one embodiment of the present application;

[0044] Figure 3 This is a flow chart of generating a global feature vector based on Transformer provided in an embodiment of the present application;

[0045] Figure 4 This is a schematic diagram of a Transformer architecture provided by an embodiment of the present application;

[0046] Figure 5 This is a structural block diagram of a device for evaluating image quality without reference provided by an embodiment of the present application;

[0047] Figure 6 It is a structural diagram of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0048] In the following description, specific details such as specific system structures and technologies are provided for illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present application with unnecessary details.

[0049] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0050] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0051] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0052] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0053] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present invention. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in yet other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0054] The technical solutions in the embodiments of the present application are described in detail below.

[0055] Figure 1This is a flow chart of a method for evaluating image quality without reference provided by an embodiment of the present application. As an example and not a limitation, this method can be applied to computer devices. Figure 1 As shown, the method includes:

[0056] Step 101 : Decompose the target image into multiple scale layers by discrete wavelet transform, and extract statistical features for each sub-band.

[0057] This step uses discrete wavelet transform to decompose the target image into multiple scales and frequency bands, thereby capturing the characteristic information of the target image at different scales and frequency bands. Statistical features include mean, standard deviation, energy, entropy, and other statistical features, which are used to characterize the content distribution of the image at different frequency levels.

[0058] In one embodiment, the target image is decomposed into multiple scale layers by discrete wavelet transform, including: decomposing the target image into multiple scale layers and multiple frequency bands by discrete wavelet transform, wherein each scale layer includes a low frequency sub-band and a high frequency sub-band, i.e., includes sub-bands of different frequency bands. For example, the pre-processed target image can be The discrete wavelet transform is used to perform multi-layer decomposition according to the following formula:

[0059] (1)

[0060] Among them, j is the scale parameter, which represents the scale level in the multi-scale layer decomposition; k is the two-dimensional translation parameter, which represents the position offset of the wavelet function in the image space domain; l is the direction parameter, which identifies different sub-band types; represents the wavelet function after translation and scaling at scale level j; represents the corresponding wavelet coefficients.

[0061] The above formula decomposes the image into j scale layers, each of which is decomposed into four subbands: the LL subband (containing the overall image structure), the LH subband (reflecting horizontal edge features), the HL subband (reflecting vertical edge features), and the HH subband (containing diagonal texture and detail information). The LL subband is a low-frequency subband, while the LH, HL, and HH subbands are high-frequency subbands.

[0062] In this step, the image can be analyzed in both space (i.e., multi-scale) and frequency through discrete wavelet transform. The low-frequency component (LL subband) reflects the global structure and basic brightness information of the image, while the high-frequency component (LH, HL, HH subband) contains the edge, texture, and local detail information of the image.

[0063] For example, for each subband, the following four statistical features can be extracted: mean μ, standard deviation σ, energy E, and entropy H. These statistical features can quantify the local brightness distribution, contrast, texture complexity, and detail information of the image. The specific calculation formula is as follows:

[0064] Mean μ:

[0065] (2)

[0066] Standard deviation σ:

[0067] (3)

[0068] Energy E:

[0069] (4)

[0070] Entropy H:

[0071] (5)

[0072] in, For a subband in pixel The coefficient at is the probability distribution of the normalized coefficient histogram, and M and N are the number of rows and columns of the subband.

[0073] In one embodiment, before performing discrete wavelet transform, the target image may be preprocessed, and the preprocessing includes at least one of the following: radiation correction, geometric correction, normalization, and noise removal.

[0074] The following example illustrates image preprocessing using geometric correction and noise removal. During image acquisition, geometric distortion may occur due to sensor motion, platform jitter, or lens distortion. Geometric correction aligns the image pixel positions to their true spatial coordinates. To ensure that pixels in each band or frame accurately correspond to their actual physical locations, this example uses geometric correction to register and resample the images.

[0075] (6)

[0076] Assumptions is a pixel position in the original image, and the new position is obtained after correction by the transmission transformation f Then use Gaussian blur filtering to remove noise. The specific process is as follows:

[0077] (7)

[0078] In formulas (7) and (8), i and j are input vectors; Indicates the image corrected by the transmission transformation in coordinates The pixel intensity value at ; is a Gaussian kernel, which is used to perform convolution smoothing on the image. The Gaussian kernel is defined as follows:

[0079] (8)

[0080] In this formula, σ represents the standard deviation of the Gaussian kernel.

[0081] In step 101, the target image is decomposed into multiple sub-bands at multiple scales and frequency bands through wavelet transform. Within each sub-band, statistical features such as mean, standard deviation, energy, and entropy are extracted to characterize the image's local structure, texture variations, and edge details. The resulting feature vector not only preserves spatial locality but also enhances frequency representation, providing more refined, structure-aware input data for subsequent image quality assessment steps.

[0082] In step 102, the statistical features of each subband are converted into structured coded frequency features and coded scale features through scale-frequency dual-path feature coding. Frequency nodes and scale nodes are defined accordingly, and the frequency nodes and scale nodes are connected to form a heterogeneous graph. The attention weight of each node is calculated using the graph attention network and fused to form a graph embedding vector.

[0083] To better extract and organize the multi-scale and multi-frequency information generated by the wavelet transform, this step introduces dual-pathway hierarchical coding to generate feature vectors. This step, based on the principle of separable modeling, treats frequency and scale information as input signals at two different levels, employing two independent neural pathways for feature encoding. Finally, a Graph Attention Network (GAT) is used to fuse these features, resulting in a clearly structured and semantically rich embedding vector for use by downstream modules.

[0084] Figure 2 This is a flow chart of step 102 provided in one embodiment of the present application. Figure 2 As shown, the process includes:

[0085] Step 201: scale-frequency dual-path feature encoding.

[0086] This application decomposes the image into multiple frequency components (subbands) through a discrete wavelet transform in step 101, and extracts statistical features (such as mean μ, standard deviation σ, energy E, and entropy H) from each subband. These statistical features describe the image's local brightness distribution, contrast, texture complexity, and detail. In step 201, using a separable modeling approach, frequency and scale features are treated as input signals at two different levels, and feature encoding is performed using two independent neural pathways.

[0087] In one embodiment, frequency feature encoding includes: constructing a frequency feature vector (such as mean, standard deviation, energy, and entropy) based on the statistical characteristics of each subband and encoding it. This process specifically includes the following steps:

[0088] 1) Construct the frequency feature vector of each sub-band based on all the statistical features of the sub-band.

[0089] For example, for each sub-band s and each sub-layer l, the mean μ, standard deviation σ, energy E, and entropy H are calculated to obtain a feature vector containing 4 elements:

[0090] (9)

[0091] The calculation formulas for the mean μ, standard deviation σ, energy E, and entropy H are specifically shown in formulas (2) to (5) above and will not be repeated here.

[0092] 2) The frequency feature vectors of all sub-bands are stacked to obtain a frequency feature sequence, and then the frequency feature sequence is encoded.

[0093] For example, the statistical vectors of all sub-bands are stacked into a frequency feature sequence (N is the total number of subbands), and then a one-dimensional convolutional neural network 1D CNN is used for in-channel modeling to achieve encoding:

[0094] (10)

[0095] Among them, Conv1D represents a one-dimensional convolution operation.

[0096] In one embodiment, scale feature encoding includes: performing global average pooling on low-frequency subbands and high-frequency subbands of each scale layer to obtain low-frequency features and high-frequency features, concatenating the low-frequency features and the high-frequency features to obtain scale features of each scale layer, and encoding the scale features of each scale layer.

[0097] For example, to obtain the scale information carried by the multi-layer wavelet decomposition, global average pooling is performed on the high-frequency sub-band and the low-frequency sub-band of each layer to form a richer scale structure representation vector, as shown in the following formula:

[0098] (11)

[0099] in Represents the set of wavelet high-frequency subbands at layer l (such as LH, HL, HH subbands), represents the low-frequency subband of layer l (such as LL subband), and GAP represents the global average pooling operation.

[0100] Then, the low-frequency features and high-frequency features obtained by global average pooling are spliced ​​together, and the spliced ​​features are encoded using a one-dimensional convolutional neural network. The splicing results and encoding results are shown in the following formulas:

[0101] (12)

[0102] in, For the The concatenation of high-frequency features and low-frequency features after global average pooling of the layer, Concatenate features from layer 1 to layer L The encoding result.

[0103] The feature encoding method provided in the embodiments of this application compresses the feature dimensions while preserving the structural relationship between subbands and providing structured input for the subsequent frequency / scale node construction of the heterogeneous graph. It should be noted that in addition to one-dimensional convolutional neural networks, other methods such as fully connected layers, self-attention mechanisms, or recurrent neural networks can also be used to encode frequency feature sequences or scale features.

[0104] Step 202: construct a heterogeneous graph based on the coding frequency feature and the coding scale feature.

[0105] To accurately model the complementary relationship between high-frequency details and global structure, this application designs a node connection strategy based on heterogeneous graphs. First, the encoded frequency features of high-frequency subbands (such as LH, HL, and HH subbands) at each scale layer are defined as frequency nodes, and the encoded scale features at each scale layer are defined as scale nodes. Based on this definition, frequency nodes are responsible for capturing local edges and textures in the image, while scale nodes are responsible for describing the global structure and brightness distribution at the corresponding level.

[0106] Based on the frequency nodes and scale nodes defined above, there are three types of edges connecting different types of nodes during the graph structure generation process:

[0107] 1) Frequency-frequency edge (FF): reflects the semantic similarity between different frequency sub-bands. The edge weight can be constructed by calculating the cosine similarity between each frequency node using the following formula:

[0108] (13)

[0109] Among them, f i and f j Used to represent different frequency nodes.

[0110] 2) Scale-scale edge (SS): reflects the hierarchical correlation between different scale layers and uses a fixed hierarchical structure to connect adjacent scale layers (such as the lth layer and the l+1th layer).

[0111] 3) Frequency-Scale Edge (FS): Connects frequency nodes with corresponding scale nodes within the same scale layer. This edge reflects the structural relationship between the subband and the scale layer to which it belongs.

[0112] For example, during graph structure generation, the following three connection principles can be followed: a) strong frequency-scale connections within the same layer, where each frequency node is connected to the scale node of the layer to which it belongs, so that the edges and texture features are directly associated with the semantic structure of that layer; b) weak cross-scale connections across layers, where only weak edges are established between scale nodes of adjacent layers to reduce cross-resolution interference and maintain semantic hierarchy; and c) weakening of nodes of the same type, where frequency-frequency or scale-scale nodes are not explicitly connected or are assigned extremely small weights (i.e., weights less than a threshold or approaching zero) to highlight the key interactions between frequency and scale. Finally, the constructed heterogeneous graph and its node features are input into a graph attention network, which adaptively aggregates neighbor information through edge weights to achieve a bidirectional fusion of frequency and scale semantics. The output graph embedding combines local details with global structure, laying the foundation for subsequent global modeling in the Transformer.

[0113] Step 203: The graph embedding vector is formed based on the attention weights of each node.

[0114] In order to achieve a more refined fusion of frequency and scale features, one embodiment of this application introduces a graph attention mechanism. By constructing a heterogeneous graph of frequency nodes and scale nodes, each node establishes a relationship with at least one other node. The graph attention mechanism can dynamically adjust the influence of each node. The specific steps are as follows:

[0115] 1) Apply the graph attention mechanism to calculate the attention weights between nodes.

[0116] Use the graph attention mechanism to learn the nodes in the heterogeneous graph constructed above and dynamically calculate the attention weights between nodes. The calculation formula is as follows:

[0117] (14)

[0118] in, is the set of neighbor nodes of node i (i.e. other nodes connected to the node), W is the shared weight matrix, σ is the nonlinear activation function, h j is the original feature vector of neighbor node j. is the attention weight between node i and node j, which is calculated by the self-attention mechanism:

[0119] (15)

[0120] Among them, d is the feature dimension, is the attention vector.

[0121] Through the self-attention mechanism of GAT, the relationships and importance between nodes can be modeled more accurately.

[0122] 2) Update the features of each node based on the attention weight and fuse them to obtain the graph embedding vector.

[0123] After processing by the GAT layer, the features of each node will be weighted and updated, and finally a new graph embedding vector will be obtained. :

[0124] (16)

[0125] above It is the fused feature representation, which contains information about frequency and scale features; is the initial feature of each node.

[0126] Finally, the feature vector after fusion of the graph attention mechanism This is passed as input to the Transformer for global modeling. Based on this, subsequent Transformers will be able to process high-level image representations, further improving the performance of the model.

[0127] Step 103: Map the image embedding vector to the Transformer input dimension to obtain a global feature vector output by the Transformer that represents the overall quality of the image.

[0128] This step primarily relies on the Transformer to generate a global feature vector that characterizes the overall image quality. The Transformer is a deep learning model architecture used for natural language processing and other sequence-to-sequence tasks. This architecture incorporates a self-attention mechanism, which enables it to excel at processing sequential data. The Transformer typically includes the following key components: a self-attention mechanism, positional encoding, an encoder, residual connections, and layer normalization.

[0129] Figure 3 This is a flow chart of generating a global feature vector based on Transformer provided by the embodiment of the present application. Figure 3 As shown, the process includes:

[0130] Step 301: linearly map the graph embedding vector to the Transformer input dimension, and fuse the spatial position, frequency information, and scale information for encoding.

[0131] In order to enable the Transformer to process image data, the graph embedding vector output in step 102 is first divided into image patches and converted into a node embedding vector of fixed dimension through linear mapping.

[0132] Assume that the input local feature vector is , converted to node embedding vector through linear mapping , the formula is:

[0133] (17)

[0134] in is the mapping matrix, For paranoid items, Indicates the number of patches.

[0135] In order to enable the Transformer model to distinguish the spatial position, frequency information and scale information of different patches, a position encoding method that integrates frequency information and scale information is introduced. Specifically: for each patch, the node embedding vector , add position coding ,in Can be determined by spatial position and frequency index and scale factor Together they constitute:

[0136] (18)

[0137] in, It is the standard sine-cosine position encoding; and are the encoding of frequency and scale information after linear or nonlinear mapping, , is the adjustment coefficient.

[0138] Then, the position encoding is added to the node embedding vector to obtain the fused input vector sequence:

[0139] (19)

[0140] Step 302: Introduce a scale weight matrix into the standard self-attention mechanism to enhance the semantic association between nodes of the same scale.

[0141] The traditional self-attention mechanism can calculate the correlation between nodes, but it cannot distinguish patches from different scales or frequency bands. This application introduces additional scale weights in the self-attention mechanism to encourage higher correlation between the same scales. To this end, the improved Transformer encoder layer is used to process the input sequence. Perform global modeling and use the self-attention mechanism to capture long-range dependencies and cross-scale associations between different patches.

[0142] The traditional self-attention calculation formula is:

[0143] (20)

[0144] Among them, Q, K, and V are three input representation vectors, Q represents the query vector, K represents the key vector, V represents the numerical vector, and D is the calculation parameter.

[0145] After the scale matrix (i.e., cross-scale weight S) is introduced in this application, the attention score is updated as follows:

[0146] (twenty one)

[0147] Among them, S can be calculated from the scale information of each patch, ensuring that patches from similar scales or frequency bands receive higher attention weights, that is, the attention weight is positively correlated with the similarity of scale or frequency band.

[0148] Step 303: stack multiple layers of Transformer encoders to output a global feature sequence, and perform average pooling on the global feature sequence to generate a global feature vector that can characterize the quality of the entire image.

[0149] By stacking multiple layers of Transformer encoders (each layer contains cross-scale self-attention and feedforward networks), a fused global feature sequence is finally generated. The sequence output by the encoder is average pooled to obtain the global quality feature vector of the entire image. :

[0150] (twenty two)

[0151] in Represents the features output by the Transformer encoder. This global feature will serve as the input for subsequent quality evaluation.

[0152] So far, the Figure 3 It should be noted that, except for Figure 3In addition to the average pooling method used in

[15] to generate global feature vectors, other methods, such as maximum pooling and self-attention aggregation, can also be used based on the global feature sequence output by the Transformer. Relatively speaking, average pooling treats all nodes equally, is computationally efficient, and is suitable for situations where node importance is evenly distributed. Maximum pooling captures the most significant distortion features and suppresses background noise, making it suitable for situations where localized severe distortion dominates quality evaluation. Self-attention aggregation can dynamically learn node importance weights and focus on key distortion areas, making it suitable for situations where complex distortion distributions require differentiated attention.

[0153] Figure 4 This is a schematic diagram of a Transformer architecture provided by an embodiment of the present application. Figure 4 As shown, It is a multi-scale, multi-band feature extracted by wavelet transform; Patch Embedding refers to dividing the image into patches and converting each patch into a vector of fixed dimension; Positional Encoding fuses space, frequency and scale information so that each node carries complete position-frequency-scale information; Cross-Scale Self-Attention uses an improved self-attention mechanism to capture the connection between patches of different scales and frequency bands on a global scale; Global feature aggregation obtains the global quality feature representation F of the entire image through global pooling, providing stable and accurate input for subsequent quality regression. Transformer global modeling fully combines the multi-scale and multi-band features extracted by wavelet transform, realizes global information integration through cross-scale self-attention mechanism, and provides powerful global feature modeling capabilities for no-reference image quality evaluation. In some embodiments, the multi-scale and multi-band features extracted by wavelet transform in step 101 can also be used as the input for the image quality evaluation. Conduct with After similar processing, it is used as the input of the Transformer. As a relatively simplified implementation method, this method can still improve the ability to capture and model global information to a certain extent compared with existing technologies, and enhance the ability to recognize global structural distortions.

[0154] Step 104: Map the global feature vector to an image quality score using a linear regression network.

[0155] For example, the mapping formula used in this step is as follows:

[0156] (twenty three)

[0157] in, is the predicted image quality score; is the regression weight; is the bias term. In addition to the formula (23) provided in this example, other linear equations that can achieve image quality score mapping can also be used.

[0158] In one embodiment, to make the predicted quality score and true quality score To ensure the best possible accuracy, the mean square error (MSE) can be used as the loss function. During training, the Adam optimization algorithm can be used to update the regression network parameters to minimize the loss function.

[0159] So far, the Figure 1 It should be understood that the order of execution of the steps in the above embodiments does not necessarily imply a specific order of execution, and the order of execution of each process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0160] based on Figure 1 The process shown in the figure is that this application provides a no-reference image quality assessment method based on discrete wavelet transform, frequency-scale dual-path coding, and graph structure fusion. It is mainly used for automated detection and evaluation of image quality (including ordinary images and hyperspectral images). This method overcomes the problems of traditional statistical characteristics and hand-designed perception models, such as insufficient local feature representation, limited global information modeling capabilities, and limited generalization capabilities. It also addresses the problem of insufficient global distortion capture caused by the local receptive field limitations of CNNs in deep learning methods. By leveraging the frequency decomposition capabilities of the discrete wavelet transform, the global modeling capabilities of the Transformer, and the ability of the graph attention network to perceive the nonlinear interaction between frequency and scale, a frequency-scale dual-path structure is constructed. A graph structure attention fusion mechanism is introduced to deeply jointly model the statistical features of the image at different frequency bands and scales. This effectively improves the accuracy and robustness of the perception of complex image distortion patterns, thereby achieving efficient and accurate no-reference image quality assessment.

[0161] Specifically, the main innovations of this application are as follows:

[0162] 1. Introducing a multi-channel frequency-scale encoding mechanism: This application decouples the frequency features obtained after wavelet transform from the multi-scale structural features to build frequency channels and scale channels, respectively extracting the image expression information at different sub-bands and resolutions, effectively improving the fine-grainedness and structural sensitivity of the feature representation.

[0163] 2. Constructing a frequency-scale heterogeneous graph structure and introducing a graph attention fusion mechanism: This application uses frequency paths and scale paths as heterogeneous graph nodes, and implements relationship modeling and feature fusion between nodes through graph neural networks. It adaptively captures the nonlinear interactions between different frequency components and scale structures, breaking through the limitations of traditional feature splicing and weighted average fusion methods.

[0164] 3. Improving the cross-scale modeling capability of the Transformer model: This application designs a patch embedding strategy that combines frequency position encoding and scale-aware position encoding, and introduces a cross-scale self-attention mechanism in the Transformer module. This enables the model to simultaneously focus on the contextual information of the image at multiple frequencies and scales, enhancing the ability to recognize global structural distortions.

[0165] 4. Establish an end-to-end reference-free image quality assessment system: The model proposed in this application does not rely on the original undistorted image as a reference. It can directly score the quality of the input image under various distortion conditions such as compression, blur, and noise, and has greater practicality, versatility, and deployment value.

[0166] 5. Universally applicable to both standard and hyperspectral image quality assessment scenarios: The designed frequency-scale joint structure has strong transferability and can be extended to various image types, including but not limited to natural images, remote sensing images, and hyperspectral images, laying the foundation for subsequent cross-modal image perception.

[0167] Corresponding to the no-reference image quality assessment method described in the above embodiment, Figure 5 A structural block diagram of a non-reference image quality assessment device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0168] Reference Figure 5 , the device comprises:

[0169] The wavelet transform feature extraction module 501 is used to perform multi-scale layer decomposition of the target image through discrete wavelet transform, where each scale layer includes a low-frequency sub-band and a high-frequency sub-band, and extract statistical features for each sub-band;

[0170] Frequency-scale coding and graph structure fusion module 502 is used to convert the statistical features of each subband into structured coded frequency features and coded scale features through scale-frequency dual-path feature coding; define frequency nodes and scale nodes based on the coded frequency features and coded scale features, connect the frequency nodes and scale nodes to form a heterogeneous graph, and use the graph attention network to calculate the attention weight of each node and fuse them to form a graph embedding vector;

[0171] Transformer global modeling module 503, which is used to map the graph embedding vector to the Transformer input dimension to obtain the global feature vector of the Transformer output that represents the overall quality of the image; the Transformer position encoding integrates frequency information and scale information, and the self-attention mechanism integrates the scale weight matrix;

[0172] The quality assessment and regression module 504 is configured to map the global feature vector into an image quality score using a linear regression network.

[0173] Furthermore, the device may also include an image preprocessing module (not shown in the figure) for performing preprocessing such as geometric correction and noise removal on the target image, and the wavelet transform feature extraction module 501 performs wavelet decomposition on the preprocessed target image.

[0174] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units / modules are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0175] To facilitate understanding of the solution provided by this application, a detailed description is given below in conjunction with an embodiment:

[0176] 1. Image preprocessing: This module uses geometric correction and noise removal methods to preprocess the image.

[0177] 1.1 Geometric Correction: Assumptions is a pixel position in the original image, and the new position is obtained after correction by the transmission transformation f :

[0178] (6)

[0179] 1.2 Noise Removal: Using Gaussian Kernel Perform convolution smoothing on the image, and obtain the smoothed image after convolution operation , as input for subsequent processing:

[0180] (7)

[0181] 2. Wavelet transform feature extraction: This module uses discrete wavelet transform to preprocess the image Perform multi-layer decomposition to capture image features at different scales and frequency bands. In this example, a three-layer wavelet decomposition is used, with each layer generating four subbands: LL, LH, HL, and HH. Statistical features are then calculated for each subband and concatenated in a fixed order to form the final feature vector.

[0182] 2.1 Wavelet decomposition: using wavelet basis The image is decomposed into three layers, and the decomposition formula can be expressed as:

[0183] (1)

[0184] in is the wavelet function after translation and scaling at scale j, is the corresponding wavelet coefficient. After decomposition, we get:

[0185] First layer:

[0186] Second layer: Yes Decompose in sequence to get

[0187] The third layer: Decompose in sequence to get

[0188] 2.2 Subband feature extraction: For each subband , extract the following statistical features:

[0189] Mean: (2)

[0190] Standard Deviation: (3)

[0191] energy: (4)

[0192] entropy: (5)

[0193] 3. Frequency-scale dual-path feature encoding and graph structure fusion: This module treats frequency domain features and scale information as two input signals at different levels, and uses two independent neural pathways for feature encoding. It then uses a graph attention network to perform feature fusion, thereby constructing a clearly structured and semantically rich embedding vector for use by downstream modules.

[0194] 3.1 Frequency Domain Feature Extraction and Coding

[0195] Calculate frequency domain features: For each sub-band s and each sub-layer l, calculate its mean, standard deviation, energy, and entropy to obtain a feature vector containing 4 elements:

[0196] (9)

[0197] Stacked frequency domain features: stack the statistical vectors of all subbands into a frequency domain feature sequence (N is the total number of subbands), and then a one-dimensional convolutional neural network (1D CNN) is used for intra-channel modeling:

[0198] (10)

[0199] 3.2 Scale feature extraction and encoding

[0200] Global average pooling is performed on the high-frequency sub-band and low-frequency sub-band of each layer to form a richer scale structure representation vector:

[0201] (11)

[0202] Then the high-frequency features are globally pooled and concatenated and encoded:

[0203] (12)

[0204] 3.3 Heterogeneous Graph Construction Guided by Subband Semantics

[0205] Heterogeneous graph node design: Frequency nodes come from F freq The high-frequency subband feature vector of each row in is responsible for capturing the local edge and texture of the image; the scale node comes from F scale The scale feature vector of each row in is responsible for describing the global structure and brightness distribution of the corresponding level.

[0206] Edge connection rules are designed as follows: 1) Strong frequency-scale connections within the same layer, that is, each frequency node is connected to the scale node of the layer to which it belongs, so that the edge and texture features are directly associated with the semantic structure of the layer; 2) Weak connections across scales, that is, only weak edges are established between scale nodes of adjacent layers to reduce cross-resolution interference and maintain semantic hierarchy; 3) Weak processing of nodes of the same type, that is, frequency-frequency or scale-scale nodes are not explicitly connected or are given extremely small weights to highlight the key interactions between frequency and scale.

[0207] 3.4 Graph Attention Mechanism Feature Fusion:

[0208] Applying the graph attention mechanism: The graph attention mechanism is used to learn the nodes in the graph. The attention weights between nodes are dynamically calculated by GAT. The formula is as follows:

[0209] (14)

[0210] in, is the attention weight between node i and node j, which is calculated by the self-attention mechanism:

[0211] (15)

[0212] Through the self-attention mechanism of GAT, the relationships and importance between nodes can be modeled more accurately.

[0213] Feature fusion: After processing by the GAT layer, the features of each node will be weighted and updated to finally obtain a new graph embedding vector :

[0214] (16)

[0215] above It is the fused feature representation, which contains information about frequency and scale features.

[0216] The constructed heterogeneous graph and its node features are input into the Graph Attention Network (GAT), and neighbor information is adaptively aggregated through edge weights to achieve a bidirectional fusion of frequency and scale semantics. The output graph embedding has both local details and global structure, laying the foundation for the subsequent Transformer global modeling.

[0217] 4. Transformer global modeling module: This module embeds the graph into a vector It is converted into a vector sequence of fixed dimension and globally modeled through specially designed frequency position encoding and cross-scale self-attention mechanism, finally generating a global feature vector describing the quality of the entire image.

[0218] 4.1PatchEmbedding: Embedding Graphs into Vectors Divided into multiple local patches, each patch has the following features Converted to an embedding vector through linear mapping :

[0219] (17)

[0220] 4.2 Frequency Position Encoding: In order to preserve the spatial position information of each patch and the frequency and scale information inherent in the wavelet decomposition, position encoding is added to each embedding vector:

[0221] (18)

[0222] Finally, the fused input sequence is obtained:

[0223] (19)

[0224] 4.3 Cross-scale self-attention mechanism: The improved Transformer encoder is used to globally model the input sequence. The self-attention calculation formula with the scale weight matrix S is:

[0225] (twenty one)

[0226] 4.4 Transformer encoder and global feature output: After passing through the multi-layer Transformer encoder, the output of all patches is average pooled to obtain the global feature vector :

[0227] (twenty two)

[0228] 5. Quality evaluation and regression module: This module uses a simple linear regression network to map the global feature vector output by Transformer into an image quality score. Specifically, a single-layer linear mapping is used to convert F into a quality score. :

[0229] (twenty three)

[0230] Loss function: During training, the mean square error (MSE) is used as the loss function, where B is the number of samples:

[0231] (twenty four)

[0232] As a simplified implementation method, the output of step 2, i.e., the multi-scale and multi-band features extracted by wavelet transform, can be converted into , after After similar processing, it is used as the input of Transformer (that is, the multi-scale and multi-band features extracted by wavelet transform Converted into a vector sequence of fixed dimensions), thereby simplifying the implementation steps and increasing the calculation speed. Compared with the existing technology, it can still improve the ability to capture and model global information to a certain extent and enhance the ability to recognize global structural distortion.

[0233] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0234] An embodiment of the present application also provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the computer program.

[0235] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned various method embodiments can be implemented.

[0236] An embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device can implement the steps of the above-mentioned method embodiments when executing the computer program product.

[0237] Figure 6 This is a schematic diagram of the structure of a computer device provided in one embodiment of the present application. Figure 6 As shown, the computer device of this embodiment includes: at least one processor 60 ( Figure 6 Only one is shown), a memory 61 and a computer program 62 stored in the memory 61 and executable on the at least one processor 60, wherein the processor 60 implements the steps of any of the above-mentioned visual programming method embodiments when executing the computer program 62.

[0238] The computer device may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will appreciate that Figure 6 The computer device is merely an example and does not constitute a limitation on the computer device. The computer device may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the computer device may also include input and output devices, network access devices, etc.

[0239] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0240] In some embodiments, the memory 61 may be an internal storage unit of the computer device, such as a hard drive or memory of the computer device. In other embodiments, the memory 61 may also be an external storage device of the computer device, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped with the computer device. Furthermore, the memory 61 may include both an internal storage unit of the computer device and an external storage device. The memory 61 is used to store an operating system, application programs, a boot loader, data, and other programs, such as the program code of the computer program. The memory 61 may also be used to temporarily store data that has been output or is about to be output.

[0241] If the integrated unit is implemented as a software functional unit and sold or used as a standalone product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process steps in the above-mentioned method embodiments by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by a processor, the computer program can implement the steps of each of the above-mentioned method embodiments. The computer program includes computer program code, which can be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / computer equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, removable hard drives, magnetic disks, or optical disks. In some jurisdictions, based on legislation and patent practice, computer-readable media cannot be electric carrier signals or telecommunication signals.

[0242] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0243] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0244] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer equipment and methods can be implemented in other ways. For example, the apparatus / computer equipment embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of the apparatus or unit, which can be electrical, mechanical or other forms.

[0245] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0246] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A method for evaluating image quality without reference, characterized in that: include: The target image is decomposed into multiple scale layers by discrete wavelet transform. Each scale layer includes low-frequency sub-bands and high-frequency sub-bands, and statistical features are extracted for each sub-band. Through scale-frequency dual-path feature encoding, the statistical features of each subband are converted into structured coded frequency features and coded scale features; frequency nodes and scale nodes are defined based on the coded frequency features and the coded scale features, and the frequency nodes and scale nodes are connected to form a heterogeneous graph. The attention weight of each node is calculated using a graph attention network and fused to form a graph embedding vector; Mapping the image embedding vector to the Transformer input dimension to obtain a global feature vector output by the Transformer that represents the overall image quality; the Transformer's positional encoding fuses frequency information and scale information, and the self-attention mechanism fuses the scale weight matrix; Mapping the global feature vector to an image quality score using a linear regression network; The method of converting the statistical features of each subband into structured coding frequency features and coding scale features through scale-frequency dual-path feature coding, defining frequency nodes and scale nodes according to the coding frequency features and the coding scale features, and connecting the frequency nodes and the scale nodes to form a heterogeneous graph includes: Through scale-frequency dual-path feature coding, a frequency feature vector of each sub-band is constructed based on all statistical features of the sub-band, the frequency feature vectors of all sub-bands are stacked to obtain a frequency feature sequence, and the frequency feature sequence is encoded to obtain a coded frequency feature; Performing global average pooling on the low-frequency subband and the high-frequency subband of each scale layer to obtain low-frequency features and high-frequency features, concatenating the low-frequency features and the high-frequency features to obtain scale features of each scale layer, and encoding the scale features of each scale layer to obtain encoded scale features; The coded frequency features of each high frequency sub-band are defined as frequency nodes, and the coded scale features of each scale layer are defined as scale nodes; Scale nodes of adjacent scale layers are connected to form scale-scale edges, and frequency nodes and scale nodes within the same scale layer are connected to form frequency-scale edges. Nodes of different frequencies are not explicitly connected or are assigned weights less than a threshold.

2. The method according to claim 1, wherein Each scale layer includes a low-frequency sub-band and three high-frequency sub-bands; the low-frequency sub-band is the LL sub-band, which contains the overall structural information of the image; the high-frequency sub-band includes the LH sub-band, the HL sub-band and the HH sub-band. The LH sub-band reflects the edge features of the image in the horizontal direction, the HL sub-band reflects the edge features of the image in the vertical direction, and the HH sub-band contains the texture and detail information of the image in the diagonal direction.

3. The method according to claim 1, wherein The extracting statistical features for each sub-band includes: For each subband, four statistical features are extracted: mean μ, standard deviation σ, energy E, and entropy H.

4. The method according to claim 1, wherein Mapping the graph embedding vector to the Transformer input dimension includes: The graph embedding vector is divided into multiple small patches, and each patch is converted into a node embedding vector of fixed dimension through linear mapping.

5. The method according to claim 4, wherein The Transformer's positional encoding fuses frequency information and scale information, including: Generate a position code based on the spatial position, frequency information, and scale information of each patch, and add the position code to the node embedding vector to obtain a fused input vector; The self-attention mechanism fusion scale weight matrix includes: A scale matrix is ​​added to the self-attention calculation formula. The scale matrix is ​​calculated based on the scale information of each patch, and the attention weight is positively correlated with the similarity of the scale or the similarity of the frequency band.

6. A non-reference image quality assessment device, characterized in that: include: The wavelet transform feature extraction module is used to decompose the target image into multiple scale layers through discrete wavelet transform. Each scale layer includes low-frequency sub-bands and high-frequency sub-bands, and statistical features are extracted for each sub-band. The frequency-scale coding and graph structure fusion module is used to convert the statistical features of each subband into structured coded frequency features and coded scale features through scale-frequency dual-path feature coding; define frequency nodes and scale nodes based on the coded frequency features and the coded scale features, connect the frequency nodes and the scale nodes to form a heterogeneous graph, and use the graph attention network to calculate the attention weight of each node and fuse them to form a graph embedding vector; A Transformer global modeling module is used to map the graph embedding vector to the Transformer input dimension to obtain a global feature vector output by the Transformer that represents the overall image quality. The Transformer's positional encoding integrates frequency and scale information, and the self-attention mechanism integrates the scale weight matrix. A quality assessment and regression module, configured to map the global feature vector into an image quality score using a linear regression network; The frequency feature vector of each sub-band is constructed based on all statistical features of the sub-band through scale-frequency dual-path feature coding, the frequency feature vectors of all sub-bands are stacked to obtain a frequency feature sequence, and the frequency feature sequence is encoded to obtain a coded frequency feature; Performing global average pooling on the low-frequency subband and the high-frequency subband of each scale layer to obtain low-frequency features and high-frequency features, concatenating the low-frequency features and the high-frequency features to obtain scale features of each scale layer, and encoding the scale features of each scale layer to obtain encoded scale features; The coded frequency features of each high frequency sub-band are defined as frequency nodes, and the coded scale features of each scale layer are defined as scale nodes; Scale nodes of adjacent scale layers are connected to form scale-scale edges, and frequency nodes and scale nodes within the same scale layer are connected to form frequency-scale edges. Nodes of different frequencies are not explicitly connected or are assigned weights less than a threshold.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

Images