A no-reference image usability evaluation method based on hierarchical feature fusion
By employing hierarchical feature fusion and adaptive feature weight adjustment, this method addresses the issues of limited image quality assessment dimensions and insufficient semantic quantization in existing technologies. It enables accurate usability assessment of images in high-level vision tasks and is applicable to autonomous driving and intelligent monitoring systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG UNIV OF SCI & TECH
- Filing Date
- 2026-03-06
- Publication Date
- 2026-05-15
AI Technical Summary
Existing no-reference image quality assessment methods have a single assessment dimension, lack quantification of the completeness of image semantic content, and have rigid feature fusion strategies that cannot adapt to different image content, resulting in a weakened correlation with performance on high-level vision tasks.
A hierarchical feature fusion method is adopted, which evaluates low-level image quality, mid-level structure and high-level semantic features in a multi-dimensional way. The hierarchical attention fusion device is used to adjust the feature weights adaptively to build a pixel-scene-object evaluation system. Combined with features such as frequency domain detail clarity, segmentation quality and target integrity, an image usability evaluation score is generated.
It enables accurate usability assessment of images in high-level vision tasks, improving the accuracy and generalization ability of the assessment. It is applicable to systems such as autonomous driving and intelligent monitoring, and can dynamically adjust the feature contribution weights to adapt to different scenario requirements.
Smart Images
Figure CN121810674B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision and image processing technology, and specifically relates to a method for evaluating the usability of no-reference images based on hierarchical feature fusion. Background Technology
[0002] With the development of applications such as autonomous driving and intelligent monitoring, images have become a core data source for machines to perceive their environment. Against this backdrop, the focus of image analysis has shifted from "human visual review" to "machine visual interpretation," and evaluation standards need to evolve accordingly. Traditional no-reference image quality assessment (NR-IQA) methods, such as BRISQUE, NIQE, and deep learning-based methods like DBCNN and HyperIQA, aim to simulate the human visual system (HVS) to evaluate pixel-level distortions in images, such as blur, noise, and compression artifacts. Their optimization objective is highly correlated with human subjective opinion scores (MOS).
[0003] However, these methods reveal a significant "semantic gap" when faced with high-level vision tasks: an image that scores highly on PSNR or SSIM may be completely ineffective in object detection tasks due to the occlusion of key targets; conversely, an image with slight motion blur but complete semantics, although scoring low on traditional IQA, can still effectively support pose estimation tasks. This indicates that evaluating only low-level image quality is insufficient to reflect the "usability" of an image for high-level tasks.
[0004] The existing technology has the following main problems: (1) the evaluation dimension is single and has a weak correlation with the performance of high-level tasks; (2) there is a lack of quantitative evaluation of the integrity of the semantic content of the image; (3) the feature fusion strategy is rigid and cannot be adaptively adjusted according to the image content.
[0005] A few studies have attempted to introduce content awareness, but most still use human subjective quality as the supervision target, failing to systematically establish a direct quantitative correlation with the performance of high-level visual tasks. Moreover, feature fusion often uses fixed weights or simple splicing, resulting in limited generalization ability. Summary of the Invention
[0006] To overcome the "semantic gap" problem in existing no-reference image quality assessment methods, this invention provides a no-reference image usability assessment method based on hierarchical feature fusion. The proposed method is capable of accurately assessing the actual usability of images in high-level vision tasks, and is particularly suitable for evaluating the actual usability of images in high-level vision tasks such as object detection, semantic segmentation, and pose estimation.
[0007] The technical solution of the present invention is as follows:
[0008] I. A No-Reference Image Usability Evaluation Method Based on Hierarchical Feature Fusion
[0009] S1: Obtain the image to be evaluated;
[0010] S2: Perform hierarchical feature extraction on the image to be evaluated to obtain low-level image quality feature vector, mid-level structure feature vector and high-level semantic feature vector respectively;
[0011] S3: Normalize the low-level image quality feature vector, the mid-level structure feature vector, and the high-level semantic feature vector respectively, and then concatenate all the normalized feature vectors into a total feature vector (i.e., multi-dimensional features).
[0012] S4: Input the total feature vector into a pre-trained image usability prediction model to obtain the usability evaluation score of the image to be evaluated; wherein, the image usability prediction model is a regression network integrating a hierarchical attention fusion unit, used to adaptively weight and fused and predict the input total feature vector.
[0013] The low-level image quality feature vector includes frequency domain detail sharpness features and a score corresponding to the degree of deviation from the prior distribution of natural images. Specifically, it is obtained through the following methods:
[0014] Extract the frequency domain detail sharpness features (such as high-frequency energy based on wavelet transform) of the image to be evaluated and generate the corresponding score;
[0015] The features of the image to be evaluated based on Natural Scene Statistics (NSS) (such as those obtained through BRISQUE or NIQE models) are extracted, and the degree to which they deviate from the prior distribution of natural images is calculated, thereby generating a corresponding score.
[0016] The mid-level structural feature vector includes a segmentation quality score, a global structural distortion score, and a global quality-perceived score. Specifically, it is obtained through the following methods:
[0017] The segmentation mask of the image to be evaluated is obtained using an image segmentation model (such as the Segment Anything Model), and the segmentation quality score is calculated based on the edge sharpness, region consistency and structural rationality of the segmentation mask.
[0018] The global structure-aware features of the image to be evaluated are extracted using a deep convolutional neural network model (such as one based on HyperIQA), and a global structure distortion score is output based on these global structure-aware features.
[0019] The global long-range dependency features of the image to be evaluated are extracted using a visual Transformer model (such as ViT-IQA), and a global quality-perceived score is output based on these global long-range dependency features.
[0020] The high-level semantic feature vector includes target integrity score and pose integrity score. It is obtained through the following method:
[0021] The target detection model (such as YOLOv8-based keypoint detection) is used to identify the preset category targets (such as human bodies) in the image to be evaluated, and the target integrity score is calculated based on the ratio of the number of detected valid keypoints to the preset total number of keypoints.
[0022] The pose of the preset category targets is identified using a pose estimation model (such as YOLOv8-pose), and a pose integrity score is calculated based on the proportion of high-confidence keypoints.
[0023] The image availability prediction model comprises a linear projection layer, a hierarchical attention fusion layer, and a regression output layer connected in sequence.
[0024] The linear projection layer is used to project the total feature vector into the feature space, that is, to project the total feature vector into a high-dimensional feature space.
[0025] The regression output layer is used to map the fused feature vector output by the hierarchical attention fusion processor to the final usability evaluation score.
[0026] The hierarchical attention fusion processor includes at least one encoder layer based on multi-head self-attention.
[0027] The multi-head self-attention encoder layer is the core component of the hierarchical attention fusion unit. It consists of a multi-head self-attention sublayer and a feedforward neural network sublayer, employing a cascaded design of sublayer-level residual connections and layer normalization. That is, the input of each encoder layer serves as the input of the multi-head self-attention sublayer. The output of the multi-head self-attention sublayer is residually connected to its own original input before layer normalization is performed. The result of layer normalization serves as the input of the feedforward neural network sublayer. The output of the feedforward neural network sublayer is then residually connected to its own original input before layer normalization is performed again, finally yielding the output of the encoder layer.
[0028] II. A No-Reference Image Usability Assessment System Based on Hierarchical Feature Fusion
[0029] The hierarchical feature extraction module is used to receive the input image and extract the low-level image quality feature vector, the mid-level structural feature vector, and the high-level semantic feature vector of the input image in parallel.
[0030] The feature preprocessing module is used to normalize and concatenate the low-level image quality feature vector, the mid-level structure feature vector, and the high-level semantic feature vector to generate a total feature vector.
[0031] The usability prediction module has a pre-trained image usability prediction model built in, which is used to receive the total feature vector and output the usability evaluation score of the input image.
[0032] III. An electronic device
[0033] The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the aforementioned method for evaluating the usability of a no-reference image based on hierarchical feature fusion.
[0034] IV. A computer-readable storage medium
[0035] The medium stores a computer program, which, when executed by a processor, implements the steps of the aforementioned method for evaluating the usability of a no-reference image based on hierarchical feature fusion.
[0036] The beneficial effects of this invention are as follows:
[0037] 1. Fundamentally solves the semantic gap: This invention pioneers a three-layer evaluation system of "pixel-scene-object", which integrates low-level image quality, mid-level structure and high-level semantic features. It builds a collaborative bridge between features at each level through a hierarchical attention fusion device, so that the evaluation target is directly linked to the performance of high-level visual tasks, filling the evaluation gap of traditional IQA.
[0038] 2. Dynamic Adaptive Fusion: The hierarchical attention fusion unit introduced in this invention serves as the core of dynamic feature weight allocation for content awareness. Based on the Transformer framework, it can align heterogeneous feature information in the feature space and mine synergistic effects according to the different distortion types and semantic content completeness in the input image. It adaptively adjusts the contribution weights of low-level image quality, mid-level structure, and high-level semantic features to achieve task-oriented intelligent fusion. Compared with static fusion strategies, it has stronger generalization ability and scene robustness.
[0039] 3. Significantly improved evaluation accuracy: Experiments on public datasets show that the correlation coefficient (PLCC / SRCC) between the prediction score and the high-level task performance of the method of this invention is significantly better than all mainstream NR-IQA methods. Ablation experiments confirm the necessity of hierarchical attention fusion and features at each level.
[0040] 4. Strong practicality and scalability: The evaluation score of this invention intuitively reflects the "usability" of an image to the machine, and can be directly used for image screening in systems such as autonomous driving and intelligent monitoring. The framework designed in this invention is universal, and the high-level semantic target can be extended from "human body" to multiple categories such as vehicles and machinery. The heterogeneous feature processing capability of the hierarchical attention fusion unit can adapt to the needs of different extended scenarios. Attached Figure Description
[0041] Figure 1 This is a flowchart of the method of the present invention.
[0042] Figure 2 A schematic diagram of generating the total feature vector of the input image.
[0043] Figure 3 This is a network diagram of a hierarchical attention fusion processor.
[0044] Figure 4 The system framework of this invention Figure 1 .
[0045] Figure 5 The system framework of this invention Figure 2 . Detailed Implementation
[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the following embodiments are for illustrative purposes only and are not intended to limit the invention.
[0047] like Figure 1 As shown, the referenceless image usability assessment method based on hierarchical feature fusion proposed in this invention includes the following steps:
[0048] S1: Obtain the image I to be evaluated;
[0049] S2: As Figure 2 As shown, hierarchical feature extraction is performed on the image to be evaluated, obtaining low-level image quality feature vectors, mid-level structural feature vectors, and high-level semantic feature vectors. The low-level image quality feature vector aims to quantify the basic visual fidelity of the image, the mid-level structural feature vector aims to evaluate the scene structural integrity of the image, and the high-level semantic feature vector aims to quantify the integrity and recognizability of key semantic content in the image.
[0050] The low-level image quality feature vector includes frequency domain detail sharpness features and a score corresponding to the degree of deviation from the prior distribution of natural images. Specifically, it is obtained through the following methods:
[0051] 1. Extract the frequency domain detail sharpness features of the image I to be evaluated and generate the corresponding score; for example, perform a two-dimensional discrete wavelet transform (2D-DWT) on image I to obtain the low-frequency approximate component CA and three high-frequency detail sub-band matrices CH, CV, and CD (horizontal, vertical, and diagonal). Then, quantify the detail sharpness S by calculating the average energy of the three high-frequency sub-band matrices. freq :S freq=(1 / N)×Σ(|CH|+|CV|+|CD|), where N is the total number of elements, || represents the absolute value operation on all elements in the matrix, and Σ represents the summation operation on all elements in the matrix within the parentheses. The average energy S of the three high-frequency subbands is... freq The score S, denoted as the frequency domain detail sharpness feature, is WSFR .
[0052] 2. Extract features of the image to be evaluated based on Natural Scene Statistics (NSS) (e.g., obtained through BRISQUE or NIQE models), calculate the degree to which they deviate from the prior distribution of natural images, and then generate the corresponding score. Specifically:
[0053] First, the mean of the image to be evaluated, I, is subtracted from the contrast normalization (MSCN) coefficient to extract its statistical moment features (mean, variance, skewness, and kurtosis). These statistical moment features are then processed using pre-trained BRISQUE and NIQE models, respectively. The BRISQUE model outputs a quality score S through support vector regression (SVR). BRISQUE The NIQE model calculates the Mahalanobis distance between the statistical moment features of the image to be evaluated (I) and the features of the natural image model, and outputs a distance score (S). NIQE The greater the distance, the worse the image quality.
[0054] Low-level image quality feature vector f low =[S BRISQUE ,S NIQE ,S WSFR ].
[0055] The mid-level structure feature vector includes segmentation quality score, global structural distortion score, and global quality-perceived score. Specifically, it is obtained through the following methods:
[0056] 1. Obtain the segmentation mask of the image to be evaluated using an image segmentation model (such as the Segment Anything Model), and calculate the segmentation quality score based on the edge sharpness E, region consistency R, and structural rationality O of the segmentation mask, i.e., perform a weighted summation, S SAM =w1E+w2R+w3O, where w1, w2, and w3 are three weights, and in this embodiment, each is taken as 1 / 3.
[0057] 2. Utilize a deep convolutional neural network model (such as HyperIQA) to extract global structure-aware features of the image to be evaluated, and output a global structure distortion score based on these features. Specifically, HyperIQA extracts and aggregates multi-scale features through a deep CNN, ultimately outputting a perceptual quality score S. HyperIQA This score is sensitive to global structural distortion.
[0058] 3. Utilize visual Transformer models (such as ViT-IQA) to extract global long-range dependency features of the image to be evaluated, and output a global quality-perceived score based on these features. Specifically, ViT-IQA divides the image into blocks and then models the global inter-block relationships through a self-attention mechanism, thereby outputting a quality score S that is sensitive to macroscopic structural anomalies. ViT .
[0059] The mid-level structure feature vector f mid =[S SAM ,S HyperIQA ,S ViT ].
[0060] The high-level semantic feature vector includes target integrity score and pose integrity score.
[0061] In this embodiment, the human body is used as the detection target, and the high-level semantic feature vector is obtained through the following method:
[0062] 1. Use an object detection model (such as YOLOv8-based keypoint detection) to identify the human body in the image to be evaluated, and output the human body bounding box and keypoints (e.g., 17 standard human keypoints). Count the number N of valid keypoints with a confidence level exceeding a threshold τ (e.g., 0.3). valid The target integrity score, S, is calculated based on the ratio of the number of detected valid keypoints to the preset total number of keypoints. integrity =N valid / N total , where N total The expected total number of key points, i.e., N total =17.
[0063] 2. Use a pose estimation model (such as YOLOv8-pose) to identify human pose, and calculate the pose integrity score, i.e., the pose integrity score S, based on the proportion of high-confidence keypoints. pose The proportion of key points with high confidence (e.g., >0.5).
[0064] High-level semantic feature vector f high =[S integrity ,S pose ].
[0065] Because the characteristic scores come from different sources, their numerical ranges and distributions vary greatly (e.g., S...). BRISQUE The range is approximately [0, 100], S SAM Since the range is approximately [0,1], direct fusion will result in features dominating the outcome due to a large numerical range. Therefore, each feature vector needs to be normalized.
[0066] S3: Normalize the low-level image quality feature vector, the mid-level structural feature vector, and the high-level semantic feature vector respectively, and then concatenate all the normalized feature vectors into a total feature vector (i.e., multi-dimensional features).
[0067] This embodiment uses the Min-Max normalization method to linearly map each feature score to the [0,1] interval. After normalization, the three feature vectors are concatenated into a total feature vector F: F = concat(f low-norm ,f mid-norm ,f high-norm In this embodiment, F is an 8-dimensional vector.
[0068] S4: Input the total feature vector into the pre-trained image usability prediction model to obtain the usability evaluation score of the image to be evaluated; wherein, the image usability prediction model is a regression network integrating a hierarchical attention fusion unit, which is used to adaptively weight and fused and predict the input total feature vector.
[0069] The image availability prediction model consists of a linear projection layer, a hierarchical attention fusion layer, and a regression output layer connected in sequence.
[0070] The linear projection layer is used to project the total feature vector into a feature space, that is, to project the total feature vector into a high-dimensional feature space (such as 64-dimensional), to obtain the initial feature sequence X. (0) .
[0071] The hierarchical attention fusion fusion unit is used to model the global dependencies between features of different dimensions in a high-dimensional feature space through a multi-head self-attention mechanism, and adaptively generate fusion weights and output a fusion feature vector based on the fusion weights.
[0072] The hierarchical attention fusion processor of this invention uses a Transformer encoder as its backbone, takes heterogeneous multi-layer features as input, and outputs a collaborative representation of these features modulated by attention weights. As a bridge connecting the semantic gaps of "pixel-scene-object," the hierarchical attention fusion processor does not simply perform feature concatenation or weighted averaging. Instead, it aligns information at different levels of abstraction in the feature space through a multi-head self-attention mechanism, mines the global dependencies between features in each dimension of the high-dimensional feature space and their synergistic effects on the usability of the final task, adaptively generates fusion weights, and outputs a fused feature vector based on these weights.
[0073] The hierarchical attention fusion unit comprises L encoder layers based on multi-head self-attention. In this embodiment, L=2, as shown below. Figure 3 As shown.
[0074] Each encoder layer based on multi-head self-attention (MHSA) comprises a MHSA sublayer and a feedforward neural network (FFN) sublayer. Both sublayers employ a sequential processing logic of "sublayer computation + residual connection + layer normalization (LayerNorm)": the encoder layer input is first computed by the MHSA sublayer; the output of the MHSA sublayer is residually connected to its original input, and then LayerNorm is executed. This normalized result serves as the input to the FFN sublayer; the output of the FFN sublayer is residually connected to its own original input, and then LayerNorm is executed again, finally yielding the output of the encoder layer. The model uses the feature sequence as a query (Q), key (K), and value (V), and dynamically determines how to balance information from different levels of features when generating the final representation by calculating attention weights. For example, for a surveillance image with a clear human body but a blurred background, the model automatically assigns appropriate attention weights to high-level semantic features. This explanation uses a human body as an example, but the invention is not limited to this. The preset category target of the present invention can be replaced with any target that needs to be visually recognized and understood, such as vehicles, pedestrians, or specific equipment, depending on the actual application scenario. Only the corresponding target detection and attitude estimation model needs to be replaced or added.
[0075] The regression output layer is used to map the fused feature vector output by the hierarchical attention fusion processor to the final usability evaluation score.
[0076] The regression output layer is a fully connected layer that maps the output of the last encoder layer to a scalar, namely the final image usability score S. usability The score can be scaled to the range of [0, 100], with a higher score indicating greater availability for higher-level tasks.
[0077] Thanks to the aforementioned adaptive fusion mechanism, the method of this invention exhibits excellent robustness in complex real-world scenarios. For example, for images with complete semantic subjects but motion blur, the model automatically increases the weights of high-level semantic features, providing a reasonable high availability score; for images with clear image quality but severely occluded key targets, it automatically decreases the weights of high-level semantic features, providing a low availability score. This aligns with the human thought process of making judgments based on task logic, an effect that traditional fixed-weight fusion methods cannot achieve.
[0078] The training process for the image availability prediction model is as follows:
[0079] A subset of images containing clear human instances was selected from the TID2013 dataset, ensuring that each image had a corresponding high-level visual task performance metric (such as pose estimation accuracy) or a usability score labeled with task performance as the true label. The mean squared error (MSE) loss function was used during training to minimize the prediction score S. usabilityThe gap between the model and the true label. Using the Adam optimizer, with a learning rate of 0.001 and a batch size of 32, the model was trained for 100 epochs, and the best-performing model was selected on the validation set.
[0080] In this embodiment, mainstream no-reference image quality assessment methods in both traditional and deep learning fields are selected for comparison. Traditional methods include BRISQUE and NIQE. Deep learning methods include DBCNN, PAQ2PIQ, HyperIQA, GraphIQA, and NLNet. All comparison methods use official code or pre-trained models and are tested on the same dataset (some methods are fine-tuned to ensure fairness).
[0081] Two core metrics commonly used in image quality assessment, namely the Pearson linear correlation coefficient (PLCC) and the Spearman rank correlation coefficient (SRCC), are employed to measure the consistency between the predicted score and the subjective MOS. The Pearson linear correlation coefficient (PLCC) assesses the degree of linear correlation between the predicted score and the subjective score, with a value ranging from -1 to 1; a value closer to 1 indicates a stronger linear correlation. Before calculation, a five-parameter logistic function is used to perform nonlinear fitting on the predicted score to eliminate system nonlinear bias. The Spearman rank correlation coefficient (SRCC) assesses the rank correlation between the predicted score and the subjective score, with a value ranging from -1 to 1; a value closer to 1 indicates stronger ranking consistency and is insensitive to absolute scores. Table 1 shows the SRCC and PLCC performance metrics of the method of this invention and all comparative methods on the test set.
[0082] Table 1. Performance comparison of different NR-IQA methods on subsets of human images.
[0083]
[0084] To verify the contributions of each module in the method framework of this invention, an ablation experiment was also designed, and the results are shown in Table 2. Table 2 shows that variant A (i.e., removing high-level semantic features) exhibits a significant performance decrease (SRCC decreases by 0.083, PLCC decreases by 0.091), indicating that high-level semantic features are the core of determining the usability of an image in high-level tasks, and a lack of assessment of the completeness of the semantic target will severely affect the accuracy of the assessment. Variant B (i.e., removing mid-level structural features) shows a slight performance decrease (SRCC decreases by 0.039, PLCC decreases by 0.057), indicating that mid-level structural features, as a bridge between low and high levels, can bridge the information gap between local image quality and discrete semantic targets, improving the comprehensiveness of the assessment. Variant C (i.e., removing low-level image quality features) shows a significant performance decrease (SRCC decreases by 0.245, PLCC decreases by 0.268), indicating that low-level image quality features are the foundation of usability assessment; even with a complete semantic target, severe low-level distortion can make the image unusable in high-level tasks. The complete model performed optimally, validating the necessity of hierarchical fusion of the three-layer features. Only by integrating features from all dimensions can accurate image usability assessment be achieved.
[0085] Table 2 Ablation Experiment Results (Impact of Each Model on Performance)
[0086]
[0087] like Figure 4 As shown, the referenceless image usability assessment system based on hierarchical feature fusion proposed in this invention includes:
[0088] The hierarchical feature extraction module is used to receive the input image and extract the low-level image quality feature vector, mid-level structure feature vector, and high-level semantic feature vector of the input image in parallel. The hierarchical feature extraction module integrates all the sub-modules required for low-level image quality, mid-level structure, and high-level semantic feature extraction (such as calling the interfaces of pyiqa library, SAM, YOLOv8, etc.).
[0089] The feature preprocessing module is used to normalize and concatenate the low-level image quality feature vector, the mid-level structural feature vector, and the high-level semantic feature vector to generate the total feature vector.
[0090] The usability prediction module has a built-in pre-trained image usability prediction model, which receives the total feature vector and outputs the usability evaluation score of the input image.
[0091] like Figure 5 As shown, the system also includes:
[0092] The results output and visualization module is used to output scores and generate visual reports.
[0093] This invention proposes an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a no-reference image usability assessment method based on hierarchical feature fusion.
[0094] This invention proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of a no-reference image usability assessment method based on hierarchical feature fusion.
[0095] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A no-reference image usability assessment method based on hierarchical feature fusion, characterized in that, Includes the following steps: S1: Obtain the image to be evaluated; S2: Perform hierarchical feature extraction on the image to be evaluated to obtain low-level image quality feature vector, mid-level structural feature vector, and high-level semantic feature vector respectively; the low-level image quality feature vector includes frequency domain detail sharpness features and scores corresponding to the degree of deviation from the prior distribution of natural images; the mid-level structural feature vector includes segmentation quality score, global structural distortion score, and global quality perception score; the high-level semantic feature vector includes target integrity score and pose integrity score; S3: Normalize the low-level image quality feature vector, the mid-level structure feature vector, and the high-level semantic feature vector respectively, and then concatenate all the normalized feature vectors into a total feature vector. S4: Input the total feature vector into a pre-trained image usability prediction model to obtain the usability evaluation score of the image to be evaluated; wherein, the image usability prediction model is a regression network integrating a hierarchical attention fusion unit, used to adaptively weight and predict the input total feature vector; the image usability prediction model includes a linear projection layer, a hierarchical attention fusion unit, and a regression output layer connected in sequence; the linear projection layer is used to project the total feature vector into the feature space; the hierarchical attention fusion unit is used to adaptively generate fusion weights and output a fused feature vector based on the fusion weights; The regression output layer is used to map the fused feature vector output by the hierarchical attention fusion processor to the final usability evaluation score.
2. The method for evaluating the usability of no-reference images based on hierarchical feature fusion according to claim 1, characterized in that, The hierarchical attention fusion processor includes at least one encoder layer based on multi-head self-attention.
3. The method for evaluating the usability of no-reference images based on hierarchical feature fusion according to claim 2, characterized in that, The multi-head self-attention-based encoder layer comprises a multi-head self-attention sub-layer and a feedforward neural network sub-layer. The input of each encoder layer serves as the input of the multi-head self-attention sub-layer. The output of the multi-head self-attention sub-layer is residually connected to its original input and then subjected to layer normalization. The result of layer normalization serves as the input of the feedforward neural network sub-layer. The output of the feedforward neural network sub-layer is residually connected to its original input and then subjected to layer normalization, finally yielding the output of the encoder layer.
4. A no-reference image usability assessment system based on hierarchical feature fusion, characterized in that, include: The hierarchical feature extraction module is used to receive the input image and extract the low-level image quality feature vector, the mid-level structural feature vector, and the high-level semantic feature vector of the input image in parallel. The low-level image quality feature vector includes frequency domain detail sharpness features and a score corresponding to the degree of deviation from the prior distribution of natural images; the mid-level structure feature vector includes segmentation quality score, global structure distortion score, and global quality perception score; the high-level semantic feature vector includes target integrity score and pose integrity score. The feature preprocessing module is used to normalize and concatenate the low-level image quality feature vector, the mid-level structure feature vector, and the high-level semantic feature vector to generate a total feature vector. The usability prediction module has a built-in pre-trained image usability prediction model as described in claim 1, which is used to receive the total feature vector and output the usability evaluation score of the input image.
5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of a no-reference image availability assessment method based on hierarchical feature fusion as described in any one of claims 1 to 3.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of a no-reference image availability assessment method based on hierarchical feature fusion as described in any one of claims 1 to 3.