No-reference vehicle image quality assessment method and device based on saliency perception
Through the combination of significant perception and self-supervised learning, a comprehensive feature map is generated to evaluate the image quality, which solves the problem that existing methods cannot distinguish image areas, achieves more accurate image quality evaluation, and improves the safety and reliability of the autonomous driving system.
Patent Information
- Application Number
- CN202411801487.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-09
- Publication Date
- 2025-08-19
- Estimated Expiration
- 2044-12-09
AI Technical Summary
The existing deep learning-based image quality evaluation method cannot distinguish different areas in the image in automotive application scenarios, resulting in inaccurate evaluation of the quality of important visual information, which may lead to safety hazards of autonomous driving systems.
A reference-free vehicle image quality evaluation method based on significance perception is adopted. Through self-supervised learning and significance region segmentation, combined with a multi-layer perceptron, a comprehensive feature map is generated to evaluate the overall quality of the image, and the attention mechanism of the human visual system is simulated.
It improves the accuracy of image quality evaluation and generalization capabilities of the model, can more accurately identify significant areas of quality degradation, and enhances the safety and reliability of the autonomous driving system.
Smart Images

Figure CN119722627B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a reference-free vehicle image quality assessment method and device based on saliency perception. Background Art
[0002] Deep learning falls within the realm of artificial intelligence and machine learning. It solves real-world problems by building neural network structures, performing feature extraction, and regression decision-making. In recent years, deep learning has made significant progress in image quality assessment, particularly in the automotive industry. Models based on convolutional neural networks (CNNs), trained on large-scale subjective databases, can automatically assess image quality. By learning features from large amounts of image data, these models can largely simulate human perception of image quality, which is particularly important for autonomous driving systems. For example, in automotive environments, images captured by onboard cameras are used to support various advanced driver assistance systems (ADAS), such as lane keeping assist, adaptive cruise control, and automatic emergency braking. High-quality images are crucial to ensuring the accuracy and reliability of these systems. However, existing deep learning-based image quality assessment methods, such as CN118608927A, suffer from the following issues in automotive applications: 1. Existing methods typically process the entire image indiscriminately, failing to distinguish between different image regions. This processing is inconsistent with the human visual system, which prioritizes salient areas in a scene. In the automotive field, this may mean that the system fails to correctly identify road signs, pedestrians or other important obstacles because these key elements are often located in specific parts of the image rather than evenly distributed; 2. Because the differences in the importance of various regions in the image are not taken into account, existing methods may not accurately reflect the overall quality of the image. For example, the blur of foreground areas (such as pedestrians or obstacles in front of the vehicle) usually has a greater impact on image quality than the blur of background areas (such as distant scenery). For autonomous driving systems, this defect may lead to potential safety hazards because it may underestimate the quality loss of important visual information. In order to meet the above challenges, new image quality evaluation methods need to understand and process different areas of the image more intelligently, imitate the human visual system's attention to salient areas, and evaluate image quality based on the actual importance of each area. Such improvements are of great significance to improving the safety and reliability of autonomous driving technology. Summary of the Invention
[0003] The present invention proposes a method, device and storage medium for non-reference vehicle image quality assessment based on saliency perception, which can solve at least one of the technical problems in the background technology; the method is suitable for image quality evaluation in various practical application scenarios.
[0004] To achieve the above object, the present invention adopts the following technical solutions:
[0005] A no-reference vehicle image quality assessment method based on saliency perception includes the following steps:
[0006] S1 uses a large-scale unlabeled vehicle image dataset and trains a self-supervised image distortion recognition model to distinguish different types and levels of image distortion.
[0007] S2 uses the salient region segmentation dataset to train a model specifically for detecting salient regions in images, and in subsequent stages focuses on those parts that are more important to the human visual system;
[0008] S3, for the input image, uses the distortion quality perception module pre-trained in S1 and the salient region segmentation module pre-trained in S2 to obtain the saliency feature map and distortion feature map in the input image, and then achieves weighted fusion between the two by constructing and applying an appropriate weight matrix to generate a new comprehensive feature map;
[0009] S4, feeds the comprehensive feature map generated by S3 into a multi-layer perceptron (MLP), which outputs a score reflecting the overall quality of the image after further processing.
[0010] Furthermore, in step S1, specifically:
[0011] S11 collects a large-scale unlabeled vehicle image dataset, including both synthetically distorted and real-world distorted images. The dataset covers various types of distortion, such as noise, blur, and compression artifacts.
[0012] S12, design an image distortion module based on a convolutional neural network (CNN). This module uses ResNet-50 as the backbone network and is followed by a multi-layer perceptron (MLP) head to extract high-level features of the image.
[0013] S13, uses a self-supervised learning method to train the image distortion module. Specifically, the model is trained through contrastive learning to enable the model to recognize and adapt to various categories and levels of image quality degradation. A self-supervised loss function is defined, using the InfoNCE loss function, which calculates the loss between the representations generated by the query encoder and the key encoder. The InfoNCE loss function can be expressed as:
[0014]
[0015] Furthermore, in step S2, specifically:
[0016] S21, collects a salient region segmentation dataset containing a large number of natural vehicle images and their corresponding salient region annotations;
[0017] S22 builds a hierarchical salient feature perception module based on the U-Net architecture, which mainly consists of encoder and decoder components. The encoder stage includes residual U blocks (RSU) of different depths, such as RSU-7, RSU-6, RSU-5, RSU-4, and the expanded version of RSU-4F, to adapt to input feature maps of different resolutions. The decoder stage adopts corresponding upsampling operations, while introducing dilated convolutions in some stages to prevent the loss of contextual information during downsampling;
[0018] S23 uses the input convolutional layer to convert the original image into an intermediate feature map to facilitate local feature extraction. A U-Net structure with a height L is used to extract multi-scale contextual information from the intermediate feature map, and a residual connection is used to merge the local features and multi-scale features to generate a comprehensive saliency feature representation.
[0019] S24, train the model on the prepared salient region segmentation dataset. The optimization goal is to enable the model to accurately segment the salient regions in the image. Use cross entropy loss as the optimization criterion.
[0020]
[0021] The Adam optimization algorithm is used to update the parameters. After training, the model parameters are frozen to facilitate integration into the subsequent image quality assessment process.
[0022] Furthermore, in step S3, the saliency feature map and the distortion feature map are extracted from the input image respectively, and a weighted fusion between the two is achieved by constructing and applying an appropriate weight matrix to generate a new comprehensive feature map, extracting the saliency feature map SS and the distortion feature map Q from the input image, and constructing a trainable weight matrix W to perform weighted processing on the saliency feature map.
[0023] Furthermore, in step S4, specifically:
[0024] In step S41, the fused feature map is fed into a multi-layer perceptron (MLP) head for further feature extraction and integration. The MLP head then outputs the final image quality score. The MLP head typically consists of multiple fully connected layers, each followed by a nonlinear activation function except the last. The final quality score q is calculated using the following formula:
[0025] q=W n+1 h n +b n+1
[0026] Among them, w n and b nare the weight matrix and bias vector of the nth layer, respectively, and n is the number of layers in the MLP.
[0027] In another aspect, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.
[0028] On the other hand, the present invention further discloses a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.
[0029] As can be seen from the above technical solution, the present invention's saliency-aware, no-reference vehicle image quality assessment method is specifically a saliency-aware no-reference image quality assessment method that incorporates distortion category information. By introducing a hierarchical saliency-aware module and a self-supervised learning quality-aware module, this method aims to more accurately assess image quality and align it more closely with human subjective judgment.
[0030] The present invention provides a saliency-aware, reference-free image quality assessment method that incorporates distortion category information. By introducing a hierarchical saliency-aware module and a self-supervised learning quality-aware module, this method aims to more accurately assess image quality and align it more closely with human subjective judgment. Specifically, the method can effectively distinguish between different regions in an image, emphasizing the importance of salient regions while minimizing the influence of background regions. This method not only improves the accuracy of image quality assessment but also enhances the model's generalization capabilities in practical applications, particularly in the absence of labeled data. By simulating the workings of the human visual system in this way, image quality assessment that is closer to human perception can be achieved.
[0031] Specifically, the present invention utilizes a hierarchical salient feature perception module based on the U-Net architecture, which can effectively distinguish different regions in an image, particularly emphasizing the importance of salient regions. This module ensures high-precision recognition of visually important parts of the image through multi-scale contextual information extraction and a residual connection mechanism, reducing the impact of background areas on the evaluation results. In addition, by designing a model specifically for detecting salient regions in images, the present invention simulates the attention mechanism of the human visual system. This design ensures that the evaluation process not only focuses on the overall distortion of the image, but also pays special attention to those parts that are critical to human visual quality, achieving an image quality assessment that is closer to human perception.
[0032] In order to further improve the generalization ability and adaptability, the present invention introduces a self-supervised learning method to train the image distortion recognition module. This enables the present invention to maintain efficient performance even in the absence of labeled data. Self-supervised learning uses the InfoNCE loss function through a comparative learning framework to enhance the adaptability and robustness of the model to various types and degrees of distortion, greatly improving the generalization ability of the model in different application scenarios. Combining distortion category information and saliency perception, the present invention can provide a more detailed and comprehensive image quality assessment. By extracting the saliency feature map and the distortion feature map of the input image respectively, and performing weighted fusion to generate a comprehensive feature map, the multi-layer perceptron (MLP) finally outputs a score reflecting the overall quality of the image, ensuring the accuracy and reliability of the evaluation results.
[0033] It is worth mentioning that the application of the self-supervised learning mechanism significantly reduces the need for large-scale annotated datasets, saving a lot of manpower and time costs, allowing the present invention to be effectively trained and evaluated on more diverse image data, further expanding its scope of application. In summary, the saliency-aware reference-free image quality assessment method that integrates distortion category information proposed in the present invention not only improves the accuracy of image quality assessment, but also enhances the practical application value and generalization ability of the model. In this way, the present invention provides a more intelligent, efficient and human-visual quality assessment solution for the field of image processing.
[0034] In the automotive industry, high-quality images are crucial to ensuring the accuracy and reliability of advanced driver assistance systems (ADAS) and autonomous driving systems. Images captured by on-board cameras are used not only for navigation and environmental perception, but also for critical tasks such as pedestrian detection and traffic sign recognition. Therefore, the point image quality assessment proposed in this paper is extremely important in this field. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 It is the overall framework diagram of the method of the present invention;
[0036] Figure 2 is a flow chart of a pre-trained image distortion recognition model according to an embodiment of the present invention;
[0037] Figure 3 An architectural diagram of a pre-trained saliency region perception module in an embodiment of the present invention;
[0038] Figure 4 The evaluation results of the vehicle reference image, background blurred image and subject blurred image of this embodiment are respectively shown. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.
[0040] This example will be explained based on the LIVE dataset. The LIVE dataset was created in 2006 by the Image and Video Engineering Laboratory at the University of Texas at Austin. The reference images for the dataset are 29 high-resolution, high-quality color images collected from the internet and photography CDs. The images include faces, animals, close-ups, wide-angle shots, natural scenes, man-made objects, and images with different foreground / background configurations.
[0041] The method of the present invention first utilizes a large-scale image dataset and adopts the MOCO (Momentum Contrast) architecture to train an image distortion perception module. Through self-supervised learning, the present invention can effectively address the problem of scarce labeled data. Next, in order to capture the content differences between different salient regions, the present invention further develops an image saliency perception module. The features extracted by the two modules are then weighted and integrated to obtain the final fused features. In this process, the present invention selects a multi-layer perceptron (MLP) as the last part of the model, which is responsible for outputting a quality score based on the above-mentioned fused features. With its powerful nonlinear fitting ability, the MLP can accurately map the complex relationship between input features and image quality scores, ensuring the accuracy and reliability of the evaluation results.
[0042] The following are specific instructions:
[0043] The embodiment of the present invention provides a no-reference vehicle image quality assessment method based on saliency perception, such as Figure 1 The specific steps are as follows:
[0044] S1, uses large-scale unlabeled datasets (such as KADID, AVA, COCO, etc.) to pre-train a quality-aware model through self-supervised learning. The model uses ResNet-50 as the backbone network and is followed by a multi-layer perceptron (MLP) head.
[0045] S2 uses the DUTS-TR dataset (including 21,106 images augmented by horizontal flipping) to train a hierarchical salient feature perception module based on the U-Net architecture. This model is specifically designed to detect salient regions in images.
[0046] S31, for the input image, first use the quality-aware model pre-trained in S1 to extract quality features. Assume that the model output is F q, and the present invention selects the output of the last mapping head and flattens it into a one-dimensional vector with a shape of [2048,1]. This step can be expressed as:
[0047] F q =Flatten(QualityAwareModule(I))
[0048] Where I is the input image, QualityAwareModule represents the quality-aware model, and Flatten represents the operation of flattening the multi-dimensional output into a one-dimensional vector.
[0049] S32 extracts salient features from the input image using the hierarchical salient feature perception module based on the U-Net architecture and trained on the DUTS-TR dataset in step S2.
[0050] S33 performs weighted fusion on the two feature vectors obtained in S32 and S31. First, the two feature vectors are concatenated to form a new feature vector, which is then weighted through a fully connected layer (MLP). The entire process can be expressed as follows:
[0051] F combined =MLP([F q ; F s ])
[0052] Among them, ([F q ; F s ]) means to set F q and F s The result of splicing along the channel dimension, MLP is a multi-layer perceptron, which is responsible for further processing and weighting the spliced feature vector.
[0053] S41, the feature vector F obtained by MLP combined It is sent to the last fully connected layer and outputs a score q that reflects the overall quality of the image. This process can be expressed as follows:
[0054] q=W n+1 h n +b n+1 (1-1)
[0055] Among them, W n+1 and b n+1 are the weight matrix and bias vector of the last layer of MLP, h n is the output of the previous layer.
[0056] Dataset Introduction
[0057] The LIVE dataset includes 779 distorted images. The dataset uses five computer distortion operations to degrade the reference images by 5 to 6 levels, resulting in 779 distorted images.
[0058] The TID2013 dataset contains more than 3,000 test images obtained from 25 reference images, each with 24 distortion types and each type of distortion is divided into 5 levels.
[0059] The CSIQ dataset contains 30 original images and 866 synthetically distorted images with six distortion types.
[0060] The KonIQ-10 dataset selects 10,073 images from the YFCC100M dataset. For each image, 120 reliable quality ratings are obtained through crowdsourcing and annotated by 1,459 volunteers, with a total of 1.2 million subjective data collected.
[0061] The LIVEC dataset contains 1,162 images captured using modern mobile devices under a wide variety of real-world distortions. These images come from different photographers using different camera equipment in the real world, so these images contain complex real-world distortions.
[0062] The KADID-10K dataset contains 10,125 distorted images derived from 81 reference images. The dataset contains 25 different types of image distortions, including Gaussian blur, JPEG compression, noise addition, chroma distortion, etc. Each distortion type is applied to different images to varying degrees.
[0063] Experimental analysis
[0064] The model is validated on multiple widely used image quality assessment (IQA) datasets, including synthetic datasets (such as LIVE IQA, TID2013, CSIQ) and real distortion datasets (such as KonIQ-10K, CLIVE, SPAQ). The performance of the model is evaluated using metrics such as the Pearson linear correlation coefficient (PLCC) and the Spearman rank correlation coefficient (SRCC). The calculation formulas for PLCC and SRCC are:
[0065]
[0066] Among them, x i and y i are the predicted rating and the true rating, and are their average values, the ranking differences between the d-test scores and the true scores, respectively.
[0067] In order to demonstrate the effectiveness of the proposed algorithm, the evaluation indicators of several other algorithms are compared as shown in Table 1, and the best results are indicated in bold.
[0068] Table 1. Comparison of SRCC and PLCC of different algorithms
[0069]
[0070] In summary, the present invention is closer to the human visual system by learning salient region features and distortion-like features separately. By learning salient region-specific and distortion-specific features in a separate manner, the hierarchical method of the present invention is better able to capture quality degradation localized in salient image regions. As demonstrated by the results shown in Table 1, among these methods, the best performing method is marked in bold, while the second and third best performing methods are underlined. It is worth noting that the method of the present invention achieved the third, first, and third positions respectively on the synthetic dataset. On the real dataset, it achieved the first place. This observation emphasizes the effectiveness of combining distortion-specific feature extraction with salient region feature extraction for image quality assessment across a variety of datasets.
[0071] In the automotive industry, high-quality images are crucial to ensuring the accuracy and reliability of advanced driver assistance systems (ADAS) and autonomous driving systems. Images captured by on-board cameras are used not only for navigation and environmental perception, but also for key tasks such as pedestrian detection and traffic sign recognition. Therefore, image quality assessment is extremely important in this field. To verify the effectiveness of the method of the present invention in real-world application scenarios, embodiments of the present invention were tested on multiple automotive-related datasets: 1. KITTI dataset: A dataset widely used in autonomous driving research, containing a large number of images and videos of real-world driving scenes, covering different weather conditions and lighting environments. 2. Cityscapes dataset: Focusing on images of urban street scenes, it provides detailed semantic segmentation annotations and is suitable for evaluating the model's ability to recognize salient areas in complex urban environments. The method of the present invention not only performs well on synthetic datasets, but also demonstrates excellent performance in real driving scenarios. In particular, on the KITTI and Cityscapes datasets, the method of the present invention can accurately capture the quality degradation of salient areas such as road signs, pedestrians, and other obstacles, which is particularly important in autonomous driving systems.
[0072] like Figure 4The figure shows the evaluation results for a vehicle reference image, a blurred background image, and a blurred subject image. By carefully capturing the differences in blur between different regions, the method of the present invention assigns a lower quality score to images with blurred vehicle subjects, thereby better meeting practical needs and the perceptual characteristics of the human visual system. Specifically, in the reference image, while other methods scored 70.37, the method of the present invention achieved a higher score of 71.88, close to the human subjective score of 71.43. This demonstrates that the present invention is more accurate in evaluating high-quality images and better reflects the true image quality. In the blurred background image, other methods scored a higher score of 66.37, but the method of the present invention scored 64.33, more consistent with the human subjective score of 64.4. This result avoids overestimation of blurred background images and ensures the accuracy of the evaluation results. In the blurred subject image, the method of the present invention scored 56.50, closer to the human subjective score of 54.8, while other methods gave an excessively high score of 60.72. This demonstrates the advantages of the present invention in evaluating severely distorted images, particularly for quality degradation in critical areas such as the vehicle subject, providing a more reliable assessment.
[0073] In summary, when presented with a reference image and an image with both a blurred vehicle and a blurred background, the method of the present invention can more finely capture the differences in blur between different regions, thereby assigning a lower quality score to the image with the blurred vehicle. This method not only better meets practical needs but also better simulates the perceptual characteristics of the human visual system.
[0074] In another aspect, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.
[0075] On the other hand, the present invention further discloses a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above method.
[0076] In another embodiment provided by the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any of the saliency perception-based no-reference vehicle image quality assessment methods in the above embodiments.
[0077] It is understandable that the system, device and storage medium provided in the embodiments of the present invention correspond to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts of the above methods.
[0078] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).
[0079] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.
[0080] Each embodiment in this specification is described in a related manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so the description is relatively simple. For related parts, refer to the description of the method embodiment.
[0081] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.
Claims
1. A no-reference vehicle image quality assessment method based on saliency perception, characterized by: The following steps are included: S1 uses a large-scale unlabeled vehicle image dataset and trains a self-supervised image distortion recognition model to distinguish different types and levels of image distortion. S2, uses the salient region segmentation dataset to train an image salient region detection model for detecting salient regions in an image; S3, input image, uses S1 pre-trained image distortion recognition model and S2 pre-trained image salient region detection model to obtain the saliency feature map and distortion feature map in the input image, and then constructs and applies a weight matrix to achieve weighted fusion between the two to generate a new comprehensive feature map; S4, sends the comprehensive feature map generated by S3 to a multi-layer perceptron, which outputs a score reflecting the overall quality of the image after further processing; In the step S1, specifically: S11, collects a large-scale unlabeled vehicle image dataset, including synthetically distorted images and real distorted images; S12, design an image distortion module based on a convolutional neural network. The image distortion module shown uses ResNet-50 as the backbone network and is followed by a multi-layer perceptron head to extract high-level features of the image. S13, use the self-supervised learning method to train the image distortion module; specifically, the model is trained by contrastive learning to enable the model to recognize and adapt to various categories and levels of image quality degradation, define the self-supervised loss function, use the InfoNCE loss function, calculate the loss between the representations generated by the query encoder and the key encoder, for the positive sample pair and negative samples , the InfoNCE loss function is expressed as: ; in, represents the cosine similarity, is the temperature hyperparameter; In the step S2, specifically: S21, collecting a salient region segmentation dataset containing natural vehicle images and their corresponding salient region annotations; S22, constructing a hierarchical salient feature perception module based on a U-Net architecture, the hierarchical salient feature perception module comprising encoder and decoder components; the encoder stage comprising residual U-blocks of varying depths to accommodate input feature maps of varying resolutions; the decoder stage employing corresponding upsampling operations, while introducing dilated convolutions in the setup stage to prevent loss of contextual information during downsampling; S23, uses the input convolutional layer to convert the original image into an intermediate feature map to facilitate local feature extraction; extracts multi-scale context information from the intermediate feature map through a U-Net structure with height L, and merges the local features and multi-scale features through residual connections to generate a comprehensive saliency feature representation; S24, train the model on the prepared salient region segmentation dataset. The optimization goal is to enable the model to accurately segment the salient regions in the image. Use cross entropy loss as the optimization criterion. is the true saliency label, is the predicted significance probability distribution, then the cross entropy loss is expressed as: The Adam optimization algorithm is used to update the parameters; after training, the model parameters are frozen to facilitate their integration into the subsequent image quality assessment process.
2. The saliency-aware no-reference vehicle image quality assessment method according to claim 1, characterized in that: In step S3, the significant feature map and the distortion feature map are extracted from the input image respectively, and a weighted fusion between the two is achieved by constructing and applying an appropriate weight matrix to generate a new comprehensive feature map. The significant feature map S and the distortion feature map are extracted from the input image. Q , and construct a trainable weight matrix W Perform weighted processing on the saliency feature map.
3. The saliency-aware no-reference vehicle image quality assessment method according to claim 2, characterized in that: The step S4 specifically includes: The fused feature map is fed into a multi-layer perceptron head to further refine and integrate features. The multi-layer perceptron head outputs the final image quality score. The multi-layer perceptron head consists of multiple fully connected layers, each of which, except the last, is followed by a nonlinear activation function. The specific formula for the final quality score q is as follows: in, and They are The layer's weight matrix and bias vector, is the number of layers in the multilayer perceptron.
4. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 3.
Citation Information
Patent Citations
Image recovery quality evaluation method based on multilevel differential learning
CN114841887A
Image classification model training method, image classification method, equipment and medium
CN115496955A