Imaging quality evaluation method based on depth map convolutional neural network
By constructing a graph convolutional neural network model and combining weak supervision pre-training and full supervision fine-tuning, the accuracy and robustness of the existing imaging quality evaluation methods in complex environments are solved, and efficient and accurate imaging quality evaluation is achieved, which is suitable for irregular data structures such as infrared imaging and hyperspectral imaging.
Patent Information
- Application Number
- CN202510351484.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-24
- Publication Date
- 2025-07-11
AI Technical Summary
The existing imaging quality evaluation methods are difficult to accurately reflect the actual quality of imaging in complex imaging environments and diverse application scenarios, with low computational efficiency and weak generalization ability, especially in irregular data structures such as infrared imaging and hyperspectral imaging.
A graph convolution neural network model consisting of graph structure feature extraction and fusion modules and feature embedding and pooling modules is constructed. Through weakly supervised pre-training and fully supervised fine-tuning training, the distortion map and differential average opinion scores are used as labels for training to optimize the performance of graph convolutional neural networks.
It improves the accuracy and robustness of imaging quality evaluation, making the evaluation results close to the human eye visual quality evaluation and are suitable for a variety of imaging types.
Smart Images
Figure FT_1
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of imaging quality evaluation. Based on the imaging characteristics of objects and using graph convolutional neural networks, an imaging quality evaluation method based on deep graph convolutional neural networks is constructed. Background Art
[0002] With the continuous development of imaging technologies, such as infrared thermal imaging, electron microscopy imaging, hyperspectral imaging, etc., imaging quality evaluation has become an important research direction in the field of image processing and analysis. The quality of the imaging directly affects the application effects of images in fields such as scientific research, industrial inspection, and medical diagnosis.
[0003] Traditional imaging quality evaluation methods mainly rely on subjective evaluation and some simple objective indicators, such as peak signal-to-noise ratio (PSNR), structural similarity (SSIM), etc. However, when faced with complex imaging environments and diverse application scenarios, these methods often have limitations and are difficult to accurately reflect the actual quality of the imaging. In recent years, deep learning technologies have made significant progress in the field of image quality evaluation. In particular, graph convolutional neural networks (GCNs), as an emerging deep learning architecture, have been proven to have unique advantages in processing the context information and node relationships of imaging. For example, GCNs can model the relationships between pixels or features in imaging through a graph structure, thereby more accurately evaluating the imaging quality. Nevertheless, existing imaging quality evaluation methods still face some challenges, such as low computational efficiency, weak generalization ability, and insufficient processing ability for large-scale data. Therefore, the present invention proposes an imaging quality evaluation method based on deep graph convolutional neural networks, aiming to further improve the accuracy and robustness of imaging quality evaluation by constructing a mapping relationship from nodes to graphs and optimizing the training strategy of graph convolutional neural networks, providing a more reliable basis for image processing and analysis. Summary of the Invention
[0004] The present invention designs an imaging quality evaluation method based on deep graph convolutional neural networks. The network model of this method consists of a graph structure feature extraction and fusion module and a feature embedding and pooling module. By combining a weakly supervised pre-training and a fully supervised fine-tuning training strategy, this method can accurately and efficiently evaluate the imaging quality. The present invention can obtain a visual quality evaluation effect close to that of the human eye for different types of imaging.
[0005] The present invention is achieved through the following technical solutions, including the following steps:
[0006] First step: Model construction;
[0007] Second step: Weakly supervised pre-training;
[0008] Third step: Fully supervised fine-tuning.
[0009] The creativity of the present invention is mainly reflected in:
[0010] (1) Considering the characteristics that images obtained from some special imaging (such as infrared imaging, hyperspectral imaging, etc.) have an irregular data structure, the present invention constructs a graph convolutional neural network model composed of a graph structure feature extraction and fusion module and a feature embedding and pooling module, and represents the key information of the irregular data in imaging through the graph convolutional neural network.
[0011] (2) During the training process, first, the present invention uses a gradient magnitude similarity operator sensitive to imaging to calculate a distortion map for the pixels in the imaging according to the node size in the graph structure, and uses this distortion map as a label for weak supervision pre-training of the graph convolutional neural network; secondly, transfer the pre-trained convolutional neural network model, and use the differential mean opinion score as a label for full supervision fine-tuning to further improve the accuracy and robustness of the model for imaging quality assessment. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 is a flowchart of the imaging quality assessment model designed based on the deep graph convolutional neural network of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0013] The following details the embodiments of the present invention. These embodiments are implemented on the premise of the technical solution of the present invention, and provide detailed implementation manners and specific operation processes. However, the protection scope of the present invention is not limited to the following embodiments.
[0014] Embodiment:
[0015] The first step: model construction;
[0016] The imaging quality assessment network model constructed by the present invention is as Figure 1 shown. This model is composed of a graph structure feature extraction and fusion module and a feature embedding and pooling module.
[0017] For the graph structure feature extraction and fusion module, first, the present invention uses the backbone part of a graph convolutional neural network for vision VSG as the graph structure feature extraction module. This module first divides the input image into nodes according to the size of n×n, then uses the KNN algorithm to find K neighbor nodes for each node to convert the imaging data into graph data, and finally realizes graph structure feature extraction through the Grapher and FNN inside the VSG network to generate the graph feature G of the distorted image D and the graph feature G of the original image O , and performs a concatenation operation on the two to obtain the fused graph feature G F . In the present invention, n = 16.
[0018] For the feature embedding and pooling module, first, the above-mentioned graph feature G F is subjected to feature embedding through a "multi-layer perceptron (MLP)" formed by cascading 3 "fully connected layers - Sigmoid activation layers" and 1 fully connected layer, that is, G F is dimensionally reduced in the channel dimension to generate G E , and then global average pooling is performed on G E to obtain the final image quality score.
[0019] Step 2: Weakly supervised pre-training;
[0020] To make the image quality assessment network have higher accuracy, the present invention adopts a gradient magnitude similarity operator sensitive to imaging, calculates the distortion map for the pixels in the image according to the node size in the graph structure, and uses this distortion map as a label to perform weakly supervised pre-training on the output G E of the above-mentioned feature embedding. This pre-training process is of great significance because the distortion map provides a standard regression score for each node in the graph structure, which will prompt the graph convolutional neural network to optimize the network in the correct direction.
[0021] Step 3: Fully supervised fine-tuning;
[0022] To obtain the final high-precision quality assessment score, the present invention first migrates the pre-trained convolutional neural network model in the second step, and then uses the differential mean opinion score as a label to perform fully supervised fine-tuning on the entire network to further improve the accuracy and robustness of the model for image quality assessment.
Claims
1. Model construction; The imaging quality assessment network model constructed in the present invention consists of a graph structure feature extraction and fusion module and a feature embedding and pooling module: For the graph structure feature extraction and fusion module, first, this paper uses the backbone part of a visual graph convolutional neural network VSG as the graph structure feature extraction module. This module first divides the input image into nodes according to the size of n×n, then uses the KNN algorithm to find K neighbor nodes for each node to convert the imaging data into graph data, and finally realizes graph structure feature extraction through the Grapher and FNN inherent in the VSG network, generating the graph feature G of the distorted image D and the graph feature G of the original image O , and performs a concatenation operation on the two to obtain the fused graph feature G F . In the present invention, n = 16 For the feature embedding and pooling module, first, the above graph feature G F is subjected to feature embedding through a "Multi-Layer Perceptron (MLP)" composed of 3 "Fully Connected Layers - Sigmoid Activation Layers" and 1 fully connected layer in series, that is, G F is dimensionally reduced in the channel dimension to generate G E , and then G E is subjected to global average pooling to obtain the final imaging quality score.
2. Weakly-supervised pre-training; To make the imaging quality assessment network more accurate, the present invention uses a gradient magnitude similarity operator sensitive to imaging, calculates a distortion map for the pixels in the imaging according to the node size in the graph structure, and uses this distortion map as a label for the output G of the above feature embedding. E Perform weakly-supervised pre-training. This pre-training process is of great significance because the distortion map provides a standard regression score for each node in the graph structure, which will prompt the graph convolutional neural network to optimize the network in the correct direction.
3. Full-supervised fine-tuning; In order to obtain the final high-precision quality assessment score, the present invention first migrates the pre-trained convolutional neural network model in the second step, and then uses the differential mean opinion score as a label to perform full-supervised fine-tuning on the entire network, further improving the accuracy and robustness of the model for imaging quality assessment.