Method and system for evaluating blind quality of slightly distorted image in complex and limited environment of mine

Abstract

The invention discloses a blind quality evaluation method and system for a slightly distorted image in a mine complex limited environment, and the method comprises the steps: training a non-reference image model through a real distortion quality evaluation data set, and obtaining a real distortion quality evaluation model; extracting quality features with content robustness in the image through a real distortion quality evaluation model; training a real distortion quality evaluation model by using the target image data set, and performing quality evaluation on the remarkably distorted image; iterative training is carried out, and on the basis of the obtained quality evaluation of the remarkably distorted image, characterization of the quality characteristics of the slightly distorted image is trained; and the quality features of the image with robustness and slight distortion are fused, a quality prediction module is trained, and an accurate quality score is obtained. According to the method, real distortion assistance and a two-stage iteration training strategy are utilized, the problem that the anti-interference capability of a model to content diversity is limited due to the fact that the content diversity of a data set is deficient is solved, and the accuracy of an evaluation result and the detection accuracy of underground potential safety hazards and abnormal conditions are improved.

Description

Technical Field

[0001] The invention relates to a blind quality evaluation method and system for slightly distorted images in a complex and restricted environment of a mine, belonging to the technical field of image quality evaluation. Background Art

[0002] In the complex and highly restricted special working environment of mines, the application of image quality evaluation algorithms is of great significance, especially in the two core areas of monitoring and daily safety management that are related to the safety of life and property. Due to many unfavorable factors such as insufficient lighting in the underground environment, the limitations of the monitoring equipment itself, and the interference of complex and changeable environmental factors, the collected image data often contains various distortions and noise. These distortions and noises seriously weaken the clarity, detail expression and overall recognizability of the image, which in turn brings difficulties to subsequent image analysis, processing and decision-making. The assistance of high-quality images can not only significantly improve the detection accuracy of various safety hazards and abnormal situations, but also effectively reduce the false alarms and missed alarms caused by poor image quality, thereby further reducing the safety risks of mine operations.

[0003] Traditional no-reference image quality assessment methods usually rely on manually designed features, which cannot effectively describe the diversity of distortion types. With the advancement of deep learning technology, progress has been made in the quality assessment of simulated distortion by using neural networks to directly predict quality scores from images in an end-to-end learning manner. However, the acquisition cost of simulated distortion image quality assessment datasets is very high. Each reference image needs to be added with simulated distortions of multiple distortion types and levels to obtain the corresponding distorted image. What is more complicated is that the mean subjective score (MOS) of each distorted image requires multiple raters to score the image at the same time, and then the scores of all raters are averaged. This is unrealistic to pursue content diversity while ensuring distortion diversity, which greatly affects the prediction performance and generalization ability of existing algorithms.

[0004] In summary, the existing quality evaluation of simulated distortion has the following problems: the existing quality evaluation model of simulated distortion without reference has limited anti-interference ability for content diversity, low prediction accuracy for images with unseen content, and limited ability to evaluate the quality of slightly distorted images. Therefore, how to evaluate the quality of slightly distorted images without reference has become an urgent problem to be solved. Summary of the invention

[0005] The purpose of the invention is to provide a method and system for blind quality evaluation of slightly distorted images in a complex and restricted environment of a mine. The method and system can improve the image quality evaluation model's anti-interference ability to content diversity and its ability to evaluate the quality of slightly distorted images, improve the accuracy of detecting safety hazards and abnormal conditions underground, reduce false alarms and missed alarms caused by poor image quality, and improve the safety of mine operations.

[0006] In order to achieve the above object, the present invention provides a blind quality evaluation method for slightly distorted images in a complex and restricted environment of a mine, comprising the following steps:

[0007] S1. Using the real distortion quality evaluation data set, the no-reference image model is trained to obtain a real distortion quality evaluation model;

[0008] S2, extracting content-robust quality features from images through a true distortion quality assessment model;

[0009] S3, using the target image dataset to train a real distortion quality assessment model and perform quality assessment on images with significant distortion;

[0010] S4, performing iterative training, based on the quality assessment of the significantly distorted image obtained in S3, training the representation of the quality characteristics of the slightly distorted image;

[0011] S5. Fusion the robust quality features with the quality features of slightly distorted images, train the quality prediction module, and obtain accurate quality scores.

[0012] Furthermore, the specific process of S1 is as follows:

[0013] S1.1. Fine-tune the ImageNet pre-trained model using a real distortion quality evaluation dataset. Based on the backbone network Resnet50, the image features are mapped to image quality scores through a quality prediction module consisting of four fully connected layers.

[0014] S1.2. During the training process, the real distortion image quality evaluation data set is randomly divided into a training set and a test set at a ratio of 8:2, and the MSE loss function is used for constraint to obtain the real distortion quality evaluation model. The formula is:

[0015]

[0016] Where y is the true label, is the model prediction value, n is the number of training images, and L is the loss value.

[0017] Furthermore, the process of S2 is as follows: taking the simulated distorted image as input, extracting content-robust quality features through the real distortion quality evaluation model; the backbone network after removing the fully connected layer is denoted as M1, the input image is denoted as I1, and the content-robust quality feature f1 is expressed as:

[0018] f1=M1(I1).

[0019] Furthermore, the specific process of S3 is as follows:

[0020] S3.1, the target image dataset is divided into training set and test set according to 8:2, and Resnet50 pre-trained on ImageNet is used as the backbone network. After the divided training set is sent to the real distortion quality evaluation model, the model weight parameters are updated using the SGD optimizer through the MSE loss function;

[0021] S3.2. The learning rate starts at 1e-3. After every ten epochs without obtaining better performance, it is multiplied by 0.3. When it is lower than 5e-5, the model training is completed. After removing the fully connected layer, the backbone network M2 is obtained. The training set is tested to obtain relevant knowledge on image quality assessment with significant distortion.

[0022] Furthermore, the specific process of S4 is as follows:

[0023] S4.1. Input the slightly distorted image I2 with distortion level 1, 2 or 3 into the backbone network M2 to obtain the extracted feature f2, and then input the feature f2 into the quality prediction module P to predict its score s:

[0024]

[0025] S4.2. Input the predicted score s into the loss function to calculate the loss and perform back propagation. The formula is:

[0026]

[0027] In the formula, y i is the true label, s d is the model prediction value, n is the number of training images, d is the distortion level of the image, and L2 is the loss value.

[0028] Furthermore, the specific process of S5 is as follows:

[0029] S5.1. Freeze the parameters of the trained backbone networks M1 and M2, keeping their model parameters unchanged, expressed as:

[0030] θ frozen ={θ1,θ2};

[0031] Where θ1 and θ2 represent the parameters of backbone networks M1 and M2 respectively;

[0032] S5.2. Extract the content-robust quality feature f1 and the quality feature f2 representing the slightly distorted image from the input image; concatenate f1 and f2 to form a composite feature f:

[0033] f = cat[f1,f2];

[0034] Among them, cat means splicing;

[0035] S5.3. The composite feature f is fed into a quality prediction module P consisting of four fully connected layers, and the quality prediction module P is trained using the mean square error MSE loss function to fit the image quality score Q i and the true mass fraction Q true The difference between them can be expressed as:

[0036] Q i =P(f);

[0037]

[0038] A blind quality assessment system for slightly distorted images in a complex and restricted environment of a mine, comprising a camera, a content-robust quality feature extractor, an iterative trainer and an image quality prediction module; the iterative trainer comprises a traditional quality assessment module and a quality feature extraction module focusing on slight distortion;

[0039] The camera and its memory card are used to collect and store image data with various contents, and to store simulated distorted image data after adding noise and annotation;

[0040] The content-robust quality feature extractor is used to receive the collected and annotated image data with diverse contents, and send it to the backbone network to extract quality features, and then constrain it through the quality prediction module and the back propagation algorithm to extract content-robust quality features;

[0041] The iterative trainer is used to ensure that the system pays attention to significant distortion information while improving the system's attention to slight distortion. The traditional quality assessment module learns general knowledge of image quality assessment through the stored simulated distorted image data. The quality feature extraction module that pays attention to slight distortion uses slightly distorted images and, based on the general knowledge learned by the traditional quality assessment module, realizes the function of paying attention to slight distortion.

[0042] The image quality prediction module is used to receive the features extracted by the content-robust quality feature extractor and the iterative trainer, predict the image quality, and complete the construction of the slight distortion blind quality evaluation system.

[0043] The present invention uses a real distortion quality evaluation data set to train a reference-free image model to obtain a real distortion quality evaluation model; extracts quality features with robust content in the image through the real distortion quality evaluation model; uses a target image data set to train the real distortion quality evaluation model to perform quality evaluation on images with significant distortion; performs iterative training, and on the basis of the obtained quality evaluation of significant distortion images, trains its characterization of quality features of slightly distorted images; integrates robust quality features with quality features of slightly distorted images, trains a quality prediction module, and obtains accurate quality scores. The present invention uses a real distortion auxiliary strategy and a two-stage iterative training strategy to solve the problem that the model's anti-interference ability to content diversity is limited due to the lack of content diversity in the data set, realizes the model's attention to and recognition of slightly distorted images, improves the accuracy of the evaluation results and the accuracy of detecting safety hazards and abnormal conditions underground, reduces false positives and false negatives caused by poor image quality, and improves the safety of mine operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a work flow chart of the method of the present invention;

[0045] Figure 2 It is a structural schematic diagram of the system of the present invention;

[0046] Figure 3 is a comparison diagram of PLCC and SRCC results of the distorted image corresponding to the baseline on the reference image of the KADID-10k test set in an embodiment of the present invention;

[0047] Figure 4 It is a comparison chart of PLCC and SRCC results of the baseline corresponding to the image at five distortion levels in the KADID-10k test set in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described below in conjunction with the accompanying drawings.

[0049] like Figure 1 As shown, a blind quality evaluation method for slightly distorted images in a complex and restricted environment of a mine comprises the following steps:

[0050] S1. Using the real distortion quality evaluation data set, the no-reference image model is trained to obtain a real distortion quality evaluation model;

[0051] S2, extracting content-robust quality features from images through a true distortion quality assessment model;

[0052] S3, using the target image dataset to train a real distortion quality assessment model and perform quality assessment on images with significant distortion;

[0053] S4, performing iterative training, based on the quality assessment of the significantly distorted image obtained in S3, training the representation of the quality characteristics of the slightly distorted image;

[0054] S5. Fusion the robust quality features with the quality features of slightly distorted images, train the quality prediction module, and obtain accurate quality scores.

[0055] like Figure 2 As shown, the present invention also provides a blind quality assessment system for slightly distorted images in a complex and restricted environment of a mine, comprising a camera, a content-robust quality feature extractor, an iterative trainer, and an image quality prediction module; the iterative trainer comprises a traditional quality assessment module and a quality feature extraction module focusing on slight distortion;

[0056] The camera and its memory card are used to collect and store image data with various contents, and to store simulated distorted image data after adding noise and annotation;

[0057] The content-robust quality feature extractor is used to receive the collected and annotated image data with diverse contents, and send it to the backbone network to extract quality features, and then constrain it through the quality prediction module and the back propagation algorithm to extract content-robust quality features;

[0058] The iterative trainer is used to ensure that the system pays attention to significant distortion information while improving the system's attention to slight distortion. The traditional quality assessment module learns general knowledge of image quality assessment through the stored simulated distorted image data. The quality feature extraction module that pays attention to slight distortion uses slightly distorted images and, based on the general knowledge learned by the traditional quality assessment module, realizes the function of paying attention to slight distortion.

[0059] The image quality prediction module is used to receive the features extracted by the content-robust quality feature extractor and the iterative trainer, predict the image quality, and complete the construction of the slight distortion blind quality evaluation system.

[0060] Embodiment: (1) The present invention trains a no-reference image model by using a real distortion quality evaluation dataset, such as Figure 3 As shown, the Koniq-10k dataset consists of 10,073 high-quality scored images with rich content diversity;

[0061] Resnet50 pre-trained on ImageNet is used as the baseline and fine-tuned through Koniq-10k. During the training process, the MSE loss function is used to achieve this goal. During the training process, the real distortion image quality evaluation dataset is randomly divided into 8:2, 80% is used for training, and the MSE loss function is used for constraint to obtain the real distortion quality evaluation model. The formula is:

[0062] Among them, y is the true label, is the model prediction value, n is the number of training images;

[0063] The model after training with all fully connected layers removed is denoted as M1. The high-level features of the model trained with real distorted images have rich content semantic information and good content generalization ability, so this feature can be used as a content-robust quality feature. The image in the simulated distorted dataset Kadid-10k is used as input, denoted as I1, and the extracted content-robust quality feature f1 can be expressed as:

[0064] f1=M1(I1);

[0065] When this embodiment uses the simulated distortion quality evaluation model to predict the score of the test reference image (quality score is 5 points), it can be found that some prediction results are as low as 2.678 points, while the score of the reference image in the training set is close to 5 points. Although it is possible to clearly distinguish the quality of the distorted image corresponding to the reference image, the overall prediction score has a serious deviation, which shows that the model without reference image has a weak ability to cope with interference caused by content changes. In contrast, the prediction results of the real distortion quality evaluation model for reference images of different contents are generally higher than 3.5 points, showing that the quality features it extracts are highly robust and can better cope with the impact of content differences;

[0066] (2) The existing model has limited ability to evaluate the quality of slightly distorted images. Since the data distribution of slightly distorted images is inconsistent with that of severely distorted images, and the existing model tends to represent significant distortions and ignore slight distortions during training, this leads to poor quality evaluation performance of slightly distorted images. An iterative training strategy is adopted to enhance the model's attention to slight distortions. In the first training phase, images in the simulated distortion dataset Kadid-10k are used as input. Resnet50 is used to capture the spatial information and local features in the image. The MSE loss function is used for back propagation, image quality-related knowledge is learned, and image quality scores are fitted. In the second training phase, based on the image quality-related knowledge obtained in the first training phase, images with distortion levels of 1, 2, and 3 in KADID-10k are used again for training using the L2 loss function. This ensures the model's ability to predict the quality of significantly distorted images while improving the model's ability to represent slight distortions:

[0067]

[0068] where y i is the true label, s d is the model prediction value, n is the number of training images, d is the image distortion level, and the loss value is L2. From the above formula, it can be seen that only slightly distorted images can affect the loss value L2;

[0069] (3) Integrate the robust quality features of the content and the quality features that focus on slight distortion, train the quality prediction module, and obtain the final quality score. Specifically, in the new training process, first freeze the model parameters completed in the first two steps of training, and keep the model parameters of the first two steps unchanged. It can be expressed as:

[0070] θ frozen ={θ1,θ2}

[0071] Among them, θ1 and θ2 represent the parameters of the first two steps of the model. Then, the content-robust quality feature f1 and the quality feature f2 that focuses on slight distortion are extracted from the input image. In order to retain the extracted information to the greatest extent, f1 and f2 are concatenated to form a composite feature f: f = cat[f1,f2]; cat represents concatenation; it is sent to a quality prediction module P consisting of four fully connected layers. In this network, the mean square error (MSE) loss function is used for training to fit the image quality score Q i and the true mass fraction Q true The difference between . It can be expressed as:

[0072] Q i =P(f);

[0073] In this way, feature information from different sources can be effectively integrated to improve the accuracy and robustness of quality prediction. The present invention is trained on the KADID-10k dataset, and the training results are used as model results. Then, the real distortion-assisted model is used for comparison on the dataset in the same training method. The ablation experiment results are shown in Table 1 and Table 2 respectively:

[0074] Table 1 PLCC results of ablation experiment

[0075] Model Level 1 Level2 Level3 Resnet50 0.482 0.647 0.753 Resnet50+iterable 0.489 0.661 0.765 Resnet50+iterable+real 0.505 0.668 0.768

[0076] Table 2 Ablation experiment SRCC results

[0077] Model Level 1 Level2 Level3 Resnet50 0.442 0.647 0.764 Resnet50+iterable 0.462 0.667 0.767 Resnet50+iterable+real 0.470 0.666 0.783

[0078] According to the Pearson linear correlation coefficient (PLCC) and Spearman rank correlation coefficient (SRCC) results of the ablation experiments in Tables 1 and 2, Figure 4 It can be seen that the present invention has achieved remarkable and excellent results in the field of blind quality evaluation of slight distortion. The quality characteristics of real distortion assistance and robustness to content diversity have greatly improved the performance of the model in evaluating slight distortion. At the same time, the two-stage iterative training strategy has also greatly enhanced the model's attention to slight distortion, proving the effectiveness of the algorithm.

Claims

1. A blind quality evaluation method for slightly distorted images in a complex and restricted environment of a mine, characterized in that: The steps include: S1. Using the real distortion quality evaluation data set, the no-reference image model is trained to obtain a real distortion quality evaluation model; S2, extracting content-robust quality features from images through a true distortion quality assessment model; S3, using the target image dataset to train a real distortion quality assessment model to perform quality assessment on images with significant distortion; S4, performing iterative training, based on the quality assessment of the significantly distorted image obtained in S3, training the representation of the quality characteristics of the slightly distorted image; S5. Fusion the robust quality features with the quality features of slightly distorted images, train the quality prediction module, and obtain accurate quality scores.

2. The blind quality evaluation method for slightly distorted images in a complex and restricted environment of a mine according to claim 1 is characterized in that: The specific process of S1 is as follows: S1.

1. Fine-tune the ImageNet pre-trained model using a real distortion quality evaluation dataset. Based on the backbone network Resnet50, the image features are mapped to image quality scores through a quality prediction module consisting of four fully connected layers. S1.

2. During the training process, the real distortion image quality evaluation data set is randomly divided into a training set and a test set at a ratio of 8:2, and the MSE loss function is used for constraint to obtain the real distortion quality evaluation model. The formula is: Where y is the true label, is the model prediction value, n is the number of training images, and L is the loss value.

3. The blind quality evaluation method for slightly distorted images in a complex and restricted environment of a mine according to claim 1 or 2, characterized in that: The process of S2 is as follows: taking the simulated distorted image as input, extracting content-robust quality features through the real distortion quality evaluation model; the backbone network after removing the fully connected layer is denoted as M1, the input image is denoted as I1, and the content-robust quality feature f1 is expressed as: f1=M1(I1).

4. The blind quality assessment method for slightly distorted images in a complex and restricted environment of a mine according to claim 3 is characterized in that: The specific process of S3 is as follows: S3.1, the target image dataset is divided into training set and test set according to 8:2, and Resnet50 pre-trained on ImageNet is used as the backbone network. After the divided training set is sent to the real distortion quality evaluation model, the model weight parameters are updated using the SGD optimizer through the MSE loss function; S3.

2. The learning rate starts at 1e-3. After every ten epochs without obtaining better performance, it is multiplied by 0.

3. When it is lower than 5e-5, the model training is completed. After removing the fully connected layer, the backbone network M2 is obtained. The training set is tested to obtain relevant knowledge on image quality assessment with significant distortion.

5. The blind quality assessment method for slightly distorted images in a complex and restricted environment of a mine according to claim 4 is characterized in that: The specific process of S4 is as follows: S4.

1. Input the slightly distorted image I2 with distortion level 1, 2 or 3 into the backbone network M2 to obtain the extracted feature f2, and then input the feature f2 into the quality prediction module P to predict its score s: S4.

2. Input the predicted score s into the loss function to calculate the loss and perform back propagation. The formula is: In the formula, y i is the true label, s d is the model prediction value, n is the number of training images, d is the distortion level of the image, and L2 is the loss value.

6. The blind quality assessment method for slightly distorted images in a complex and restricted environment of a mine according to claim 5 is characterized in that: The specific process of S5 is as follows: S5.

1. Freeze the parameters of the trained backbone networks M1 and M2, keeping their model parameters unchanged, expressed as: i frozen ={θ1,θ2}; Where θ1 and θ2 represent the parameters of backbone networks M1 and M2 respectively; S5.

2. Extract the content-robust quality feature f1 and the quality feature f2 representing the slightly distorted image from the input image; concatenate f1 and f2 to form a composite feature f: f =cat[f1,f2]; Among them, cat means splicing; S5.

3. The composite feature f is fed into a quality prediction module P consisting of four fully connected layers, and the quality prediction module P is trained using the mean square error MSE loss function to fit the image quality score Q i and the true mass fraction Q true The difference between them can be expressed as: Q i =P(f); 7. A blind quality assessment system for slightly distorted images in a complex and restricted environment of a mine, comprising a camera, characterized in that: It also includes a content-robust quality feature extractor, an iterative trainer, and an image quality prediction module; the iterative trainer includes a traditional quality evaluation module and a quality feature extraction module that focuses on slight distortion; The camera and its memory card are used to collect and store image data with various contents, and to store simulated distorted image data after adding noise and annotation; The content-robust quality feature extractor is used to receive the collected and annotated image data with diverse contents, and send it to the backbone network to extract quality features, and then constrain it through the quality prediction module and the back propagation algorithm to extract content-robust quality features; The iterative trainer is used to ensure that the system pays attention to significant distortion information while improving the system's attention to slight distortion. The traditional quality assessment module learns general knowledge of image quality assessment through the stored simulated distorted image data. The quality feature extraction module that pays attention to slight distortion uses slightly distorted images and, based on the general knowledge learned by the traditional quality assessment module, realizes the function of paying attention to slight distortion. The image quality prediction module is used to receive the features extracted by the content-robust quality feature extractor and the iterative trainer, predict the image quality, and complete the construction of the slight distortion blind quality evaluation system.

Images