A referenceless quality assessment method for low-light image enhancement based on a pre-trained fine-tuning strategy

Abstract

This invention discloses a referenceless quality assessment method for low-light image enhancement based on a pre-training and fine-tuning strategy. The method includes: constructing a large-scale pre-training dataset and a fine-tuning dataset of a certain size; the pre-training dataset includes low-light images, enhanced images generated from low-light images, and corresponding reference normal-light images; the fine-tuning dataset includes low-light images, enhanced images generated from low-light images, and subjective quality scores of the enhanced images; a two-stage training strategy of pre-training and fine-tuning is used to train the image quality assessment network, which is a dual-branch network structure with decoupled low-frequency and high-frequency branches; after training, the low-light images and corresponding enhanced images are used as network inputs, and the quality score of the enhanced images is output to achieve referenceless quality assessment of low-light enhanced images.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, and in particular to a method for non-reference quality assessment of low-light image enhancement based on a pre-training fine-tuning strategy. Background Technology

[0002] In various applications such as security monitoring, autonomous driving systems, medical imaging equipment, and mobile terminal photography, images often exhibit degradation due to limited lighting conditions, including insufficient brightness, low contrast, increased noise levels, color shifts, and loss of texture details. Low-light image enhancement technology, belonging to the field of image processing, aims to improve the quality of images acquired in low-light environments, thereby enhancing their visual readability and semantic interpretability.

[0003] In recent years, with the rapid development of deep learning technology, a large number of low-light image enhancement algorithms have been proposed, achieving significant improvements in mitigating the aforementioned degradation problems. However, lighting conditions in real-world scenes are highly complex and uncertain, and existing enhancement methods often struggle to maintain stable and consistent generalization performance across different types of low-light environments. Some methods perform exceptionally well in specific scenarios, but may exhibit under-enhancement or over-enhancement under other lighting conditions. Therefore, there is an urgent need for reliable image quality assessment methods to accurately evaluate the performance of enhanced images.

[0004] Existing low-light image enhancement quality assessment techniques mainly fall into two categories: full-reference assessment methods and no-reference assessment methods. Full-reference methods typically rely on high-quality reference images paired with the enhancement result, evaluating enhancement quality by comparing the differences between the enhanced image and the reference image. However, in real-world low-light applications, due to limitations in imaging conditions, environmental variations, and equipment, it is often difficult to obtain corresponding high-quality reference images, rendering full-reference assessment methods ineffective. Currently, no-reference image quality assessment methods primarily target traditional image distortion types (such as noise, compression distortion, and blur), while no-reference quality assessment methods specifically for low-light image enhancement remain relatively scarce. The distortion types of images enhanced in low light are more complex, potentially including noise amplification, color shift, abnormal brightness and contrast, structural damage, and artifacts, making it difficult for traditional no-reference quality assessment methods to accurately evaluate their visual quality. Summary of the Invention

[0005] The main objective of this invention is to provide a referenceless quality assessment method for low-light image enhancement based on a pre-trained fine-tuning strategy.

[0006] Another objective of this invention is to provide a referenceless quality assessment device for low-light image enhancement based on a pre-trained fine-tuning strategy.

[0007] The third objective of this invention is to provide an electronic device.

[0008] A fourth objective of this invention is to provide a non-transitory computer-readable storage medium.

[0009] To achieve the above objectives, a first aspect of the present invention proposes a referenceless quality assessment method for low-light image enhancement based on a pre-trained fine-tuning strategy, comprising:

[0010] Construct a large-scale pre-training dataset and a fine-tuning dataset of a certain size; the pre-training dataset includes low-light images, enhanced images generated from low-light images, and corresponding reference normal-light images; the fine-tuning dataset includes low-light images, enhanced images generated from low-light images, and subjective quality scores of the enhanced images.

[0011] A two-stage training strategy of pre-training and fine-tuning is adopted to train the image quality assessment network. In the pre-training stage, the full reference image quality assessment index is calculated using the enhanced image and the corresponding reference image to obtain an objective quality score. The objective quality score is then discretized and quantized into multiple quality levels, transforming the image quality assessment regression task into a classification task. The network is pre-trained to learn general feature representations related to the quality of low-light enhanced images. In the fine-tuning stage, the pre-trained network is fine-tuned using the low-light image, the corresponding enhanced image, and the subjective quality score of the enhanced image. At the same time, absolute quality constraints and relative quality constraints are introduced to optimize the model's prediction results.

[0012] After training, the low-light image and its corresponding enhanced image are used as network inputs, and the quality score of the enhanced image is output to achieve a referenceless quality evaluation of the low-light enhanced image.

[0013] Optionally, the pre-training phase includes:

[0014] The image quality assessment network is randomly initialized;

[0015] Randomly select a set of training samples from the pre-training dataset. , , ];

[0016] The objective quality score is calculated using the full-reference image quality assessment metric SSIM:

[0017]

[0018] continuous mass fraction Quantized into K discrete levels:

[0019]

[0020] in, For the threshold set;

[0021] by and As input, using a classification network Predicted quality grade distribution:

[0022]

[0023] The image quality assessment network is optimized using the cross-entropy loss function, which is expressed as:

[0024]

[0025] in, Indicates the number of pre-trained images. Indicates the first One sample was predicted to be the correct quality level. The probability of;

[0026] Determine if the network has converged. If not, update the network parameters using the gradient descent algorithm and randomly select training samples from the pre-training dataset again. If converged, end the training.

[0027] Optionally, the fine-tuning stage includes:

[0028] Construct images containing low light and its corresponding enhanced image The dataset, and on Subjective evaluation is conducted to obtain a subjective quality score;

[0029] The network is initialized using pre-trained weights, and the network structure in the fine-tuning stage is the same as that in the pre-training stage.

[0030] Randomly selected from the fine-tuning dataset and As input;

[0031] Obtain the logits of the network classifier;

[0032] Normalizing logits yields the probability distribution:

[0033]

[0034] Calculate the expectation and obtain mass fraction:

[0035]

[0036] Calculate the loss function, including absolute quality constraints. Two parts: relative mass constraints The final fine-tuning loss function is a weighted sum of two parts:

[0037]

[0038] in, These are the weighting coefficients;

[0039] Determine if the network has converged. If not, update the network parameters using the gradient descent algorithm and randomly select new parameters from the fine-tuning dataset. and If convergence is achieved, the training process ends.

[0040] Optionally, the absolute quality constraint supervises the model by minimizing the absolute error between the predicted quality score and the subjective human score, as expressed as:

[0041]

[0042] in, It is a subjective rating by humans. To fine-tune the number of images;

[0043] The relative quality constraint is the marginal ranking loss, expressed as:

[0044]

[0045]

[0046] in, and These represent the prediction scores of the two enhancement results for the same low-light image. For marginal control parameters, For sorting labels.

[0047] The image quality assessment network is a dual-branch network structure with low-frequency and high-frequency branches decoupled, and its processing procedure is as follows:

[0048] Input the original low-light image and its enhanced image The discrete wavelet transform decomposes the components into low-frequency and high-frequency components.

[0049] A dual encoder structure is used to extract low-frequency and high-frequency features respectively. Each encoder consists of three layers of standard visual Transformer encoder modules, and average pooling operation is introduced between adjacent encoder modules.

[0050] Global average pooling and global max pooling are applied to low-frequency and high-frequency features respectively to obtain global representations;

[0051] Low-frequency and high-frequency features are cascaded along the channel dimension, and the features are mapped to image quality categories through two multilayer perceptrons.

[0052] Optional, also includes:

[0053] Four independent test sets were constructed to evaluate the generalization performance of the model. Test set A contained image pairs with the same augmentation method and the same data distribution, test set B contained image pairs with the new augmentation method, test set C contained image pairs with the new data distribution, and test set D contained image pairs with the new augmentation method and the new data distribution.

[0054] The model's performance under the same augmentation method is verified using test set A. The model's adaptability to the new augmentation method is verified using test set B. The model's generalization ability under the new data distribution is verified using test set C. The model's joint robustness under the new augmentation method and the new data distribution is verified using test set D.

[0055] To achieve the above objectives, a second aspect of the present invention provides a referenceless quality assessment device for low-light image enhancement based on a pre-trained fine-tuning strategy, comprising:

[0056] The first module is used to construct a large-scale pre-training dataset and a fine-tuning dataset of a certain size; the pre-training dataset includes low-light images, enhanced images generated from low-light images, and corresponding reference normal-light images; the fine-tuning dataset includes low-light images, enhanced images generated from low-light images, and subjective quality scores of the enhanced images.

[0057] The second module is used to train the image quality assessment network using a two-stage training strategy of pre-training and fine-tuning. In the pre-training stage, the full reference image quality assessment index is calculated using the enhanced image and the corresponding reference image to obtain an objective quality score. The objective quality score is then discretized and quantized into multiple quality levels, transforming the image quality assessment regression task into a classification task. The network is then pre-trained to learn general feature representations related to the quality of low-light enhanced images. In the fine-tuning stage, the pre-trained network is fine-tuned using the low-light image, the corresponding enhanced image, and the subjective quality score of the enhanced image. At the same time, absolute quality constraints and relative quality constraints are introduced to optimize the model's prediction results.

[0058] The third module is used to take the low-light image and the corresponding enhanced image as network input after training, and output the quality score of the enhanced image to achieve a referenceless quality evaluation of the low-light enhanced image.

[0059] To achieve the above objectives, a third aspect of this application provides an electronic device, including a processor and a memory; wherein the processor runs a program corresponding to the executable program code by reading executable program code stored in the memory, for implementing the method described in the first aspect embodiment.

[0060] To achieve the above objectives, a fourth aspect of this application provides a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method described in the first aspect.

[0061] The embodiments of the present invention have the following beneficial effects: (1) It can more accurately perceive the complex distortions unique to low-light enhanced images, and the prediction results are more consistent with human visual perception. The present invention specifically models various types of complex distortions that are common in low-light enhanced images, such as noise amplification, color shift, abnormal brightness and contrast, structural damage and artifacts. Unlike traditional no-reference image quality evaluation methods that are only applicable to conventional distortions such as blur, noise or compression, the present invention can comprehensively evaluate the complex distortion features unique to low-light enhancement, so that the evaluation results can more accurately reflect the real subjective visual perception quality.

[0062] (2) A two-stage annotation and learning framework, from structural similarity prior to subjective MOS, is constructed to significantly improve the trainability and perceptual consistency of the model. This invention utilizes automatically generated SSIM quantized labels to achieve large-scale pre-training, thereby learning deep quality-sensitive representations; subsequently, fine-tuning is performed through subjective MOS, enabling the model to achieve prediction results highly consistent with human subjective evaluation without requiring a large amount of manual annotation, effectively solving the problems of data scarcity and expensive annotation in low-light scenarios.

[0063] (3) Significantly enhances the model's generalization ability and cross-domain adaptability, making it suitable for different algorithms, different scenarios, and different data distributions. This invention constructs a training system covering multiple enhancement methods, multiple data sources, and multiple lighting conditions, and combines a phased optimization strategy to ensure that the model maintains stable and accurate quality prediction performance on both "unseen enhancement methods" and "unseen scene distributions," thus possessing higher robustness and engineering value in practical low-light enhancement applications. Attached Figure Description

[0064] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0065] Figure 1 A flowchart of a low-light image enhancement no-reference quality assessment method based on a pre-trained fine-tuning strategy is provided for an embodiment of the present invention;

[0066] Figure 2 A flowchart of a pre-training scheme provided in an embodiment of the present invention;

[0067] Figure 3 A flowchart of the fine-tuning scheme provided in the embodiments of the present invention;

[0068] Figure 4 A schematic diagram comparing the quality prediction results of the present invention with other existing solutions, provided for an embodiment of the present invention;

[0069] Figure 5 This is a structural diagram of a low-light image enhancement no-reference quality assessment device based on a pre-trained fine-tuning strategy, provided in an embodiment of the present invention. Detailed Implementation

[0070] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0071] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0072] The existing solution has the following drawbacks:

[0073] 1. Existing no-reference image quality assessment methods are typically trained on datasets of distorted images under natural lighting conditions. Therefore, their feature extraction and quality discrimination mechanisms are generally applicable to common distortion types. However, when these methods are directly applied to the quality assessment of low-light enhanced images, they often fail to achieve ideal results. The main reason is that the distortions exhibited in low-light images before and after enhancement, such as noise amplification, color distortion, contrast distortion, and over-enhancement, differ significantly from the common distortions learned by traditional no-reference image quality assessment models, such as blurring, noise, and compression. Therefore, directly applying existing no-reference image quality assessment methods often leads to evaluation results that are inconsistent with human visual perception, limiting their practical application value in low-light enhancement tasks.

[0074] 2. Low-light scenes inherently possess more complex visual perception characteristics, and different low-light enhancement algorithms produce varying degrees of distortion when processing the same image. Under such complex and variable conditions, the generalization ability of evaluation models to achieve stable and accurate quality judgments faces significant challenges. However, existing no-reference image quality assessment methods generally rely on limited datasets constructed from subjective human ratings for training, which significantly limits their generalization ability and makes it difficult to accurately perceive subtle and variable subjective quality differences in enhanced images. Therefore, a new evaluation paradigm needs to be designed to improve the accuracy and generalization of evaluation models.

[0075] In summary, constructing a referenceless quality evaluation method for low-light image enhancement that is more in line with human subjective visual perception and has stability and generalization ability is of great technical significance and application value for automatically selecting the most suitable enhancement strategy, improving the overall performance of image processing systems, and providing effective feedback for the design, training and optimization of corresponding image enhancement algorithms.

[0076] This invention proposes a referenceless quality assessment method and apparatus for low-light image enhancement based on a pre-trained fine-tuning strategy, which can solve the following technical problems:

[0077] (1) The mismatch between the distortion distribution and the perception mechanism makes it impossible for existing no-reference image quality assessment methods to accurately evaluate low-light enhanced images. Because the types of distortion generated during low-light enhancement, such as noise amplification, color shift, contrast distortion, and over-enhancement, are significantly different from the conventional distortions on which traditional model training is based, existing methods cannot accurately reflect human visual subjective perception in low-light enhanced image quality assessment.

[0078] (2) The data scale and diversity are insufficient, and the generalization ability of the existing evaluation paradigm is limited, making it unable to adapt to complex and ever-changing low-light enhancement scenarios. Current no-reference image quality assessment methods are limited by small-scale human subjective rating datasets, which cannot cover complex and ever-changing low-light scenarios and various distortion modes introduced by different low-light enhancement algorithms, resulting in insufficient model generalization performance and difficulty in achieving stable and reliable quality judgment in real scenarios.

[0079] The following description, with reference to the accompanying drawings, describes a method and apparatus for low-light image enhancement without reference based on a pre-trained fine-tuning strategy, according to an embodiment of the present invention.

[0080] This embodiment provides a referenceless quality assessment method for low-light image enhancement based on a pre-trained fine-tuning strategy. For example... Figure 1 As shown, the method includes the following steps:

[0081] S1, construct a large-scale pre-training dataset and a fine-tuning dataset of a certain size; the pre-training dataset includes low-light images, enhanced images generated from low-light images, and corresponding reference normal-light images; the fine-tuning dataset includes low-light images, enhanced images generated from low-light images, and subjective quality scores of the enhanced images.

[0082] S2, a two-stage training strategy of pre-training and fine-tuning is adopted to train the image quality assessment network. In the pre-training stage, the full reference image quality assessment index is calculated using the enhanced image and the corresponding reference image to obtain an objective quality score. The objective quality score is then discretized and quantized into multiple quality levels, transforming the image quality assessment regression task into a classification task. The network is pre-trained to learn general feature representations related to the quality of low-light enhanced images. In the fine-tuning stage, the pre-trained network is fine-tuned using the low-light image, the corresponding enhanced image, and the subjective quality score of the enhanced image. At the same time, absolute quality constraints and relative quality constraints are introduced to optimize the model prediction results.

[0083] S3. After training, the low-light image and the corresponding enhanced image are used as network inputs, and the quality score of the enhanced image is output to achieve a referenceless quality evaluation of the low-light enhanced image.

[0084] In this embodiment of the invention, a large-scale low-light image enhancement quality evaluation dataset was constructed, containing approximately 40,000 enhanced images and 18 low-light image enhancement methods, covering typical distortion types such as illumination, contrast, color, noise, blur, saturation, and enhancement artifacts. As shown in Table 1, the dataset is divided into non-overlapping subsets for pre-training, fine-tuning, and evaluation.

[0085] During the pre-training phase, 2611 low-light images were enhanced using 14 low-light enhancement methods, generating a total of 36554 enhanced images for training. An additional 291 low-light images and their corresponding 4074 enhanced images were used for performance validation of the pre-trained model. It should be noted that all low-light images used in the pre-training phase had corresponding normal-light images, and these low-light-normal-light paired datasets were all sourced from existing publicly available low-light image enhancement datasets.

[0086] In the fine-tuning phase, 402 low-light images were selected, and 3216 enhanced images were generated using eight representative enhancement methods. Specifically, in the evaluation phase, this embodiment of the invention sets up four independent test sets: Test set A contains 41 low-light images and 328 enhanced images generated using the same methods in the fine-tuning phase; Test set B contains 41 low-light images and 164 images generated using four unseen enhancement methods, used to evaluate the model's adaptability to the new methods; Test set C contains 101 low-light images from an external dataset and 808 enhanced images generated using the same methods in the fine-tuning phase, used to test the model's generalization ability under unseen lighting conditions; Test set D contains the same 101 low-light images as Test set C, but uses four unseen enhancement methods to generate 404 images, used to jointly evaluate the model's robustness under the new methods and new lighting scenarios.

[0087] It should be noted that four independent test sets are constructed to evaluate the generalization performance of the model. Test set A contains image pairs with the same augmentation method and the same data distribution; test set B contains image pairs with the new augmentation method; test set C contains image pairs with the new data distribution; and test set D contains image pairs with the new augmentation method and the new data distribution. Test set A is used to verify the model's performance under the same augmentation method; test set B is used to verify the model's adaptability to the new augmentation method; test set C is used to verify the model's generalization ability under the new data distribution; and test set D is used to verify the model's joint robustness under the new augmentation method and the new data distribution.

[0088] Furthermore, regarding objective and subjective data annotation, during the pre-training phase, this invention automatically generates annotations by calculating the SSIM score between the enhanced image and its corresponding reference image. Specifically, this invention quantizes SSIM into five discrete quality levels, corresponding to the intervals [0,0.6), [0.6,0.73), [0.73,0.82), [0.82,0.90), and [0.90,1], and assigns them quality scores of 1, 2, 3, 4, and 5. By transforming the image quality regression task into a classification task, this invention can mitigate the adverse effects of the long-tail effect of continuous quality score distribution on model training. Moreover, since there is a certain deviation between the SSIM index and the true subjective quality of the image, discretizing the continuous SSIM scores can improve the reliability of the supervision signal and the training effect.

[0089] In the fine-tuning phase, this invention further evaluates the perceptual quality of the enhanced images through subjective testing and collects corresponding MOS scores. The specific steps are as follows: Twenty subjects with normal or corrected vision and experience in image quality assessment were invited to independently rate the visual quality of the enhanced images. Subjective quality assessment was conducted in a laboratory environment with constant lighting and professionally calibrated display equipment. For each test sample, the original low-light image and its corresponding enhanced image were presented side-by-side on a high-resolution display screen for direct visual comparison by the subjects. A standard five-point scale was used for scoring, where "5" indicates excellent visual quality and "1" indicates poor visual quality. The final subjective score (MOS) for each image was obtained by taking the arithmetic mean of all subjects' scores.

[0090] Table 1 provides an overview of the pre-training and fine-tuning datasets, including four independent test sets.

[0091]

[0092] In S2 of this embodiment of the invention, a two-stage training strategy is adopted, including a pre-training stage and a fine-tuning stage.

[0093] Specifically, such as Figure 2 As shown, the pre-training phase includes the following steps:

[0094] 1. Construct a large-scale pre-trained dataset containing low-light images. and its corresponding enhanced image Compared with reference image .

[0095] 2. Perform random initialization on the network.

[0096] 3. Randomly select a set of training samples from the dataset. , , ].

[0097] 4. Calculate the objective quality score using the Full Reference Image Quality Assessment Metric (SSIM):

[0098]

[0099] 5. To facilitate the model learning of more generalizable feature representations, continuous quality scores will be used. Quantized into K discrete levels:

[0100]

[0101] in, This represents the threshold set. By transforming the regression task into a classification task, the influence of the long-tailed distribution caused by continuous scores can be reduced. Furthermore, the SSIM value still has some error compared to the true image quality; discretization can improve the accuracy of the supervised information.

[0102] 6. with and As input, using a classification network Predicted quality grade distribution:

[0103]

[0104] 7. The network is optimized using the cross-entropy loss function:

[0105]

[0106] in, Indicates the number of pre-trained images. Indicates the first One sample was predicted to be the correct quality level. The probability of.

[0107] 8. Determine if the network has converged. If it has not converged, update the network parameters using the gradient descent algorithm and go back to step 3; if it has converged, end the training.

[0108] Specifically, such as Figure 3 As shown, the fine-tuning stage includes the following steps:

[0109] 1. Construct a dataset of a certain size containing low-light images. and its corresponding enhanced image The dataset, and on Subjective evaluation is conducted to obtain a subjective quality score.

[0110] 2. The network structure in the fine-tuning stage is the same as that in the pre-training stage, and the pre-trained weights are used for network initialization.

[0111] 3. Randomly select from the fine-tuning dataset and As input.

[0112] 4. Obtain the logits of the network classifier.

[0113] 5. Normalize logits to obtain the probability distribution:

[0114]

[0115] 6. Calculate the expected value, and obtain... mass fraction:

[0116]

[0117] 7. Calculate the loss function, which includes both absolute quality constraints and relative quality constraints.

[0118] a) Absolute quality constraint: The model is supervised by minimizing the absolute error between the predicted quality score and the subjective human score.

[0119]

[0120] in, It is a subjective rating by humans. To fine-tune the number of images.

[0121] b) Relative quality constraint: To ensure that the model can maintain the correct quality ranking between image pairs, this invention introduces a margin-ranking loss:

[0122]

[0123] in, and These represent the prediction scores of the two enhancement results for the same low-light image. For marginal control parameters, For sorting labels:

[0124]

[0125] The final fine-tuning loss function is a weighted sum of two parts:

[0126]

[0127] in, These are the weighting coefficients.

[0128] 8. Determine if the network has converged. If it has not converged, update the network parameters using the gradient descent algorithm and go back to step 3; if it has converged, end the training.

[0129] Furthermore, it should be noted that, in order to fully model the diverse distortions in low-light enhanced images, the image quality assessment network proposed in this invention is a dual-branch network structure with decoupled low-frequency and high-frequency branches. Its processing procedure is as follows:

[0130] 1. Input the original low-light image and its enhanced image It is decomposed into low-frequency components and high-frequency components through discrete wavelet transform (DWT);

[0131] 2. Unlike existing methods based on single encoders and convolutional neural networks (CNNs), this invention employs a dual-encoder structure to extract low-frequency and high-frequency features separately. Each encoder consists of three standard visual Transformer encoder modules. Average pooling is introduced between adjacent encoder modules to progressively reduce feature resolution, thereby improving computational efficiency while enhancing the network's global perception capability.

[0132] 3. Global average pooling and global max pooling are applied to low-frequency and high-frequency features respectively to obtain global representations. This operation enables the network to support image inputs of arbitrary sizes.

[0133] 4. Concatenate low-frequency and high-frequency features along the channel dimension, and map the features to image quality categories using two multilayer perceptrons.

[0134] Since the distortion of low-light enhanced images is distributed in both low-frequency and high-frequency information, the dual-branch network structure proposed in this invention can extract the distortion features in both frequency domains simultaneously, thereby significantly improving the prediction accuracy of low-light enhanced image quality.

[0135] It should be noted that all experiments in this invention were performed using an NVIDIA RTX3090 GPU within the PyTorch framework. During the pre-training phase, low-light images, enhanced images, and reference images were randomly cropped into 256×256 image patches, and their Structural Similarity Index (SSIM) values ​​were calculated. Continuous SSIM scores were quantized and used as supervision signals. The pre-trained model was trained for 100 epochs with a learning rate of 0.0001 and a batch size of 24. On the pre-training test set, this invention achieved PLCC of 0.891 and SRCC of 0.904, respectively. Subsequently, the model weights obtained in the pre-training phase were used as the initial values ​​for the fine-tuning phase. In the fine-tuning phase, the low-light images and enhanced images were input into the network at full size without cropping to ensure that the predicted quality score remained consistent with the MOS of the entire image. Fine-tuning training was performed for 100 epochs with a batch size of 1 and an initial learning rate of 0.0001, which was halved every 50 epochs. The loss function... Parameters and weights Set them to 0.5 and 0.5 respectively.

[0136] Furthermore, to verify the superiority of the proposed method, this invention employs Spearman's Rank Correlation Coefficient (SRCC) and Pearson Linear Correlation Coefficient (PLCC) to quantitatively evaluate model performance. PLCC measures the linear correlation between the predicted quality score and the actual subjective rating, reflecting the order-of-magnitude accuracy of the prediction. SRCC assesses the monotonic relationship between the predicted image quality ranking and the actual subjective ranking, reflecting the model's consistency in relative quality ranking. Higher SRCC and PLCC values ​​indicate better model performance.

[0137] As shown in Table 2 and Figure 4 As shown, this invention achieves optimal performance, and the evaluation results are more consistent with human subjective perception. In contrast, existing methods often suffer from insufficient generalization ability when applied to unseen enhancement algorithms or external datasets, while this invention remains robust under such challenging conditions, demonstrating its adaptability and reliability in practical applications. Traditional methods (NIQE and BRISQUE) rely on manually designed features and shallow regression models, resulting in limited performance. Although deep learning-based no-reference image quality assessment methods (CLIP-IQA and DeQA-Score) outperform traditional methods to some extent, they fail to specifically address the distortion types unique to low-light image enhancement, thus remaining insufficient in practical applications. Notably, DeQA-Score demonstrates superior performance on test set D. This excellent performance can be attributed to its integration of prior knowledge from a pre-trained large-scale language model, further illustrating the crucial role of pre-training strategies in improving the model's generalization ability to unseen low-light image enhancement methods and different lighting conditions.

[0138] Table 2 Performance Comparison

[0139]

[0140] To implement the methods of the above embodiments, this invention also provides a low-light image enhancement no-reference quality assessment device based on a pre-trained fine-tuning strategy, such as... Figure 5 As shown, the device includes:

[0141] The first module 100 is used to construct a pre-training dataset, which contains low-light images. Enhanced images generated from low-light images and the corresponding reference normal illumination image ;

[0142] The second module 200 is used to train the image quality assessment network using a two-stage training strategy of pre-training and fine-tuning. In the pre-training stage, the full reference image quality assessment index is calculated using the enhanced image and the corresponding reference image to obtain an objective quality score. The objective quality score is then discretized and quantized into multiple quality levels, transforming the image quality assessment regression task into a classification task, and the network is pre-trained. In the fine-tuning stage, the pre-trained network is fine-tuned using the low-light image, the corresponding enhanced image, and the subjective quality score of the enhanced image. At the same time, absolute quality constraints and relative quality constraints are introduced to optimize the model's prediction results.

[0143] The third module 300 is used to take the low-light image and the corresponding enhanced image as network input after training, and output the quality score of the enhanced image to realize the referenceless quality evaluation of the low-light enhanced image.

[0144] To implement the methods of the above embodiments, the present invention also provides an electronic device, which includes a memory and a processor; wherein the processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the various steps of the methods described above.

[0145] To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing embodiments.

[0146] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0147] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0148] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A referenceless quality assessment method for low-light image enhancement based on a pre-trained fine-tuning strategy, characterized in that, include: Construct a large-scale pre-trained dataset and a fine-tuned dataset of a certain size; The pre-training dataset includes low-light images, enhanced images generated from low-light images, and corresponding reference normal-light images; The fine-tuning dataset includes low-light images, enhanced images generated from the low-light images, and subjective quality scores for the enhanced images; A two-stage training strategy of pre-training and fine-tuning is employed to train the image quality assessment network. In the pre-training stage, a full-reference image quality assessment index is calculated using the enhanced image and its corresponding reference image to obtain an objective quality score. This objective quality score is then discretized and quantized into multiple quality levels, transforming the image quality assessment regression task into a classification task. The network is then pre-trained to learn general feature representations related to the quality of low-light enhanced images. In the fine-tuning stage, the pre-trained network is fine-tuned using the low-light image, the corresponding enhanced image, and the subjective quality scores of the enhanced image. Absolute and relative quality constraints are introduced simultaneously to optimize the model's prediction results. The fine-tuning stage also includes: constructing... Four independent test sets are used to evaluate the model's generalization performance. Test set A contains image pairs with the same augmentation method and data distribution; test set B contains image pairs with the new augmentation method; test set C contains image pairs with the new data distribution; and test set D contains image pairs with the new augmentation method and data distribution. Test set A verifies the model's performance under the same augmentation method; test set B verifies the model's adaptability to the new augmentation method; test set C verifies the model's generalization ability under the new data distribution; and test set D verifies the model's joint robustness under the new augmentation method and data distribution. The image quality evaluation network is a dual-branch network structure with low-frequency and high-frequency branches decoupled. Its processing procedure is as follows: input the original low-light image. and its enhanced image The features are decomposed into low-frequency and high-frequency components using discrete wavelet transform. A dual encoder structure is used to extract low-frequency and high-frequency features respectively. Each encoder consists of three layers of standard visual Transformer encoder modules, and average pooling is introduced between adjacent encoder modules. Global average pooling and global max pooling are applied to the low-frequency and high-frequency features respectively to obtain global representations. The low-frequency and high-frequency features are concatenated along the channel dimension, and the features are mapped to image quality categories through two multilayer perceptrons. After training, the low-light image and its corresponding enhanced image are used as network inputs, and the quality score of the enhanced image is output to achieve a referenceless quality evaluation of the low-light enhanced image.

2. The method according to claim 1, characterized in that, The pre-training phase includes: The image quality assessment network is randomly initialized; Randomly select a set of training samples from the pre-training dataset. , , ]; The objective quality score is calculated using the full-reference image quality assessment metric SSIM: continuous mass fraction Quantized into K discrete levels: in, For the threshold set; by and As input, using a classification network Predicted quality grade distribution: The image quality assessment network is optimized using the cross-entropy loss function, which is expressed as: in, Indicates the number of pre-trained images. Indicates the first One sample was predicted to be the correct quality level. The probability of; Determine if the network has converged. If not, update the network parameters using the gradient descent algorithm and randomly select training samples from the pre-training dataset again. If converged, end the training.

3. The method according to claim 2, characterized in that, The fine-tuning phase includes: Construct images containing low light and its corresponding enhanced image The dataset, and on Subjective evaluation is conducted to obtain a subjective quality score; The network is initialized using pre-trained weights, and the network structure in the fine-tuning stage is the same as that in the pre-training stage. Randomly selected from the fine-tuning dataset and As input; Obtain the logits of the network classifier; Normalizing logits yields the probability distribution: Calculate the expectation and obtain mass fraction: Calculate the loss function, including absolute quality constraints. Two parts: relative mass constraints The final fine-tuning loss function is a weighted sum of two parts: in, These are the weighting coefficients; Determine if the network has converged. If not, update the network parameters using the gradient descent algorithm and randomly select new parameters from the fine-tuning dataset. and If convergence is achieved, the training process ends.

4. The method according to claim 3, characterized in that, The absolute quality constraint supervises the model by minimizing the absolute error between the predicted quality score and the subjective human score, and is expressed as: in, It is a subjective rating by humans. To fine-tune the number of images; The relative quality constraint is the marginal ranking loss, expressed as: in, and These represent the prediction scores of the two enhancement results for the same low-light image. For marginal control parameters, For sorting labels.

5. A referenceless quality assessment device for low-light image enhancement based on a pre-trained fine-tuning strategy, characterized in that, include: The first module is used to construct a large-scale pre-trained dataset and a fine-tuned dataset of a certain size. The pre-training dataset includes low-light images, enhanced images generated from low-light images, and corresponding reference normal-light images; The fine-tuning dataset includes low-light images, enhanced images generated from the low-light images, and subjective quality scores for the enhanced images; The second module is used to train the image quality assessment network using a two-stage training strategy of pre-training and fine-tuning. In the pre-training stage, a full-reference image quality assessment index is calculated using the enhanced image and its corresponding reference image to obtain an objective quality score. This objective quality score is then discretized and quantized into multiple quality levels, transforming the image quality assessment regression task into a classification task. The network is then pre-trained to learn general feature representations related to the quality of low-light enhanced images. In the fine-tuning stage, the pre-trained network is fine-tuned using the low-light image, the corresponding enhanced image, and the subjective quality scores of the enhanced image. Simultaneously, absolute and relative quality constraints are introduced to optimize the model's prediction results. The fine-tuning stage also includes... The process includes: constructing four independent test sets to evaluate the model's generalization performance. Test set A contains image pairs with the same augmentation method and data distribution; test set B contains image pairs with the new augmentation method; test set C contains image pairs with the new data distribution; and test set D contains image pairs with the new augmentation method and data distribution. Test set A verifies the model's performance under the same augmentation method; test set B verifies the model's adaptability to the new augmentation method; test set C verifies the model's generalization ability under the new data distribution; and test set D verifies the model's joint robustness under the new augmentation method and data distribution. The image quality evaluation network is a dual-branch network structure with low-frequency and high-frequency branches decoupled. Its processing procedure is as follows: input the original low-light image. and its enhanced image The features are decomposed into low-frequency and high-frequency components using discrete wavelet transform. A dual encoder structure is used to extract low-frequency and high-frequency features respectively. Each encoder consists of three layers of standard visual Transformer encoder modules, and average pooling is introduced between adjacent encoder modules. Global average pooling and global max pooling are applied to the low-frequency and high-frequency features respectively to obtain global representations. The low-frequency and high-frequency features are concatenated along the channel dimension, and the features are mapped to image quality categories through two multilayer perceptrons. The third module is used to take the low-light image and the corresponding enhanced image as network input after training, and output the quality score of the enhanced image to achieve a referenceless quality evaluation of the low-light enhanced image.

6. An electronic device, characterized in that, Including processor and memory; The processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the method as described in any one of claims 1-4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-4.

Images