Low-illumination video image enhancement method based on automatic brightness perception

Abstract

A low-light video image enhancement method based on brightness perception is disclosed. First, a paired dataset of low-light images and corresponding reference images is constructed. Then, the images are converted to the HSV color space and the V channel is enhanced. The low-light images in the dataset are enhanced using the CLAHE algorithm and the cutting threshold is iterated to obtain the enhancement result. The optimal cutting threshold of the image is determined according to the evaluation result of the low-light enhanced image. The K-means algorithm is used to cluster the optimal cutting threshold and select the best one. The brightness perception of the image is performed using the convolutional neural network, and the clustering center of K-means is assigned to different brightness levels as the best cutting threshold. Finally, the CLAHE algorithm with adaptive cutting threshold is used for processing. This method combines deep learning methods and traditional image enhancement methods CLAHE algorithm, while ensuring the efficiency and enhancement effect of the algorithm.

Description

Technical Field

[0001] The invention relates to a low-illumination video image enhancement method based on automatic brightness perception, belongs to the technical field of image processing, and is suitable for improving the quality of monitoring images. Background Art

[0002] Images provided by surveillance footage under low-light conditions are often of low quality. Problems such as insufficient contrast and loss of detail are common defects in current image acquisition. Low-quality images fail to provide sufficient and accurate information, severely impacting subsequent analysis and processing of surveillance images and hindering the realization of corresponding application requirements. To address this, image enhancement technology can be used to pre-process surveillance images and improve image quality, making certain features more distinct and prominent. This technology is also more suitable for human visual characteristics or computer recognition and analysis, facilitating subsequent more advanced image processing and analysis to meet practical application requirements.

[0003] Currently, low-light image enhancement methods can be mainly divided into three categories: Retinex-based methods, histogram equalization-based methods, and convolutional neural network (CNN)-based methods. The first two methods are based on physical priors, and the latter is based on deep learning.

[0004] The Retinex algorithm is based on a model of illumination and reflectance separation in human visual perception. Single-scale Retinex (SSR) and Multi-scale Retinex (MSR) estimate illumination from the original image and then calculate reflectance as the enhancement result. These algorithms can preserve image details, but their enhancement effectiveness is significantly affected by the accuracy of illumination estimation. Equalizing the image histogram can enhance contrast and improve image quality. Global histogram equalization (GHE) is a classic image enhancement technique. While simple and effective, it is prone to noise enhancement and over-enhancement. Adaptive histogram equalization (AHE) can perform histogram equalization on local regions of an image, enhancing local details, but can still cause over-enhancement. Contrast-limited adaptive histogram equalization (CLAHE) is an improvement on the AHE algorithm, reducing noise and artifacts by limiting contrast in local regions. Algorithms based on convolutional neural networks can adaptively enhance image quality and are highly flexible. They can adjust the model structure according to different image conditions and task requirements and adapt to various types of low-light images. However, they are highly data-dependent and require a large amount of data for training. They have high computational costs and may lead to over-enhancement in certain situations, affecting the naturalness of the image. Summary of the Invention

[0005] The purpose of this invention is to provide a low-light image enhancement algorithm that can adaptively enhance contrast according to changes in the environmental scene, achieve video image clarity, reduce the impact of noise, and improve the robustness of video images to environmental changes. Based on the CLAHE method, this method uses video brightness perception to achieve adaptive implementation of the clipping threshold in the CLAHE algorithm. A set of adaptive low-light video enhancement techniques is proposed and established. Taking into account the efficiency of video image processing, the algorithm is optimized to improve video visual effects while ensuring real-time performance.

[0006] The working steps of the low-light video image enhancement method based on automatic brightness perception are:

[0007] (1) Select images from the LOL dataset and use Lightroom to change the exposure level of the images so that the image brightness is divided into 15 levels (1, 15) to obtain the image dataset;

[0008] (2) traversing the cropping threshold ranging from 0.001 to 0.099 with a step size of 0.002, converting the low-light image in the image dataset into the HSV space and processing the V channel in the HSV space using the contrast-limited adaptive histogram equalization (CLAHE) algorithm, and obtaining the optimal cropping threshold for the low-light image in the image dataset based on the image quality evaluation result;

[0009] (3) setting different numbers of cluster groups, clustering the optimal cropping threshold used in the low-light image enhancement process in the image dataset using the K-means algorithm, and determining the optimal number of cluster groups k using the elbow rule based on the clustering effect; and dividing the low-light images in the image dataset into k groups according to the brightness level, thereby obtaining a low-light image training dataset with brightness level labels; wherein k is a natural number;

[0010] (4) using the low-light image training data set to train a convolutional neural network with brightness perception function; using the convolutional neural network to perceive the brightness of the low-light image to be enhanced, using the center of the result of the K-means algorithm clustering as the clipping threshold, and after converting the low-light image to be enhanced into the HSV space, using the CLAHE algorithm based on the clipping threshold to enhance the V channel of the low-light image to be enhanced.

[0011] The beneficial effects of the present invention lie in the following: the proposed low-light image enhancement method trains a brightness-aware convolutional neural network, capable of determining image brightness under different scene conditions, thereby achieving adaptive implementation of the cropping threshold in the CLAHE algorithm. This approach enables the method to efficiently adapt to various low-light conditions and achieve significant results under different conditions. This improved CLAHE algorithm can significantly increase image contrast, improve low-light image quality, and maintain a certain efficiency to meet the real-time requirements of video image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 The following is a flow chart of a low-light video image enhancement method based on automatic brightness perception;

[0013] Figure 2 This is the result of the “elbow rule”;

[0014] Figure 3 The clustering results when k is 3, 4, and 5; (a) is the clustering result when k=3; (b) is the clustering result when k=4; (c) is the clustering result when k=5;

[0015] Figure 4 For low-light images;

[0016] Figure 5 The best threshold image is processed by traversal;

[0017] Figure 6 is the result of clustering threshold processing. DETAILED DESCRIPTION

[0018] The present invention provides a low-light video image enhancement method based on automatic brightness perception, the overall process is as follows: Figure 1 As shown, the specific implementation steps are as follows:

[0019] Step 1: Get the dataset. Gradually reduce the exposure of the normal brightness image to form images with different brightness levels as the dataset. Figure 4 shown.

[0020] Step 2: Crop threshold screening. Traverse the crop threshold and enhance the image using the CLAHE algorithm in HSV space. According to the image enhancement index, the best crop threshold is screened. The CLAHE algorithm enhancement result of the best crop threshold is as follows: Figure 5 shown.

[0021] Step 3: Clipping threshold clustering. Cluster the clipping threshold and reasonably compress the brightness levels in step 1 to obtain a new dataset.

[0022] Step 4: Image enhancement. Using the image dataset from step 3, train a brightness-aware neural network. Use this network to perceive image brightness and enhance images of different brightness levels.

[0023] Combined with attachment Figures 1 to 6 The technical solution of the present invention is further described in detail with reference to the implementation case: 1. Step 1: The specific steps of obtaining the image data set are as follows:

[0024] 400 images of normal brightness were selected from the LOL image dataset. Lightroom software was used to gradually reduce the exposure of the images to obtain 15 images with different brightness levels (1, 15). These images were labeled with brightness levels to obtain the initial image dataset. 2 Step 2: Crop threshold screening. The specific steps are as follows:

[0025] (1) Convert the image from RGB space to HSV space:

[0026] R′=R / 255 (2.1)

[0027] G′=G / 255 (2.2)

[0028] B′=B / 255 (2.3)

[0029] C max =max(R′,G′,B′) (2.4)

[0030] C min =min(R′,G′,B′) (2.5)

[0031] Δ=C max -C min (2.6)

[0032]

[0033] V=C max (2.9)

[0034] Where R, G, and B are the values ​​of the red, green, and blue channels of the pixel in the image, ranging from 0 to 255;

[0035] R′, G′, B′ are the normalized red, green and blue channel values, C max is the maximum value among R′, G′, and B′, and C min is the minimum value among R′, G′, and B′, and Δ is C max and C min H, S, and V are the values ​​of the hue, saturation, and lightness channels of the pixels in the image.

[0036] (2) Use the CLAHE algorithm of the HSV channel to enhance the image. First, divide the V channel image into 8×8 blocks. Then, calculate the histogram of each image block. Then, traverse all the clipping thresholds in the range of 0.001 to 0.099 with a step size of 0.002, clip the part of the histogram that exceeds the threshold, and evenly distribute the clipped part to all gray levels. Then equalize the clipped histogram. Perform bilinear interpolation on each image block. The enhanced V channel image is combined with the H and S channels and converted to RGB space.

[0037] (3) Calculate the SSIM of the enhanced image and select the cropping threshold when the SSIM is maximum as the optimal cropping threshold. The SSIM calculation method is as follows:

[0038]

[0039] Where N represents the number of pixels, x i represents the gray value of the pixel, μ represents the average gray value of the image, σ represents the standard deviation of the gray value, subscripts x and y represent the enhanced image and the reference image, σ xy It represents the covariance between the enhanced image and the reference image, reflecting the correlation between the two images. C1, C2, and C3 represent parameters. In this patent, C1, C2, and C3 can be expressed as:

[0040] C1=(k1·L) 2 (2.14)

[0041] C2=(k2·L) 2 (2.15)

[0042]

[0043] In this patent, k1 is set to 0.01, k3 is set to 0.03, and L is set to 255.

[0044] 3. Step 3: Clipping threshold clustering. The specific steps are as follows:

[0045] (1) K are randomly selected from the clipping threshold as the initial sample cluster centers. In order to test the number of cluster groups with better results, a total of 10 k values ​​from 1 to 10 are selected in the experiment.

[0046] (2) Then, each sample point is assigned to each centroid according to the distance from the sample point to the centroid, and the sample points are clustered according to the proximity principle to obtain K clusters.

[0047] (3) For each cluster, calculate the average distance of all sample points assigned to the cluster as the new centroid.

[0048] (4) Repeat steps 2 and 3 until the center of mass position no longer changes.

[0049] (5) Calculate the sum of squared errors (SSE) for different k value groups and select a more appropriate k value according to the “elbow rule”.

[0050]

[0051] Among them, K is the number of cluster groups, N j is the number of samples in each group, x i is the image cropping threshold, μ j is the mean of the cropping thresholds of each group of images. Figure 2 As shown, 3, 4, and 5 are gradually screened as appropriate k values.

[0052] (6) Further statistical clustering results are collected and the k value is further determined based on the number of samples in each group after clustering. Since the number of images of different brightness levels used for clustering is the same, and images of the same brightness level are basically classified into the same cluster, the number of brightness levels in each cluster can be inferred based on the number of samples, thereby obtaining a specific classification result. Figure 3 Figure (a) is the clustering result of three clusters, Figure (b) is the clustering result of four clusters, and Figure (c) is the clustering result of five clusters. Analysis shows that the clustering effect is best when k = 3, so the clustering result of k = 3 is selected.

[0053] (7) The image dataset obtained in step 1 is further classified according to the clustering result when k=3 to obtain a new image dataset.

[0054] 4 Step 4 The specific steps of image enhancement are as follows:

[0055] (1) The image dataset is divided into a test set and a training set at a ratio of 1 to 4. A convolutional neural network model is trained based on the training set images. The convolution layer extracts brightness features, the pooling layer reduces the feature dimension, and the fully connected layer maps the output category.

[0056] (2) Design a loss function and use the gradient descent algorithm to update the network parameters to minimize the loss function.

[0057] (3) Use convolutional neural network to perceive the brightness of the image, and use the cluster center of the clipping threshold in step 3 as the appropriate clipping threshold. Use the CLAHE algorithm in HSV space according to the method in step 2 to enhance the image. The specific effect is as follows Figure 6 shown.

Claims

1. A low-light video image enhancement method based on automatic brightness perception, characterized in that: The steps are as follows: (1) Select images from the LOL dataset and use Lightroom to change the exposure level of the images so that the image brightness is divided into 15 levels to obtain the image dataset; (2) Traversing the cropping threshold range from 0.001 to 0.099 with a step size of 0.002, the low-light image in the image dataset is converted to HSV space and the V channel in the HSV space is processed using the contrast-limited adaptive histogram equalization (CLAHE) algorithm. The enhanced V channel image is combined with the H and S channels and then converted to RGB space. The SSIM of the enhanced image is calculated, and the cropping threshold at which the SSIM is maximized is selected as the optimal cropping threshold. (3) setting different numbers of cluster groups, clustering the optimal cropping threshold used in the low-light image enhancement process in the image dataset using the K-means algorithm, and determining the optimal number of cluster groups k using the elbow rule based on the clustering effect; and dividing the low-light images in the image dataset into k groups according to the brightness level, to obtain a low-light image training dataset with brightness level labels; wherein k is a natural number; (4) Using the low-light image training data set to train a convolutional neural network with brightness perception function; using the convolutional neural network to perceive the brightness of the low-light image to be enhanced, classifying the image, using the center of the result of the K-means algorithm clustering as the cropping threshold, assigning it to images of different brightness levels, and after converting the low-light image to be enhanced into HSV space, using the CLAHE algorithm based on the center of the result of the K-means algorithm clustering as the cropping threshold to enhance the V channel of the low-light image to be enhanced.

2. The low-light video image enhancement method based on automatic brightness perception according to claim 1, characterized in that: The steps of converting the low-light image in the image dataset into the HSV space and processing the V channel in the HSV space using the contrast-limited adaptive histogram equalization (CLAHE) algorithm are as follows: (1) Separate the image into three channels: R, G, and B, and then convert them into H (hue), S (saturation), and V (value) channels; (2) Divide the V channel image into several blocks evenly; (3) Calculate the histogram of each small block; (4) Cut the part of the histogram that exceeds the threshold and evenly distribute the cut pixels to each gray level; (5) Perform histogram equalization on each image block; (6) Bilinear interpolation at the block boundaries; (7) Convert the image from HSV channel to RGB channel.

3. The low-light video image enhancement method based on automatic brightness perception according to claim 1, characterized in that: The optimal clipping threshold used in the low-light image enhancement process in the image dataset is clustered using the K-means algorithm, and the steps are as follows: (a) First, randomly select k centroids from n samples as the initial cluster centers; (b) Then assign each sample point to each centroid according to the distance from the sample point to the centroid, cluster the samples into the nearest centroid cluster, and obtain K clusters; (c) For each cluster, calculate the average distance of all sample points assigned to the cluster as the new centroid; (d) Repeat steps (b) and (c) until the center of mass position no longer changes; (e) According to the elbow rule, a suitable k value is selected. Based on the clustering results of the threshold, the original 15 brightness levels are compressed into k, and the images are reclassified to form a new image dataset. The cluster center of the cropping threshold is used as the optimal cropping threshold for each brightness level image.

4. The low-light video image enhancement method based on automatic brightness perception according to claim 1, characterized in that: The convolutional neural network is used to perceive the brightness of the low-light image to be enhanced and classify the image, and the steps are as follows: (1) The image dataset is divided into a test set and a training set in a ratio of 1 to 4. The training set images with brightness level labels are input into the neural network model. The brightness features are extracted through the convolution layer, the feature dimension is reduced through the pooling layer, and the output category is mapped through the fully connected layer; (2) Design a loss function and use the gradient descent algorithm to update the network parameters to minimize the loss function.

Images