Low-illumination video image enhancement method based on brightness automatic perception

Abstract

The invention discloses a low-illumination video image enhancement method based on brightness perception. The method comprises the following steps: firstly, constructing a paired data set of a low-illumination image and a corresponding reference image; converting the image into an HSV color space, enhancing a V channel, enhancing the low-illumination image in the data set by using a CLAHE algorithm, traversing a cutting threshold to obtain an enhancement result, and determining an optimal cutting threshold of the image according to an evaluation result of the low-illumination enhanced image; clustering and preferentially selecting the optimal cutting threshold value by using a K-means algorithm; performing brightness perception on the image by using a convolutional neural network, and allocating different brightness levels by taking a clustering center of K-means as an optimal cutting threshold value; and finally, processing by using a CLAHE algorithm of a self-adaptive clipping threshold. According to the method, a deep learning method and a traditional image enhancement method CLAHE algorithm are fused, and meanwhile the efficiency and the enhancement effect of the algorithm are guaranteed.

Description

Technical Field

[0001] The invention relates to a low-illuminance video image enhancement method based on automatic brightness perception, belongs to the technical field of image processing, and is suitable for improving the quality of monitoring images. Background Art

[0002] The images provided by the surveillance screen under low illumination conditions are often of low quality. Problems such as insufficient contrast and loss of details are common defects in current image acquisition. Low-quality images are difficult to provide sufficient and accurate information, which will seriously affect the subsequent analysis and processing of the surveillance images and is not conducive to achieving the corresponding application requirements. In this regard, pre-processing the surveillance images through image enhancement technology to improve the image quality can make some of the features more distinct and prominent, and more suitable for human visual characteristics or computer recognition analysis, which is convenient for subsequent more advanced processing and analysis of the images to meet the actual application requirements.

[0003] Currently, low-light image enhancement methods can be mainly divided into three categories: Retinex-based methods, histogram equalization-based methods, and convolutional neural network (CNN)-based methods. The first two methods are based on physical priors, and the latter is based on deep learning.

[0004] The Retinex algorithm is based on the illumination and reflection separation model of human visual perception. Single-scale Retinex (SSR) and multi-scale Retinex (MSR) estimate the illumination from the original image and then calculate the reflectance as the enhancement result. This type of algorithm can retain the details of the image, but its enhancement effect is greatly affected by the accuracy of illumination estimation. Equalizing the image histogram can enhance the contrast and improve the image quality. Global Histogram Equalization (GHE) is a classic image enhancement technology. This method is simple and effective, but it is prone to noise enhancement and over-enhancement problems; Adaptive Histogram Equalization (AHE) can perform histogram equalization on local areas of the image to enhance the local details of the image, but it may still have the problem of over-enhancement; Contrast-limited Adaptive Histogram Equalization (CLAHE) is an improvement on the AHE algorithm, which reduces noise and artifacts by limiting the contrast of local areas. Algorithms based on convolutional neural networks can adaptively enhance image quality and have strong flexibility. They can adjust the model structure according to different image conditions and task requirements and adapt to various types of low-light images. However, they are highly data-dependent and require a large amount of data for training. They have high computational costs and may lead to over-enhancement in certain situations, affecting the naturalness of the image. Summary of the invention

[0005] The purpose of the present invention is to provide a low-light image enhancement algorithm that can adaptively enhance contrast according to changes in environmental scenes, achieve video image clarity, reduce the impact of noise, and improve the robustness of video images to environmental changes. Based on the CLAHE method, the adaptive implementation of the clipping threshold in the CLAHE algorithm is achieved by perceiving the video brightness, and a set of adaptive low-light video enhancement technologies is proposed and established. Considering the efficiency of video image processing, the algorithm is optimized to improve the video visual effect while ensuring real-time performance.

[0006] The working steps of the low-light video image enhancement method based on automatic brightness perception are:

[0007] (1) Select images from the LOL dataset and use Lightroom to change the exposure level of the image so that the image brightness is divided into 15 levels (1, 15) to obtain an image dataset;

[0008] (2) traversing the cropping threshold ranging from 0.001 to 0.099 with a step size of 0.002, converting the low-light image in the image dataset into the HSV space and using the contrast-limited adaptive histogram equalization CLAHE algorithm to process the V channel in the HSV space, and obtaining the optimal cropping threshold of the low-light image in the image dataset according to the image quality evaluation result;

[0009] (3) setting different numbers of cluster groups, clustering the optimal cropping threshold used in the low-light image enhancement process in the image data set using the K-means algorithm, and determining the optimal number of cluster groups k using the elbow rule according to the clustering effect; and dividing the low-light images in the image data set into k groups according to the brightness level, to obtain a low-light image training data set with brightness level labels; wherein k is a natural number;

[0010] (4) using the low-light image training data set to train a convolutional neural network with brightness perception function; using the convolutional neural network to perceive the brightness of the low-light image to be enhanced, using the center of the result of the K-means algorithm clustering as the clipping threshold, and converting the low-light image to be enhanced into the HSV space, and then using the CLAHE algorithm based on the clipping threshold to enhance the V channel of the low-light image to be enhanced.

[0011] The beneficial effect of the present invention is that a brightness-aware convolutional neural network is trained in the proposed low-light image enhancement method, which can determine the brightness of the image under different scene conditions, thereby achieving the adaptive implementation of the clipping threshold in the CLAHE algorithm. This method enables the method to efficiently adapt to various low-light conditions and achieve significant results under different conditions. Through this improved CLAHE algorithm, the image contrast can be greatly increased, the low-light image quality can be improved, and a certain efficiency can be guaranteed to meet the real-time requirements of video image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 The flowchart of the low-light video image enhancement method based on automatic brightness perception is as follows;

[0013] Figure 2 This is the result of the “elbow rule”;

[0014] Figure 3 The clustering results when k is 3, 4, and 5; (a) is the clustering result when k=3; (b) is the clustering result when k=4; (c) is the clustering result when k=5;

[0015] Figure 4 For low-light images;

[0016] Figure 5 The best threshold image is processed by traversal;

[0017] Figure 6 is the result of clustering threshold processing. DETAILED DESCRIPTION

[0018] The present invention provides a low-light video image enhancement method based on automatic brightness perception, the overall process is as follows: Figure 1 As shown, the specific implementation steps are as follows:

[0019] Step 1: Obtain a data set. Gradually reduce the exposure of normal brightness images to form images with different brightness levels as a data set. Figure 4 shown.

[0020] Step 2: Crop threshold screening. Traverse the crop thresholds and enhance the image using the CLAHE algorithm in the HSV space. According to the image enhancement index, select the best crop threshold. The CLAHE algorithm enhancement result of the best crop threshold is as follows: Figure 5 shown.

[0021] Step 3: Clipping threshold clustering: Cluster the clipping thresholds and reasonably compress the brightness levels in step 1 to obtain a new data set.

[0022] Step 4: Image enhancement. Using the image data set in step 3, train a brightness-aware neural network. Use the network to perceive the brightness of the image and enhance images of different brightness levels.

[0023] Combined with Figures 1 to 6 The technical solution of the present invention is further described in detail with reference to the implementation cases: 1. The specific steps of obtaining the image data set in step 1 are as follows:

[0024] Select 400 images of normal brightness from the LOL image dataset, use Lightroom software to gradually reduce the exposure of the images, obtain 15 images of different brightness levels (1, 15), label them with brightness levels, and obtain the initial image dataset. 2 Step 2 The specific steps of cropping threshold screening are as follows:

[0025] (1) Convert the image from RGB space to HSV space:

[0026] R′=R / 255 (2.1)

[0027] G′=G / 255 (2.2)

[0028] B′=B / 255 (2.3)

[0029] C max =max(R′,G′,B′) (2.4)

[0030] C min =min(R′,G′,B′) (2.5)

[0031] Δ=C max -C min (2.6)

[0032]

[0033] V=C max (2.9)

[0034] Where R, G, B are the values ​​of the red, green, and blue channels of the pixels in the image, ranging from 0 to 255;

[0035] R′, G′, B′ are the normalized red, green and blue channel values, C max is the maximum value among R′, G′, and B′, and C min is the minimum value among R′, G′, and B′, and Δ is C max and C min H, S, and V are the values ​​of the hue, saturation, and brightness channels of the pixels in the image.

[0036] (2) Use the CLAHE algorithm of the HSV channel to enhance the image. First, divide the V channel image evenly into 8×8 blocks. Then, calculate the histogram of each image block. Then traverse all the clipping thresholds in the range of 0.001 to 0.099 with a step size of 0.002, clip the part of the histogram that exceeds the threshold, and evenly distribute the clipped part to all gray levels. Then equalize the clipped histogram. Perform bilinear interpolation on each image block. The enhanced V channel image is combined with the H and S channels and converted to RGB space.

[0037] (3) Calculate the SSIM of the enhanced image and select the cropping threshold when the SSIM is maximum as the optimal cropping threshold. The SSIM calculation method is as follows:

[0038]

[0039] Where N represents the number of pixels, x i represents the gray value of the pixel, μ represents the average gray value of the image, σ represents the standard deviation of the gray value, and the subscripts x and y represent the enhanced image and the reference image. xy represents the covariance between the enhanced image and the reference image, reflecting the correlation between the two images. C1, C2, and C3 represent parameters. In this patent, C1, C2, and C3 can be expressed as:

[0040] C1=(k1·L) 2 (2.14)

[0041] C2=(k2·L) 2 (2.15)

[0042]

[0043] In this patent, k1 is 0.01, k3 is 0.03, and L is 255.

[0044] 3 Step 3 Clipping threshold clustering The specific steps are as follows:

[0045] (1) K are randomly selected from the clipping threshold as the initial sample cluster centers. In order to test the number of cluster groups with better results, a total of 10 k values ​​from 1 to 10 are selected in the experiment.

[0046] (2) Then, each sample point is assigned to each centroid according to the distance from the sample point to the centroid, and the sample points are clustered according to the proximity principle to obtain K clusters.

[0047] (3) For each cluster, calculate the average distance of all sample points assigned to the cluster as the new centroid.

[0048] (4) Repeat steps 2 and 3 until the center of mass position no longer changes.

[0049] (5) Calculate the sum of squared errors (SSE) for different k value groups and select a more appropriate k value based on the “elbow rule”.

[0050]

[0051] Among them, K is the number of cluster groups, N j is the number of samples in each group, x i is the image cropping threshold, μ j is the mean of the cropping thresholds of each group of images. Figure 2 As shown, 3, 4, and 5 are gradually screened as appropriate k values.

[0052] (6) Further statistical clustering results are obtained, and the k value is further determined based on the number of samples in each group after clustering. Since the number of images of different brightness levels used for clustering is the same, and images of the same brightness level are basically classified into the same cluster, the number of brightness levels in each cluster can be inferred based on the number of samples, thereby obtaining specific classification results. Figure 3 Figure (a) is the clustering result of three clusters, Figure (b) is the clustering result of four clusters, and Figure (c) is the clustering result of five clusters. Analysis shows that the clustering effect is best when k=3, so the clustering result of k=3 is selected.

[0053] (7) The image data set obtained in step 1 is further classified according to the clustering result when k=3 to obtain a new image data set.

[0054] 4 Step 4 The specific steps of image enhancement are as follows:

[0055] (1) The image dataset is divided into a test set and a training set at a ratio of 1 to 4. A convolutional neural network model is trained based on the training set images. The convolutional layer extracts brightness features, the pooling layer reduces the feature dimension, and the fully connected layer maps the output category.

[0056] (2) Design the loss function and use the gradient descent algorithm to update the network parameters to minimize the loss function.

[0057] (3) Use a convolutional neural network to perceive the brightness of the image, and use the cluster center of the clipping threshold in step 3 as the appropriate clipping threshold. Use the CLAHE algorithm in HSV space according to the method in step 2 to enhance the image. The specific effect is as follows Figure 6 shown.

Claims

1. A low-light video image enhancement method based on automatic brightness perception, characterized in that: The steps are as follows: (1) Select images from the LOL dataset and use Lightroom to change the exposure level of the image so that the image brightness is divided into 15 levels (1, 15) to obtain an image dataset; (2) traversing the cropping threshold ranging from 0.001 to 0.099 with a step size of 0.002, converting the low-light image in the image dataset into the HSV space and using the contrast-limited adaptive histogram equalization CLAHE algorithm to process the V channel in the HSV space, and obtaining the optimal cropping threshold of the low-light image in the image dataset according to the image quality evaluation result; (3) setting different numbers of cluster groups, clustering the optimal cropping threshold used in the low-light image enhancement process in the image data set using the K-means algorithm, and determining the optimal number of cluster groups k using the elbow rule according to the clustering effect; and dividing the low-light images in the image data set into k groups according to the brightness level, to obtain a low-light image training data set with brightness level labels; wherein k is a natural number; (4) using the low-light image training data set to train a convolutional neural network with brightness perception function; using the convolutional neural network to perceive the brightness of the low-light image to be enhanced, using the center of the result of the K-means algorithm clustering as the clipping threshold, and converting the low-light image to be enhanced into the HSV space, and then using the CLAHE algorithm based on the clipping threshold to enhance the V channel of the low-light image to be enhanced.

2. The low-light video image enhancement method based on automatic brightness perception according to claim 1, characterized in that: The steps of obtaining the optimal cropping threshold of the low-light image in the image data set according to the image quality evaluation result in (2) are as follows: (1) Separate the image into three channels: R, G, and B, and then convert them into H (hue), S (saturation), and V (value) channels; (2) Divide the V channel image evenly into several blocks; (3) Calculate the histogram of each small block; (4) Cut the part of the histogram that exceeds the threshold and evenly distribute the cut pixels to each gray level; (5) Perform histogram equalization on each image block; (6) Bilinear interpolation at the block boundaries; (7) Convert the image from HSV channel to RGB channel; (8) Use an objective low-light enhancement evaluation method to evaluate the enhanced result image and screen out the optimal cropping threshold.

3. The low-light video image enhancement method based on automatic brightness perception according to claim 1, characterized in that: The optimal clipping threshold used in the low-light image enhancement process in the image data set described in (3) is clustered using the K-means algorithm, and the steps are as follows: (1) First, randomly select k centroids from n samples as the initial cluster centers; (2) Then, each sample point is assigned to each centroid according to the distance from the sample point to the centroid, and the samples are clustered into the nearest centroid cluster to obtain K clusters; (3) For each cluster, calculate the average distance of all sample points assigned to the cluster as the new centroid; (4) Repeat steps (2) and (3) until the center of mass position no longer changes; (5) According to the "elbow rule", select the appropriate k value. According to the clustering result of the threshold, compress the original 15 brightness levels into k, reclassify the image, and form a new image data set; and use the cluster center of the cropping threshold as the optimal cropping threshold for each brightness level image.

4. The low-light video image enhancement method based on automatic brightness perception according to claim 1, characterized in that: (4) Construct a neural network to classify images. The steps are as follows: (1) The image dataset is divided into a test set and a training set in a ratio of 1 to 4. The training set images with brightness level labels are input into the neural network model. The brightness features are extracted through the convolution layer, the feature dimension is reduced through the pooling layer, and the output category is mapped through the fully connected layer; (2) Design a loss function and use the gradient descent algorithm to update the network parameters to minimize the loss function; (3) Using a convolutional neural network to perceive the brightness of the image, selecting the center of the cropping threshold cluster in claim 3 as the appropriate cropping threshold to be assigned to images of different brightness levels, and using the CLAHE algorithm in the HSV space according to the method in claim 2 to enhance the image.

Images