A low-light image enhancement method based on light intensity classification

Abstract

The present invention belongs to the field of image enhancement application technology, and relates to low-light images, and in particular to a low-light image enhancement method based on light intensity classification. The present invention uses multiple segments of images in different degrees of low-light scenes as a data set, which can obtain low-light images under different degrees, and constructs a low-light image mapping enhancement model based on the diffusion model framework. Different model structures are used according to the low-light level of the image, thereby achieving better enhancement effects and solving the problems of color distortion, detail loss, and other phenomena that affect the appearance of the enhancement results in traditional methods, as well as the problem of uneven exposure and model overfitting. At the same time, the two-stage enhancement model is more universal and better adaptable to low-light image restoration and enhancement under different degrees.

Description

Technical Field

[0001] The present invention belongs to the technical field of image enhancement applications, relates to low-illumination images, and in particular to a low-illumination image enhancement method based on light intensity classification. Background Art

[0002] In the field of image processing, low-light image enhancement, as an important basic task, lays the foundation for subsequent advanced image tasks such as image classification, target detection, image segmentation, and target tracking. It has been widely used in military, security, transportation and other fields.

[0003] Currently, low-light image enhancement methods can be roughly divided into those based on underlying images and those based on machine learning, based on their model structure. Low-light images are characterized by low brightness and contrast, resulting in poor detail recognition. From this perspective, the simplest and most direct way to increase image brightness is to use functions to adjust them, such as power functions and logarithmic functions. Histogram equalization, the most common method, stretches the spatial distribution of the image's histogram to increase the brightness of some pixels and enhance image contrast. Researchers have improved histogram equalization by adding various constraints, such as brightness and contrast limits, and image pyramids. Histogram equalization is now widely used in many image processing fields.

[0004] Most machine learning-based methods are implemented using a codec architecture. For example, LightenNet uses low-light images as input and normal-light images as labels to establish a one-to-one mapping, which is then used to enhance the image using a Retinex model. Another example is EnlightenGAN, which uses a Generative Adversarial Network (GAN) to construct an unpaired mapping between low-light and normal-light images, addressing the over-reliance of existing low-light enhancement methods on paired data.

[0005] While the aforementioned methods can address the brightness issues of low-light images, some challenges remain. For example, methods based on the underlying image suffer from color distortion and detail loss, which can affect the perceived quality of the enhanced results. Furthermore, uneven exposure remains a concern. Machine learning-based methods construct a supervised mapping between low-light and normal images. However, this approach uses a large data span, presenting only low-light and normal images, lacks a gradual progression, and is prone to overfitting. Summary of the Invention

[0006] In response to the technical problems existing in traditional low-light image enhancement, the present invention proposes a low-light image enhancement method based on light intensity classification, which is simple, easy to operate and can effectively solve the problems of color distortion, uneven exposure and overfitting.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is as follows: the present invention provides a low-light image enhancement method based on light intensity classification, comprising the following steps:

[0008] a. First, obtain an image of the same scene under normal lighting and multiple images under different degrees of low lighting;

[0009] b. Then, the acquired images are divided into groups and classified according to the degree of low illumination;

[0010] c. Allocate the image set into training set and test set according to the proportion;

[0011] d. Based on the diffusion model framework, the forward and reverse processes are improved, and the features between different levels are integrated to construct a low-light image mapping enhancement model;

[0012] e. Input the training set into the low-light image mapping enhancement model for training. After the training is completed, input the test set for verification. After passing the verification, the trained low-light image mapping enhancement model is obtained;

[0013] Finally, according to the classification results of step b, image enhancement models of different depths are constructed.

[0014] Preferably, in step d, the improvement scheme for the forward process of the diffusion model framework is: marking the label of the low-level image with a label one level higher than the low-level image.

[0015] Preferably, in the step d, the improvement scheme for the reverse process of the diffusion model framework is: the public DeepLabv3+ model is used as the hierarchical framework between the layers, and RepBlock and depth-separable deconvolution are integrated to construct the pixel-level mapping relationship between low-light images.

[0016] Compared with the prior art, the advantages and positive effects of the present invention are:

[0017] This method uses multiple images from scenes with varying degrees of low illumination as a dataset, capturing images of varying degrees of low illumination. It then constructs a low-illumination image mapping enhancement model based on a diffusion model framework. Different model structures are employed depending on the image's low illumination level, resulting in improved enhancement results. This approach addresses issues such as color distortion and detail loss that affect the perceived quality of the enhancement, as well as issues like uneven exposure and model overfitting. Furthermore, this two-stage enhancement model is more universal and better suited to low-illumination image restoration and enhancement at varying levels. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 Improved diagram of the forward process provided for the embodiment;

[0020] Figure 2 An operational diagram of the reverse process provided for an embodiment;

[0021] Figure 3 Diagram of the low-light image enhancement model training process;

[0022] Figure 4 This is the low-light image enhancement model diagram when n=3. DETAILED DESCRIPTION

[0023] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described below in conjunction with the accompanying drawings and embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0024] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways than those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0025] This embodiment provides a low-light image enhancement method based on light intensity classification.

[0026] First, an image of the same scene under normal lighting and multiple images under varying degrees of low lighting are acquired. In this embodiment, two image acquisition methods are used. The first method uses a fixed camera position to capture a fixed scene under different lighting conditions, capturing images at five different light intensity levels. This method primarily uses the light intensity of the lights to acquire the images. The second method uses a fixed camera to capture a fixed scene at different time periods, dividing the images into five levels. In this embodiment, the levels are artificially defined; however, six or four levels are also possible. In this embodiment, an artificial five-level classification is used for illustrative purposes.

[0027] The acquired images are then grouped. In this embodiment, the image set is U, which includes images N with normal illumination and images L with low illumination, with 1 to 5 levels and x pieces of image data.

[0028] That is: U=(N, L1, L2, L3, L4, L5)x

[0029] Then, the image set U is divided into training set and test set in a ratio of 8:2.

[0030] After classifying the images, it is necessary to construct image enhancement models of different categories, taking low illumination into consideration. Furthermore, considering that traditional image enhancement uses a single model for training regardless of the level of low illumination, this can lead to color distortion, detail loss, and other issues that affect the appearance of the enhanced results, as well as uneven exposure and model overfitting, a hierarchical image mapping enhancement model is provided in this embodiment. In this embodiment, the hierarchical image mapping enhancement model is implemented by setting up multiple layers of image mapping enhancement models according to the levels.

[0031] The image mapping enhancement model is based on the public diffusion model (DDPM) framework and its forward and backward processes are improved.

[0032] In the forward process, Figure 1 The dotted part from right to left is constructed using all the image training sets obtained. Different grayscale fillings in the box represent low-light images of different levels. The label of L5 is L4, the label of L4 is L3, the label of L3 is L2, the label of L2 is L1, and the label of L1 is N.

[0033] Reverse process Figure 1 From left to right, the solid line part uses the public DeepLabv3+ model as the hierarchical framework, integrating RepBlock, Deeply Separable Deconvolution (DSD), and Convolutional Block Attention Module (CBAM), named RDSD-DeepLabv3+ to build the pixel-level mapping relationship between low-light images, as shown in the figure. Figure 2 shown.

[0034] A detailed introduction to the RDSD-DeepLabv3+ model:

[0035] First, an encoder is constructed using RepBlock. Each RepBlock downsamples the input feature map by a factor of 2. The encoder reduces the size of the input low-light image by a factor of 16. The image is then fed into the ASPP multi-branch module to aggregate receptive fields of different scales. Multiple branches are combined through a splicing operation, and the number of channels is adjusted to 256 through a 1×1 convolution to obtain a high-level semantic feature map containing multiple scales.

[0036] Then, an encoder (Decode) is constructed, and two depth-wise separable deconvolution layers are used to upsample the obtained high-level semantic feature map by 4 times. The map is aligned with the shallow feature map obtained by the second RepBlock layer, and then feature connection is performed through Cancot. CBAM is used to obtain channel and spatial attention, effectively fusing the high-level semantic feature map and the shallow feature map.

[0037] Finally, the fused feature map containing high-level semantics and shallow features is upsampled 4 times through two depth-wise separable deconvolution layers to obtain the final low-light enhanced image.

[0038] The low-light image enhancement model training process is as follows Figure 3 In the training phase, the same RDSD-DeepLabv3+ model is used with parameter sharing. All image training sets are fed into the low-light image enhancement model. The learning rate change strategy uses warmup, and the loss function uses the KL loss function, as shown below. After 100 rounds of training, the final result is L n With L n-1 The mapping relationship between them is used to obtain the trained low-light image enhancement model.

[0039]

[0040] All image test sets are fed into the low-light image enhancement model, and the evaluation indicators PSNR and SSIM are used to judge the L n-1 · and L n-1 To distinguish, verify the effectiveness of the low-light image enhancement model. If it is effective, build image enhancement models of different depths according to the classification level N. Figure 4 Taking n=3 as an example, if the lowest category n=5 in this embodiment, there are 5 layers of low-light image enhancement models. In this way, by classifying the low-light image, the image enhancement model suitable for its category is selected. In this way, the processing of the entire image is more refined, and the color distortion, detail loss and other phenomena that affect the appearance of the enhancement result caused by the existing same processing are effectively avoided, as well as the problem of uneven exposure and model overfitting.

[0041] The above description is merely a preferred embodiment of the present invention and does not constitute any other form of limitation to the present invention. Any person skilled in the art may utilize the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes for application in other fields. However, any simple modification, equivalent change, and modification of the above embodiments made in accordance with the technical essence of the present invention without departing from the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A low-light image enhancement method based on light intensity classification, characterized in that: a. First, obtain an image of the same scene under normal lighting and multiple images under different degrees of low lighting; b. Then, the acquired images are grouped together and classified according to the degree of low illumination; c. Allocate the image set into training set and test set according to the proportion; d. Based on the diffusion model framework, the forward and reverse processes are improved, and the features between different levels are integrated to construct a low-light image mapping enhancement model; e. Input the training set into the low-light image mapping enhancement model for training. After the training is completed, input the test set for verification. After passing the verification, the trained low-light image mapping enhancement model is obtained; f. Finally, according to the classification results of step b, build image enhancement models of different depths Among them, in the step d, the improvement scheme for the forward process of the diffusion model framework is: marking the label of the low-level image with a label one level higher than the low-level image, and the improvement scheme for the reverse process of the diffusion model framework is: using the public DeepLabv3+ model as the hierarchical framework between levels, integrating RepBlock and depth-separable deconvolution to construct a pixel-level mapping relationship between low-light images.

Images