A method for constructing a training dataset for a low-light enhancement task

Abstract

This invention provides a method for constructing a training dataset for a low-light enhancement task. The method includes the following steps: S1. Image collection: Using richly colored close-up images taken under natural light with normal exposure as the label image (ground truth), darkening the scene brightness, and taking a series of images of the same scene with different brightness levels in different exposure ranges as training data, denoted as T. Here, the correspondence between the label image and the training image is 1 to n; S2. Further image augmentation: Here, the correspondence between the ground truth (gt) domain and the T domain is 1 to m (m > n); S3. Storage of augmented data: Using floating-point type to store the data after gamma transformation; S4. Adding noise: After converting the training images to HSV, randomly adding the required noise distribution; S5. Image cropping. This method improves the generalization ability of the training model, better handles real-world scenes, and the added noise is more consistent with the distribution of real-world noise, resulting in a good effect on adaptive denoising of the model.

Description

Technical Field

[0001] This invention relates to the field of low-light enhancement technology in low-level vision tasks of deep learning neural networks, and particularly to a method for constructing a training dataset for low-light enhancement tasks. Background Technology

[0002] Existing supervised deep learning-based low-light enhancement essentially learns a function mapping from the value domain of a dark image to the bright image domain. To improve image quality, tasks such as brightening and denoising are often accomplished using a single model. However, current mainstream training loss functions are often multi-task loss functions, and objective evaluation criteria such as structural similarity and peak signal-to-noise ratio (PSNR) often don't accurately reflect the visual effects. Therefore, adaptive learning by the network plays a crucial role, making the quality of the dataset critical. Datasets for low-light enhancement are difficult to collect; existing public datasets are relatively small and their noise simulations are often coarse.

[0003] Most solutions extract a portion from public datasets such as VOC and ImageNet, use gamma transform to obtain dark image data, and then simply add noise such as Gaussian or Poisson distribution to the dark image randomly.

[0004] Simply using gamma transformation to darken data often results in data with a very consistent distribution. In order to avoid information loss, the gamma value is also within a small range. This operation leads to poor generalization ability of the trained model and it cannot handle real-world scenes well. Simply adding noise cannot effectively remove image noise from real-world scenes that have undergone image signal processing.

[0005] In addition, the commonly used terms in the prior art are as follows:

[0006] Image Signal Processing (ISP): This unit is mainly used to process the output signals of the front-end image sensor to match image sensors from different manufacturers.

[0007] Short exposure and long exposure: By changing the exposure time, different quality data can be obtained. Short exposure time results in less light and a darker image, while long exposure time results in more light and a brighter image.

[0008] HSV (Hue, Saturation, Value): A color space created by ARSmith in 1978 based on the intuitive characteristics of color, also known as the hexagonal pyramid model. Summary of the Invention

[0009] To address the aforementioned issues, this application aims to: capture images of varying brightness levels within the same scene, from low to high, corresponding to a high-quality ground truth image; then perform a gamma transform on each training image and save it in floating-point format; simultaneously, add noise in the HSV domain to resolve the problem of weak generalization ability of the model training dataset and its inability to adapt to the denoising requirements of real-world low-light scenes, while also reducing the cost of collecting real training data.

[0010] Specifically, the present invention provides a method for constructing a training dataset for a low-light enhancement task, the method comprising the following steps:

[0011] S1. Image Collection:

[0012] A colored close-up image taken under natural light with normal exposure is used as the label image ground truth. The scene brightness is darkened. Darkening means adjusting the scene brightness to different ranges of [1, 50) lux. The specific step size can be determined according to actual needs. A series of images of the same scene with different brightness in different exposure ranges are taken as training data, denoted by T. The correspondence between the label image and the training image is 1 to n. Here, 1 to n means that the label image and the training image have a one-to-many relationship, and n means many. n is a positive integer, that is, one label image corresponds to n training images.

[0013] S2. Image further augmentation: Here, the gt domain, which is the label image domain, has a correspondence with the T domain of 1 to m (m>n), where m is a positive integer greater than n. This means that if there were originally n training images of the same scene, all corresponding to the same label image, these n images would become m images after data augmentation.

[0014] S3. Storage of augmented data:

[0015] Choose to use floating-point type to store the data after gamma transformation. RGB images are often stored using uint8, which is a number from 0 to 255. Gamma transformation often scales the image to between 0 and 1 before transformation. If uint8 is used to store the data, the value will drop to 0 after rounding, which will cause the original data information to be lost.

[0016] S4. Add noise:

[0017] After converting the training images to the hexagonal pyramid model HSV, the desired noise distribution is randomly added.

[0018] S5. Cropping the image.

[0019] In step S1, the normal exposure value may vary depending on the shooting device. The normal exposure should be such that there is no underexposure or overexposure. In this case, the exposure is EV0. The different exposure ranges include EV-4 to EV4. The images of the same scene with different brightness include images that are too dark and images that are too overexposure.

[0020] In step S2, gamma transform amplification of T is used:

[0021] S = Cr γ

[0022] C is a constant, usually 1, r is the normalized pixel value of the input, S is the result, and γ is the gamma coefficient.

[0023] In step S3, the transformed data format includes TIFF format.

[0024] In step S4, the noise is Gaussian noise or Poisson noise.

[0025] In step S4, the specific steps for adding noise are as follows:

[0026] S4.1, convert the image from the RGB domain to the HSV domain;

[0027] S4.2, randomly add Gaussian and Poisson noise;

[0028] S4.3, Gaussian blur the noise;

[0029] S4.4, Add noise to the original image in the HSV domain;

[0030] S4.5 converts the HSV domain back to the RGB domain.

[0031] In step S5, images are captured simultaneously on gt and its corresponding T using the same window and the same step size.

[0032] Therefore, the advantage of this application is:

[0033] 1. Low-light enhancement datasets have always faced the problem of being difficult to collect. Some real-shot datasets face the problem of data alignment difficulties, while fabricated datasets have weak generalization ability. This method uses a series of fixed steps to easily obtain a large amount of data that meets the training requirements, reducing costs.

[0034] 2. The addition of noise is more consistent with the distribution of real-world noise, and has a good effect on adaptive denoising of the model;

[0035] 3. The method of saving data will not result in information loss when the image is darkened;

[0036] 4. Many-to-one data allows the model training to better adapt to task requirements under different lighting conditions. Attached Figure Description

[0037] The accompanying drawings, which are provided to further illustrate the invention and form part of this application, are not intended to limit the scope of the invention.

[0038] Figure 1 This is a flowchart illustrating the method of the present invention.

[0039] Figure 2 This is a schematic diagram illustrating the construction of a dark light dataset according to the present invention. Detailed Implementation

[0040] To better understand the technical content and advantages of the present invention, the present invention will now be described in further detail with reference to the accompanying drawings.

[0041] This invention proposes a method for constructing a training dataset for a low-light enhancement task, such as... Figure 1 , 2 As shown, the process for constructing a low-light dataset is described below. The specific implementation steps of this method are as follows: Step S1. Image Collection

[0042] In practical applications, the goal is not merely to brighten images, but to brighten dark areas while controlling overexposure. Images taken in low-light conditions often suffer from various problems. Therefore, by varying the exposure time while keeping the ambient light adequate, one can obtain images with different brightness levels.

[0043] First, a near-field image with rich colors taken under natural light and normal exposure, such as (128), is used as the label image (ground truth), denoted by gt. The scene brightness is slightly darkened, and a series of images of the same scene with different brightness (including overexposed and underexposed images) with different exposure ranges, such as (5-256), are taken as training data, denoted by T. The correspondence between the label image and the training image is 1 to n.

[0044] In actual execution, a close-up image with rich colors is considered to be a colored close-up image if more than 80% of the image consists of scenes of different colors and the grayscale image (i.e., R:G:B is approximately 1:1:1) portion does not exceed 20%.

[0045] The dimming of the scene brightness is assumed to be about 50 lux after supplemental lighting (the natural light) (this value changes according to the actual scene conditions). Dimming means adjusting the scene brightness to different ranges of [1, 50), and the specific step size can be determined according to actual needs.

[0046] Step 2. Further image augmentation:

[0047] Amplifying T using gamma transform:

[0048] S = Cr γ

[0049] C is a constant, usually 1; r is the input normalized pixel value; S is the result; γ is the gamma coefficient; here the correspondence between the gt domain and the T domain is 1 to m (m > n).

[0050] Step S3. Storage of augmented data

[0051] Many current public datasets use gamma transformation to convert data into uint8 (0-255) numbers for storage. This causes the loss of information in the dark light areas, which is very detrimental to model training.

[0052] RGB images are often stored using uint8, which contains numbers from 0 to 255. Gamma transformation often scales the image down to between 0 and 1 before transforming it. Assuming the coefficient is 3, if a pixel in the original image has a value of 2, the normalized value is approximately 0.008. After a gamma transformation with a coefficient of 3, the value is approximately 0.00000012. If uint8 is used for storage, the value will be reduced to 0 after rounding, resulting in the loss of the original data. Therefore, using floating-point numbers can preserve as much image information as possible.

[0053] For example, if a pixel in the original image has a value of 1, it becomes 0.003921569 after normalization. After undergoing a gamma transform (assuming a coefficient of 2), the value will be reduced to 0, which is equivalent to a value between 0 and 255. Therefore, the method presented in this paper uses floating-point numbers to store the data after the gamma transform.

[0054] Step S4. Add noise

[0055] Image noise after image signal processing (ISP) no longer conforms to the original distribution because much of the noise is generated not in the RGB domain but in the HSV domain. This paper converts the training image to HSV and then randomly adds the desired noise distribution, such as Gaussian noise. To simulate the operation of image signal processing, the noise is Gaussian blurred before being added to the original image.

[0056] Step S5. Cropping the image

[0057] The captured images are generally very large. In actual training, it is neither necessary nor suitable to use excessively large images for training. Therefore, this paper chooses to use the same window and the same stride to crop the image at both the ground truth (gt) and the corresponding time step (T). This further expands the size of the dataset. Since the captured images selected in this paper are close-up images with rich colors, the information richness of the dataset is sufficient to meet the training requirements.

[0058] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for constructing a training dataset for a low-light enhancement task, characterized in that, The method includes the following steps: S1. Image Collection: Using normally exposed, colored close-up images taken under natural light as the ground truth, the scene brightness is darkened. Darkening refers to adjusting the scene brightness to different ranges within the range of [1, 50) lux, with the specific step size depending on actual needs. A series of images of the same scene with different brightness levels within different exposure ranges are taken as training data. T This indicates that the correspondence between the label graph and the training graph here is 1~ n , here 1~ n The relationship between the label image and the training images is one-to-many, where n stands for many and is a positive integer, meaning one label image corresponds to n training images. S2. Image further augmentation: gt The domain is the label graph domain, and... T The correspondence between the domains is 1~ m ,( m>n ), where m is a positive integer greater than n, means that if there were originally n training images of the same scene, each corresponding to the same label image, then the n images would become m images after data augmentation; In step S2, gamma transformation amplification is used. T : ； C It is a constant, usually 1 is sufficient. r The input is the normalized pixel value. S The result is given by γ, which is the gamma coefficient. S3. Storage of augmented data: Choose to use floating-point type to store the data after gamma transformation; RGB images are often stored using uint8. Gamma transformation often scales the image to between 0 and 1 before transformation. If uint8 is used for storage, the value will drop to 0 after rounding, which will cause the original data information to be lost. S4. Add noise: After converting the training images to the hexagonal pyramid model HSV, the desired noise distribution is randomly added. S5. Cropping the image: Choose to use the same window size and the same step size, while in gt and the corresponding T Capture the image above.

2. The method for constructing a training dataset for a low-light enhancement task according to claim 1, characterized in that, In step S1, the specific value of normal exposure may vary depending on the shooting device. Normal exposure should ensure that there is no underexposure or overexposure. In practice, the exposure under these conditions is EV0. The different exposure ranges include EV-4 to EV4. The images of the same scene at different brightness levels include both overexposed and underexposed images.

3. The method for constructing a training dataset for a low-light enhancement task according to claim 1, characterized in that, In step S3, the transformed data format includes TIFF format.

4. The method for constructing a training dataset for a low-light enhancement task according to claim 1, characterized in that, In step S4, the noise is Gaussian noise or Poisson noise.

5. The method for constructing a training dataset for a low-light enhancement task according to claim 4, characterized in that, In step S4, the specific steps for adding noise are as follows: S4.1, convert the image from the RGB domain to the HSV domain; S4.2, randomly add Gaussian and Poisson noise; S4.3, Gaussian blur the noise; S4.4, Add noise to the original image in the HSV domain; S4.5 converts the HSV domain back to the RGB domain.

Images