A method for simulating dark band noise from low-light enhanced training data

Abstract

The application provides a method for simulating dark area color strip noise by low-light enhancement training data, wherein the luminance range of a dark area is extracted on a dark image, and according to the range, noise is randomly generated by simulating the distribution of color strip noise of a real scene on the RGB three channels of the dark area of the dark image. The method solves the problem that the model training cannot adapt to the denoising requirement of a real dark-light scene, and reduces the cost of collecting real training data. The method comprises the following steps: S1, image pre-statistics; S2, image light and dark partition; S3, dark area luminance statistics; S4, noise image template generation; S5, noise generation; S6, image splicing, and noise synthesis is completed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of low-light enhancement in low-level visual tasks of deep learning neural networks, and particularly relates to a method for simulating dark color bar noise of low-light enhancement training data in a real scene. BACKGROUND

[0002] The existing supervised deep learning low-light enhancement is essentially learning a function mapping from the value domain of dark light images to the bright domain. In order to improve image quality, brightening, denoising and other tasks are expected to be completed based on a model. At present, the mainstream training loss function is mostly based on signal-to-noise ratio or structural similarity, which is a loss function for some comprehensive tasks. Therefore, the denoising part is mainly relied on the adaptive ability of the model, which requires the noise of the data set to be as close to the real scene as possible, and a large number of strictly aligned paired data are needed for training. It is difficult to obtain enough noise data through real shooting, so artificial synthesis becomes a cost-effective solution. The current main method is to randomly add noise such as Gaussian distribution and Poisson distribution to dark light images in the network adaptive denoising aspect.

[0003] Most of the schemes are to randomly add noise such as Gaussian distribution and Poisson distribution to dark light images. However, simply adding noise such as Gaussian noise cannot simulate dark light images in real scenes. Common noise has changed its original distribution after image signal processing, and the current synthetic data method does not simulate the noise that appears as color stripes after dark area image signal processing, which causes the quality of the enhanced dark light image to decrease seriously and cannot meet the low-light enhancement task in real scenes.

[0004] In addition, the common terms in the prior art are as follows:

[0005] Image signal processing: mainly used for units for processing signals output by front-end image sensors to match image sensors of different manufacturers.

[0006] Binaryzation: is the simplest method of image segmentation. Binaryzation can convert a grayscale image into a binary image. The pixel grayscale greater than a certain critical grayscale value is set to the maximum grayscale, and the pixel grayscale less than the value is set to the minimum grayscale, thereby realizing binaryzation.

[0007] Low-level vision: refers to some image enhancement tasks in deep learning, which mainly focuses on extracting the characteristics of images from retinal images. SUMMARY

[0008] To solve the above problems, the purpose of the present application is to extract the brightness range of the dark area on a dark map, and to generate random noise according to the distribution of the color strip noise in the RGB three channels of the dark area of the dark map. The model training cannot adapt to the denoising needs of the real dark light scene, and the cost of collecting real training data is reduced.

[0009] Specifically, the present application provides a method for simulating dark strip noise in low-light enhanced training data, which comprises the following steps:

[0010] S1. Image pre-statistics:

[0011] The average brightness of the region where the color strip noise appears is obtained by statistical analysis of the color strip noise in the real scene. The color strip noise is in the dark area of the image, and its brightness is in [1, 10], where the value of the RGB image is in the value range of [0, 255] of the whole image;

[0012] S2. Image light-dark partition: obtain a Mask image by binaryzation;

[0013] S3. Statistics of dark area brightness:

[0014] S3.1, obtain the dark area extraction Mask1 by inverting the light-dark partition Mask;

[0015] S3.2, obtain the brightness feature map of the image according to the formula brightness=R*0.299+G*0.587+B*0.114;

[0016] S3.3, multiply brightness and Mask1 to obtain the brightness feature map of the dark area of the image, which is represented by brightness dark ;

[0017] S3.4, calculate the mean value of the brightness of the dark area of the image, Where the numerator represents the sum of all pixels of the dark area brightness feature map, and the denominator represents the number of pixels of the dark area brightness feature map;

[0018] S4. Generate noise map template:

[0019] Generate a graph with the same size as the original graph and value 0 as the template of the dark graph;

[0020] S5. Generate noise:

[0021] S5.1, randomly select coordinates (x, y, z) on the noise map template N, x∈(0, width), y∈(0, height), z∈(0, 3);

[0022] S5.2, according to the coordinates to build a vector (width-x, y, z), (x, height-y, z);

[0023] S5.3, according to the vector in N, with mean as the value, width as 1, draw the vector;

[0024] S5.4, repeat S5.1-S5.3, draw n times according to the empirical value, n e (200, 500);

[0025] S5.5, multiply N with Mask1, that is, obtain the color strip noise graph containing only the dark area;

[0026] S6. Image stitching:

[0027] Directly perform pixel addition operation on the original image and the image obtained in step S5.5, that is, obtain the stitching image of the two images, and complete the noise synthesis.

[0028] The one Mask graph in the step S2 is understood as a light and dark graph, the pixel points with image brightness higher than the threshold value are set to 1, and the pixel points with image brightness lower than the threshold value are set to 0, here referring to the value of the common RGB image, the value range of the whole image is in [0, 255].

[0029] The step S2 further comprises:

[0030] S2.1, according to the value range of the color strip noise appearing in the real scene in the dark image, it is statistically found that the color strip noise is in the dark area of the image, and the brightness is about [1, 10], here referring to the value of the common RGB image, the value range of the whole image is in [0, 255], the image is binarized, that is, the pixel points with image brightness higher than the threshold value are set to 1, and the pixel points with image brightness lower than the threshold value are set to 0, the threshold value used in the implementation process is 15, here referring to the value of the common RGB image, the value range of the whole image is in [0, 255];

[0031] S2.2, the binarized graph is taken as a Mask graph, that is, when the Mask graph is multiplied with the original graph, the bright area does not change, and the dark area is 0.

[0032] In the step S3.1, the bright and dark partition Mask is inverted, since the Mask has only two values 0 and 1, 1-MaskM is used to invert, Mask1 is multiplied with the original graph, and the obtained is the part of the dark area of the original graph, so Mask1 is understood as the dark area.

[0033] In the step S4, the template is a graph with the same size as the original graph and the value of 0, the noise is constructed on the template, so that the noise can be directly added to the original graph without affecting other areas of the original graph, and the image containing the noise is obtained.

[0034] The step S6, the pixel addition operation: construct the related formula F=I+Noise, wherein F is the synthesized image with noise, I is the original image, and Noise is the noise image constructed in S5.

[0035] Therefore, the advantages of the present application are:

[0036] 1. Since the noise in the real dark light scene is not simply consistent with the theoretical Gaussian, Poisson and other noises due to the influence of various factors, through the light-dark binary processing and the statistics of the dark area brightness, the color strip noise consistent with the real scene is well simulated on the image. Through actual application, the purpose of adaptively removing such noise can be well achieved, and the blank of such noise synthesis is filled.

[0037] 2. The operation process is simple, and can be directly used as an image preprocessing step before training.

[0038] 3. The noise is controllable. Since the statistical experience value basically meets the needs of most scenes, the parameters can be adjusted according to the required effect, and the needs can be well met. BRIEF DESCRIPTION OF DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present application, constitute a part of the present application, and do not constitute a limitation of the present application.

[0040] Figure 1 is a flowchart of the method of the present application.

[0041] Figure 2 is a flowchart of constructing a dark light noise image in the method of the present application. DETAILED DESCRIPTION

[0042] In order to more clearly understand the technical content and advantages of the present application, the present application will be further described in detail in conjunction with the drawings.

[0043] The present application relates to a method for generating dark area color strip noise in low light enhancement training data under a real scene, i.e. a method for generating training data simulating real image signal processing to capture dark area in the form of color strip noise similar to a smear, as shown in Figure 1 The method for synthesizing dark light image color strip noise is shown in the figure, and the specific implementation steps of the method are as follows:

[0044] Step S1. Image pre-statistics

[0045] Step S1. Statistics of color strip noise in real scene

[0046] S2.1, according to the statistics of the value range of color strip noise in real scene, the color strip noise is in the dark area of the image, and the brightness is about [1, 10], here referring to the value of common RGB image, the value range of the whole image is [0, 255], the image is binarized, the pixel points with brightness higher than the threshold value are set to 1, and the pixel points with brightness lower than the threshold value are set to 0, the threshold value used in the implementation process is 15, here referring to the value of common RGB image, the value range of the whole image is [0, 255];

[0047] S2.2, the binarized image is taken as a Mask image, that is, when the Mask image is multiplied by the original image, the bright area remains unchanged and the dark area is 0.

[0048] Step S3. Statistics of dark area brightness

[0049] S3.1, the inverse of the light and dark partition Mask is obtained, that is, the dark area extraction Mask1, since the Mask has only two values 0 and 1, the inverse can be obtained by using 1-Mask; multiplying Mask1 and the original image, the result is the dark area of the original image, so Mask1 can also be understood as the dark area;

[0050] S3.2, according to the formula brightness = R*0.299 + G*0.587 + B*0.114, the brightness feature map of the image is obtained;

[0051] S3.3, multiplying brightness and Mask1, the brightness feature map of the dark area of the image is obtained, which is represented by brightness dark ;

[0052] S3.4, calculating the mean value of the brightness of the dark area of the image mean, Where the numerator represents the sum of all pixels of the dark area brightness feature map, and the denominator represents the number of pixels of the dark area brightness feature map;

[0053] Step S4. Generating noise map template

[0054] A graph with the same size as the original image and value 0 is generated as the template N of the dark graph; the template is a graph with the same size as the original image and value 0, and the noise is constructed on this template, so that the noise can be directly added to the original image without affecting other areas of the original image, and the image containing the noise can be obtained;

[0055] Step S5. Generate noise

[0056] S5.1 Randomly select coordinates (x, y, z) on the noise graph template N, where x∈(0, width), y∈(0, height), and z∈(0, 3).

[0057] S5.2 Construct vectors (width-x, y, z) and (x, height-y, z) based on coordinates;

[0058] S5.3 Draw a vector on N with the value of mean and a width of 1;

[0059] S5.4 Repeat steps S5.1-S5.3, drawing n times based on empirical values, where n∈(200, 500);

[0060] S5.5 Multiply N by Mask1 to obtain the color bar noise map contained only in the dark areas; Step S6. Image stitching

[0061] By directly performing pixel addition operations on the original image and the image obtained in step S5.5, a stitched image of the two images can be obtained. The pixel addition operation is based on the formula F = I + Noise, where F is the synthesized image with noise, I is the original image, and Noise is the noise image constructed in step S5.

[0062] In summary, such as Figure 2 As shown, a mask is generated from the dark light image, then the mask is inverted, the brightness of the dark area is calculated based on the dark light image, and a noise template is generated. Finally, a noise map is generated. The noise map and the dark light image are stitched together to obtain a dark light noise image.

[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for simulating the generation of dark area color bar noise using low-light enhancement training data, characterized in that, The method includes the following steps: S1. Image Prestatistics: Statistical analysis was performed on the colored bar noise in real-world scenes to obtain the average brightness of the areas where colored bar noise appeared. According to the statistics, the colored bar noise is located in the dark area of ​​the image, and its brightness is in the range of [1, 10]. Here, the value of the RGB image is used, and the value range of the entire image is in the range of [0, 255]. S2. Image brightness and darkness partitioning: Obtain a mask image using binarization; S3. Statistical analysis of dark area brightness: S3.1, Invert the mask for the light and dark partitions to obtain the dark area extraction mask1; S3.2, according to the formula brightness=R*0299+G*0.587+B*0.114, the brightness feature map of the image is obtained; S3.3, multiply brightness by Mask1 to obtain the brightness feature map of the dark areas of the image. dark express; S3.4, Calculate the mean brightness of the dark areas in the image. The numerator represents the sum of all pixels in the dark area brightness feature map, and the denominator represents the number of pixels in the dark area brightness feature map. S4. Generate a noise map template: Generate an image of the same size as the original image, with a value of 0, as a template for the dark image; S5. Noise generation: S5.1, randomly select coordinates (x, y, z) on the noise map template N, where x∈(0, width), y∈(0, height), and z∈(0, 3); S5.2, construct vectors (width-x, y, z) and (x, height-y, z) based on the coordinates; S5.3, Draw the vector with mean as its value and width of 1, given that the vector is on N; S5.4, Repeat steps S5.1-S5.3, and draw n times based on empirical values, where n∈(200,500); S5.5 Multiply N by Mask1 to obtain the color bar noise map that is only present in the dark area; S6. Image stitching: By directly performing pixel addition operations on the original image and the image obtained in step S5.5, a stitched image of the two images can be obtained, thus completing noise synthesis.

2. The method for simulating dark area color bar noise using low-light enhancement training data according to claim 1, characterized in that, In step S2, a Mask image is understood as a brightness and darkness image. Pixels with brightness higher than a threshold are set to 1, and those with brightness lower than the threshold are set to 0. This refers to the values ​​of a common RGB image, and the value range of the entire image is [0, 255].

3. The method for simulating dark area color bar noise using low-light enhancement training data according to claim 2, characterized in that, Step S2 further includes: S2.1, based on the statistically determined value range of colored bar noise in the real scene on the low-light image, it is found that the colored bar noise is located in the dark area of ​​the image, and its brightness is approximately [1, 10]. Here, it refers to the value of a common RGB image, and the value range of the entire image is [0, 255]. The image is binarized, and pixels with brightness higher than the threshold are set to 1, and those with brightness lower than the threshold are set to 0. The threshold used in the implementation is 15, which refers to the value of a common RGB image, and the value range of the entire image is [0, 255]. S2.2, the binarized image is used as a mask image, that is, when the original image is multiplied by this mask image, the bright area remains unchanged and the dark area is 0.

4. The method for simulating dark area color bar noise using low-light enhancement training data according to claim 1, characterized in that, In step S3.1, the Mask of light and dark partitions is inverted. Since Mask only has two values, 0 and 1, it can be inverted by using 1-Mask. Multiplying Mask1 by the original image gives the dark part of the original image, so Mask1 can be understood as the dark area.

5. The method for simulating dark area color bar noise using low-light enhancement training data according to claim 1, characterized in that, In step S4, the template is a generated image with the same size and value of 0 as the original image. Noise is built on this template so that noise can be directly added to the original image without affecting other areas of the original image, thus obtaining an image containing noise.

6. The method for simulating dark area color bar noise using low-light enhancement training data according to claim 1, characterized in that, In step S6, the pixel addition operation involves constructing the relevant formula F = I + Noise, where F is the synthesized image with noise, I is the original image, and Noise is the noise map constructed in step S5.

Images