Method for enhancing contrast ratio of RGB domain based on LTMNET network

Abstract

The invention provides an RGB domain contrast enhancement method based on an LTMNET network, and the method comprises the steps: S1, obtaining a low-contrast image: collecting and sorting low-contrast-high-contrast image sample pairs, and constructing a training set and a test set; s2, clipping (512, 512, 3): converting the image data of the sample pair into the size (512, 512, 3) as the input and output size of network training; s3, constructing an LTMNET network model with enhanced and adjustable RGB (Red, Green, Blue) domain contrast ratio; S3.1, optimizing an LTMNET network; s3.2, processing is carried out after interpolation; s3.3, the loss is calculated; and S4, updating parameters. A low-contrast image is sent into a deep learning network to learn a mapping curve, and a high-contrast image is obtained through interpolation post-processing, so that an efficient and generalization RGB domain contrast enhancement technical method is obtained.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a method for contrast enhancement in RGB domain based on an LTMNET network. Background Art

[0002] In the prior art, with the widespread application of computer vision technology, high-quality images are not only used for people's entertainment, but many visual tasks also require high-quality images as support to achieve. Contrast enhancement plays an important role in improving the visual quality of computer vision, pattern recognition and digital image processing. In actual scenes, due to various conditions such as harsh shooting environment and difficult to set camera parameters, the images obtained by the imaging device may have problems such as low contrast and lack of details. At present, low-contrast image enhancement mainly adopts traditional algorithms or is learned as a subtask of deep learning image enhancement, which is cumbersome and has a significant brightness improvement effect, and the enhancement intensity cannot be effectively adjusted.

[0003] The disadvantages of the prior art are:

[0004] There are many classic enhancement methods in the current contrast enhancement research, such as gamma transformation and histogram equalization for global grayscale adjustment. Although these methods are efficient, they cannot achieve adaptive contrast enhancement. Grayscale adjustment based on local conditions, such as contrast-limited adaptive histogram equalization and adaptive contrast enhancement, can alleviate the problems caused by global grayscale adjustment and improve local contrast effects. However, these methods can only be used for single image processing, and the same parameters cannot be adapted to different scenes. That is, there are problems of low contrast, low transparency, and low saturation.

[0005] In addition, common terms in the prior art include:

[0006] Contrast: A measurement of the difference in brightness between the brightest white and the darkest black in the light and dark areas of an image.

[0007] RGB: The three optical primary colors, R, G, and B stand for red, green, and blue respectively. Summary of the invention

[0008] In order to solve the above problems, the purpose of this application is to propose a method of sending a low-contrast image into a deep learning network to learn a mapping curve, and obtaining a high-contrast image through interpolation post-processing, so as to obtain an efficient and generalized RGB domain contrast enhancement technology method.

[0009] Specifically, the present invention provides a method for contrast enhancement in RGB domain based on LTMNET network, the method comprising the following steps:

[0010] S1, low contrast image: collect and organize low contrast-high contrast image sample pairs to construct training sets and test sets;

[0011] S2, cropping (512, 512, 3): convert the image data of the sample pair to the size of (512, 512, 3) as the input and output size of network training;

[0012] S3, build an LTMNET network model with adjustable RGB domain contrast enhancement:

[0013] S3.1, optimized LTMNET network;

[0014] S3.2, interpolation post-processing;

[0015] S3.3, calculation of losses;

[0016] S4, parameter update.

[0017] The step S3 further comprises:

[0018] S3: Construct an LTMNET network model with adjustable contrast enhancement in the RGB domain; adjust the model structure by cropping and post-processing the low-contrast-high-contrast dataset to adapt to the adjustable contrast enhancement in the RGB domain; the specific modifications in the LTMNET curve mapping algorithm are as follows: the result after the fifth layer curve mapping is directly downsampled four times to obtain mapping curves of different sizes, and mapping curves of different sizes are simultaneously used as the output of the network; the curve morphology transformation operation is transferred from the inside of the network to the post-processing part to achieve contrast enhancement of curve mappings of different sizes.

[0019] The specific network structure of step S3 is:

[0020] The step S3.1 further comprises:

[0021] ①Adjustable contrast enhancement LTMNET model includes

[0022] One layer of convolution processing maps the input three-channel image to a four-channel space;

[0023] Four-layer feature learning module, performs deep feature extraction and gradually learns the mapping curve;

[0024] A layer of curve transformation pooling convolution layer to map to a 3*256 mapping curve space;

[0025] Four maximum downsampling layers to obtain mapping curves of different sizes;

[0026] ②Adjustable contrast enhancement LTMNET model feature learning module, each layer contains a layer of maximum pooling and a layer of convolution processing; the output size of each layer is (256,256,8), (128,128,16),

[0027] (64,64,32), (32,32,64);

[0028] ③ The fifth layer curve mapping module of the adjustable contrast enhancement LTMNET model maps the above learned features to the RGB three-channel curve mapping space, that is, the (32,32,64) result obtained by the feature learning module is downsampled to (16,16,64) and then mapped to (16,16,768) through a layer of convolution;

[0029] ④ The four maximum downsampling layers of the adjustable contrast enhancement LTMNET model obtain the mapping curves of (8, 8, 768), (4, 4, 768), (2, 2, 768), and (1, 1, 768) respectively;

[0030] That is, step S3.1 corresponds to steps ①, ②, ③, and ④;

[0031] The step S3.2 further comprises:

[0032] ⑤ The post-processing module of the adjustable contrast enhancement LTMNET model divides the input low-contrast image into 16*16, 8*8, 4*4, and 2*2 respectively, and performs interpolation mapping calculation with the corresponding curve to obtain the predicted image;

[0033] That is, step S3.2 corresponds to step ⑤;

[0034] The step S3.3 further comprises:

[0035] ⑥ The 16 post-processing results and input are used to calculate the loss using L1loss and Vggloss as the loss of the network gradient calculation, as shown in the following formula (1):

[0036] Loss(x,y)=1-Vgg(x,y)+L1(x,y) (1)

[0037] Where x represents the predicted output of the network, y represents the high contrast image corresponding to the low contrast image, and Vgg() represents the Vgg loss calculation function;

[0038] That is, step S3.3 corresponds to step ⑥.

[0039] Therefore, the advantages of this application are:

[0040] Deep learning contrast enhancement is performed on low contrast in the RGB domain, and it has good adaptive contrast enhancement for images of different scenes, which improves AIISP performance. At the same time, the intensity of contrast enhancement can be adjusted to achieve better visual effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0042] Figure 1 It is a flow chart of an embodiment of the method of the present application.

[0043] Figure 2 It is a schematic diagram of specific modifications in the LTMNET curve mapping algorithm in the embodiment of the method of the present application. DETAILED DESCRIPTION

[0044] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.

[0045] The embodiment of the present application proposes a method for adjustable contrast enhancement based on LTMNET network, such as Figure 1 As shown, the method comprises the following steps:

[0046] S1, low contrast image: collect and organize low contrast-high contrast image sample pairs to construct training sets and test sets;

[0047] S2, cropping (512, 512, 3): convert the image data of the sample pair to the size of (512, 512, 3) as the input and output size of network training;

[0048] S3, build an LTMNET network model with adjustable RGB domain contrast enhancement:

[0049] S3.1, LTMNET network;

[0050] S3.2, interpolation post-processing;

[0051] S3.3, calculation of losses;

[0052] S4, parameter update.

[0053] Specifically, the steps further include the following:

[0054] Step S1: Collect and organize low-contrast-high-contrast image sample pairs to construct training sets and test sets.

[0055] Step S2: Convert the image data of the sample pair into a size of (512, 512, 3) as the input and output size for network training.

[0056] Step S3: Construct an LTnmET network model with adjustable RGB domain contrast enhancement. By trimming and post-processing the low-contrast-high-contrast dataset, adjust the model structure to adapt to the RGB domain adjustable contrast enhancement. Figure 2 As shown in the figure, the specific modification in the LTMNET curve mapping algorithm is as follows: the result after the fifth layer curve mapping is directly downsampled four times to obtain mapping curves of different sizes, and the mapping curves of different sizes are simultaneously used as the output of the network; the curve morphology transformation operation is transferred from the inside of the network to the post-processing part to achieve contrast enhancement of curve mapping of different sizes. The LTMNET: Local Tone Mapping Network, Adjustable Contrast Enhancement LTMNET is a model proposed in this paper.

[0057] The specific structure of the network is:

[0058] The step S3.1 further comprises:

[0059] ① The adjustable contrast enhancement LTMNET model mainly includes a layer of convolution processing, which maps the input three-channel image processing to a four-channel space; four layers of feature learning modules, which perform deep feature extraction and gradually learn the mapping curve; a layer of curve transformation pooling convolution layer to map to a 3*256 mapping curve space; four layers of maximum downsampling layers to obtain mapping curves of different sizes;

[0060] ②Adjustable contrast enhancement LTMNET model feature learning module, each layer contains a layer of maximum pooling and a layer of convolution processing; the output sizes of each layer are (256, 256, 8), (128, 128, 16), (64, 64, 32), and (32, 32, 64).

[0061] ③ The fifth layer curve mapping module of the adjustable contrast enhancement LTMNET model maps the above learned features to the RGB three-channel curve mapping space, that is, the (32, 32, 64) result obtained by the feature learning module is downsampled to (16, 16, 64) and then mapped to (16, 16, 768) through a layer of convolution.

[0062] ④ The four maximum downsampling layers of the adjustable contrast enhancement LTMNET model obtain the mapping curves of (8, 8, 768), (4, 4, 768), (2, 2, 768), and (1, 1, 768) respectively.

[0063] That is, step S3.1 corresponds to steps ①, ②, ③, and ④.

[0064] The step S3.2 further comprises:

[0065] ⑤ The post-processing module of the adjustable contrast enhancement LTMNET model divides the input low-contrast image into 16*16, 8*8, 4*4, and 2*2, and performs interpolation mapping calculations with the corresponding curves to obtain the predicted image. The 16 post-processing results and the input are used to calculate the loss using L1loss and vggloss as the loss of the network calculation gradient.

[0066] Loss(x,y)=1-Vgg(x,y)+L1(x,y) (6)

[0067] Where x represents the predicted output of the network, y represents the high contrast image corresponding to the low contrast image, and Vgg() represents the Vgg loss calculation function.

[0068] That is, step S3.2 corresponds to step ⑤.

[0069] The step S3.3 further comprises:

[0070] ⑥ The 16*16 post-processing result and input are used to calculate the loss using Llloss and Vggloss as the loss of the network gradient calculation, as shown in the following formula (1):

[0071] Loss(x,y)=1-Vgg(x,y)+L1(x,y) (1)

[0072] Where x represents the predicted output of the network, y represents the high contrast image corresponding to the low contrast image, and Vgg() represents the Vgg loss calculation function;

[0073] That is, step S3.3 corresponds to step ⑥.

[0074] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for contrast enhancement in RGB domain based on LTMNET network, characterized in that: The method comprises the following steps: S1, low contrast image: collect and organize low contrast-high contrast image sample pairs to construct training sets and test sets; S2, cropping (512, 512, 3): convert the image data of the sample pair to the size of (512, 512, 3) as the input and output size of network training; S3, build an LTMNET network model with adjustable RGB domain contrast enhancement: S3.1, optimized LTMNET network; S3.2, interpolation post-processing; S3.3, calculation of losses; S4, parameter update.

2. According to claim 1, a method for contrast enhancement in RGB domain based on LTMNET network is characterized in that: The step S3 further comprises: S3: Construct an LTMNET network model with adjustable contrast enhancement in the RGB domain; adjust the model structure by cropping and post-processing the low-contrast-high-contrast dataset to adapt to the adjustable contrast enhancement in the RGB domain; the specific modifications in the LTMNET curve mapping algorithm are as follows: the result after the fifth layer curve mapping is directly downsampled four times to obtain mapping curves of different sizes, and mapping curves of different sizes are simultaneously used as the output of the network; the curve morphology transformation operation is transferred from the inside of the network to the post-processing part to achieve contrast enhancement of curve mappings of different sizes.

3. The method for adjustable contrast enhancement based on LTMNET network according to claim 2 is characterized in that: The specific network structure of step S3 is as follows: step S3.1 further includes: ①Adjustable contrast enhancement LTMNET model includes One layer of convolution processing maps the input three-channel image to a four-channel space; Four-layer feature learning module, performs deep feature extraction and gradually learns the mapping curve; A layer of curve transformation pooling convolution layer to map to a 3*256 mapping curve space; Four maximum downsampling layers to obtain mapping curves of different sizes; ②Adjustable contrast enhancement LTMNET model feature learning module, each layer contains a layer of maximum pooling and a layer of convolution processing; the output size of each layer is (256,256,8), (128,128,16), (64,64,32), (32,32,64) respectively; ③ The fifth layer curve mapping module of the adjustable contrast enhancement LTMNET model maps the above learned features to the RGB three-channel curve mapping space, that is, the (32,32,64) result obtained by the feature learning module is downsampled to (16,16,64) and then mapped to (16,16,768) through a layer of convolution; ④ The four maximum downsampling layers of the adjustable contrast enhancement LTMNET model obtain the mapping curves of (8, 8, 768), (4, 4, 768), (2, 2, 768), and (1, 1, 768) respectively; That is, step S3.1 corresponds to steps ①, ②, ③, and ④; The step S3.2 further comprises: ⑤ The post-processing module of the adjustable contrast enhancement LTMNET model divides the input low-contrast image into 16*16, 8*8, 4*4, and 2*2 sizes, and performs interpolation mapping calculations with the corresponding curves to obtain the predicted image; That is, step S3.2 corresponds to step ⑤; The step S3.3 further comprises: ⑥ The 16*16 post-processing result and input are used to calculate the loss using L1loss and Vggloss as the loss of the network calculation gradient, as shown in the following formula (1): Loss(x,y)=1-Vgg(x,y)+L1(x,y) (1) Where x represents the predicted output of the network, y represents the high contrast image corresponding to the low contrast image, and Vgg() represents the Vgg loss calculation function; That is, step S3.3 corresponds to step ⑥.

Images