Low-Light Enhancement Method and System

Abstract

The present invention provides a low-light image improvement system and method comprising: an image separation unit that separates a low-light image into a color component and an illuminance component using the low-light image as an input value; a noise removal unit that removes noise from the color component separated by the image separation unit; an illuminance improvement unit that improves noise by combining the color component with improved noise from the noise removal unit and the illuminance component separated by the image separation unit; an output unit that outputs a final improved image by combining the illuminance component improved by the illuminance improvement unit and the color component with improved noise from the noise removal unit; and a low-light image improvement module that processes the low-light image improvement.

Description

Technology Field

[0001] The present invention relates to a low-light image improvement system and method, and more specifically, to a system and method for improving the illuminance of a low-light image using an artificial neural network. Background Technology

[0003] Systems that provide convenience to users through object recognition or systems that enhance security using facial recognition are being applied in various fields.

[0004] With On-Device AI technology recently gaining significant attention, unlike existing AI methods utilizing cloud servers, it is now possible to execute tasks directly on local devices and process results even in environments where internet connectivity is unavailable, thereby providing stable and rapid outcomes.

[0005] On-device AI technology improves traffic safety by enabling accurate and rapid object recognition, particularly in the field of autonomous vehicles, while technologies such as Face Connect offer innovative services that allow keyless driving using only facial recognition.

[0006] In addition, it helps with crime prevention and response in the field of night security surveillance.

[0007] However, the accuracy of recognition systems that require cameras like this is greatly affected by the weather or surrounding environment.

[0008] Among these, low light or backlighting is the most challenging environment due to the nature of light-recording cameras, and since facial recognition systems are closely linked to security, a decrease in accuracy has a fatal impact.

[0009] To improve the above problem, a deep learning-based low-light enhancement algorithm is used.

[0010] Existing deep learning-based low-light enhancement algorithms consist of many channels and layers. Consequently, the high computing power required to correct images consumes a significant amount of time, which can lead to difficulties in direct application across various fields. Furthermore, conventional technologies improve illumination by directly processing low-light images without performing noise processing or separate processing of illumination and color information, resulting in low processing quality. Prior art literature

[0011] Korean Registered Patent 10-2368677 (Published Feb. 28, 2022) Korean Registered Patent 10-2611606 (Published Dec. 8, 2023) The problem to be solved

[0012] The present invention provides a low-light image improvement system and method capable of solving the problems of existing algorithms for low-light image improvement.

[0013] Furthermore, the present invention provides a low-light improvement method that processes a low-light image by separating it into color information and illuminance information in order to improve the quality of low-light improvement. Additionally, the present invention provides a neural network structure that performs low-light image processing by configuring a neural network to reduce computational load. means of solving the problem

[0014] The present invention provides a low-light image improvement system comprising: an image separation unit that receives a low-light image as an input value and separates the input low-light image into a color component and an illuminance component for output; a noise removal unit that takes the color component output from the image separation unit as an input, removes noise, and outputs the result; an illuminance improvement unit that improves and outputs the illuminance component of the sum of the illuminance component separated from the image separation unit and the color component output from the noise removal unit; and an output unit that combines the output from the illuminance improvement unit and the output from the noise removal unit to output a final improved image.

[0015] Each of the above image separation unit, the above noise removal unit, and the above illumination improvement unit is configured to include a Context Spatial Block that extracts features by applying spatial weights to the input feature map and combines the features of the overall image.

[0016] The above context spatial block is configured to include a spatial attention part that gives weight to spatial information of the input feature map and a global context block part that gives weight to each pixel by reflecting the overall correlation of the input feature map.

[0017] The noise removal unit and the illumination improvement unit are configured to include a Selective Kernel Feature Fusion unit that combines the output feature map obtained through the context spatial block with the output feature map of the convolution performed in the previous step.

[0018] The image separation unit is composed of an artificial intelligence neural network that machine learns using a data set consisting of a predetermined {low-light image, color component image, illuminance component image} as training data to receive a low-light image as an input value and output the input low-light image by separating it into a color component and an illuminance component; the noise removal unit is composed of an artificial intelligence neural network that machine learns using a data set consisting of a predetermined {color information image, noise-improved color information image} as training data to reduce and output the noise of the color component separated and output by the image separation unit; and the illuminance improvement unit is composed of an artificial intelligence neural network that machine learns using a data set consisting of a predetermined {image, illuminance-improved image} as training data to output an image map that improves illuminance from the illuminance component separated and output by the image separation unit and the color component with reduced noise, which is the output of the noise removal unit.

[0019] The present invention is a method for improving low-light images, wherein

[0020] A low-light image improvement method is used, comprising: an image separation step of receiving a low-light image as input and separating it into a color component and an illuminance component; a noise removal step of performing a noise removal process to improve the noise of the separated color component; an illuminance recovery step of restoring the low illuminance to a normal level using the sum of the separated illuminance component and the improved color component as an input value; and a final improved image output step of outputting a final improved image by combining the illuminance component with restored illuminance and the color component with improved noise.

[0021] The above image separation step uses a low-light image enhancement method that inputs an input image into a DICOMNET, an artificial neural network trained to separate and output a reflexance map having a color component and an illumination map having an illuminance component, and performs an illuminance-color separation procedure to separate the illuminance component and the color component of each pixel of the input image.

[0022] The above illuminance color separation procedure uses a low-light image enhancement method that performs an image information concentration inference step, which performs inference by focusing on key image information using a global context block and a spatial attention unit connected in parallel with it in the above DICOMNET.

[0023] The above noise removal step uses a low-light image enhancement method that performs an image information concentration inference step in which the replicity map separated in the above image separation step is input and inference is performed by focusing on key image information using a global context block and spatial attention connected in parallel therewith, and performs an image information combination step in which the output feature map obtained through the image information concentration inference step and the convolution feature map performed in the previous step are combined and output using selective kernel feature fusion.

[0024] The above illumination recovery step uses a low-light image improvement method that performs an image information concentration inference step in which an illumination map separated in the above image separation step and a reflexance map improved in the above noise removal step are concatenated as input values, and inference is performed by concentrating on key image information using a global context block and spatial attention connected in parallel therewith, and an image information combination step in which an output feature map obtained through the above image information concentration inference step and a convolution feature map performed in the previous step are combined and output using selective kernel feature fusion.

[0025] The above final improved image output step performs a low-light image improvement method by performing an image synthesis procedure that combines the illumination map with restored illuminance and the reflexance map with improved noise into a single image, and outputs the combined single image as the final improved image. Effects of the invention

[0027] The low-light image enhancement system according to the present invention provides a low-light image enhancement system capable of solving the problems of existing algorithms, thereby improving the quality of the low-light enhancement system, and provides a low-light enhancement method that processes low-light images by separating them into color information and illuminance information. Furthermore, the present invention provides a neural network structure that performs low-light image processing by configuring a neural network to reduce computational load.

[0028] Through the use of a neural network that reduces computational load via the present invention, it can be applied in various fields such as autonomous driving and security systems, in addition to the fields of face or object recognition. Brief explanation of the drawing

[0030] The following drawings attached to this specification illustrate preferred embodiments of the present invention and serve to further enhance understanding of the technical concept of the present invention in conjunction with the detailed description thereof; therefore, the present invention should not be interpreted as being limited only to the matters described in such drawings. Figure 1 is a block diagram of the low-light image enhancement system of the present invention. FIG. 2 is a block diagram relating to the structure of the image separation unit of the present invention. FIG. 3 is a block diagram relating to the structure of the noise removal unit of the present invention. FIG. 4 is a block diagram relating to the structure of the illumination improvement part of the present invention. FIG. 5 is a block diagram relating to a context spatial block included in the image separation unit, noise removal unit, and illumination improvement unit of the present invention. FIG. 6 is a block diagram relating to a spatial attention block constituting the context spatial block of the present invention. FIG. 7 is a block diagram relating to a global context block constituting a context spatial block of the present invention. FIG. 8 is a flowchart relating to the low-light image improvement method of the present invention. Specific details for implementing the invention

[0031] A low-light image improvement system (100) and a low-light image improvement method (200) according to the present invention will be described in detail with reference to drawings.

[0032] 1. Low light image improvement system (100) according to the present invention

[0033] The present invention improves a low-light image using an artificial intelligence neural network. It is a system that uses a low-light image as an input value and outputs a final improved image through a low-light image improvement system (100).

[0034] FIG. 1 is a block diagram of a low-light image enhancement system (100).

[0035] The low-light image enhancement system (100) uses a low-light image as an input value.

[0036] The input low-light image is separated into a color component and an illuminance component through the image separation unit (10).

[0037] The separated color component is output with noise improved through the noise removal unit (20), and the noise-improved color component is combined with the illuminance component separated in the image separation unit (10) to become the input data of the illuminance improvement unit (40).

[0038] The above input data is improved in illumination through the illumination improvement unit (40) and combined with the color component with improved noise from above to output a final improved image.

[0039] Each part of the module of the low-light image improvement system (100) of the present invention is configured as follows.

[0040] 1.1. Image separation unit (10)

[0041] As shown in FIG. 1, the image separation unit (10) is composed of a Decom-net, which is an artificial intelligence neural network that machine learns using a data set consisting of a low-light image, a reflection map having color components, and an illumination map having light components as training data.

[0042] DICOMNET is a neural network that receives a low-light image as input and outputs the input image by separating it into a reflexance map containing color components and an illumination map containing illuminance components according to Retinex Theory.

[0043] Existing deep learning-based low-light enhancement algorithms consist of many channels and layers, which increases computing power and therefore requires a relatively long time to correct images.

[0044] Accordingly, the present invention uses Softmax Attention, one of the self-attention techniques, to measure the positional relationships of each layer and represents the importance of each position using normalized weights, thereby employing a network structure with relatively low computational load.

[0045] In order to measure the positional relationship of each layer described above and represent the importance of each position with normalized weights, the AI ​​neural network, DICOMNET, configured in the image separation unit (10) as in FIG. 2, uses a Context Spatial Block (11) (hereinafter referred to as the CSB block).

[0046] The decomnet configured in the image separation unit (10) outputs a result by including the CSB block (11) described above in addition to the structure that outputs a result through the conventionally used 3x3 convolution (12) layer.

[0047] FIG. 5 is a block diagram showing the detailed structure of a CSB block (11). The CSB block (11) is a module that extracts rich information from an input feature map by integrating a spatial attention block (14) and a global context block (15) (hereinafter referred to as the GCB block).

[0048] The operation of the CSB block (11) is described as follows.

[0049] Initial data is processed through two stages of 3x3 convolution (12) layers on the input feature map.

[0050] The processed data passes through an attention mechanism via the spatial attention block (14) and the GCB block (15), respectively, and then the information is comprehensively collected and integrated into a single channel.

[0051] The features obtained in this way pass through a 1x1 convolution (13) layer to generate a final output, and the generated output is combined with the original input data. This utilizes a residual structure, which is a type of structure used in deep learning networks where the original input data is directly added to the output after the input data passes through the network. This structure makes it easier for the network to learn the difference between the function it intends to learn and the input data, and helps the network obtain richer feature representations while preserving the information it has learned.

[0052] FIG. 6 is a block diagram of a spatial attention block (14) configured in a CSB block (11).

[0053] The spatial attention block (14) is an attention mechanism that focuses on the spatial information of the input data.

[0054] The spatial attention block (14) performs Global Average Pooling (14a) and Global Max Pooling (14b) on each pixel of the input feature map processed through two stages of 3x3 convolution (12) layers, and then combines them into one channel.

[0055] The global average pooling (14a) described above calculates the average value for each channel of the input feature map, which helps to concisely summarize the spatial information of the input image or feature map and enables the network to be trained using a small number of parameters.

[0056] Global max pooling (14b) calculates the maximum value for each channel of the input feature map, thereby extracting important characteristics of each channel. It also preserves important information that helps identify the location of objects or patterns in the image, while being relatively less affected by spatial changes in the image.

[0057] After performing the above global average pooling (14a) and global max pooling (14b), the data combined into one channel passes through a 5x5 convolution layer and then applies a sigmoid activation function to generate a spatial attention map.

[0058] Compared to the ReLU activation function, which is commonly used for its high efficiency, the sigmoid activation function involves an exponential function and requires complex calculations. However, the sigmoid activation function has an output range between 0 and 1 and is useful for representing probabilities or percentages. Therefore, it is highly efficient when the output value falls within a desired specific range. For this reason, the sigmoid activation function was applied in the present invention.

[0059] The spatial attention map generated above adjusts the weights for the feature maps of each specific location by multiplying them with the input values ​​that have passed through the 3x3 convolution (12) used as input twice, thereby allowing more focus on the information of the specific location.

[0060] As a result, the spatial attention block (14) allows for effective feature extraction by dynamically highlighting important locations based on the spatial characteristics of the input values.

[0061] FIG. 7 is a block diagram of a GCB block (15). As in FIG. 5, the GCB block (15), which is arranged in parallel with the spatial attention block (14), includes a structure of a Modeling section (15a), a Transform section (15b), and a Fusion section (15c) as a method to reflect the overall correlation of the image.

[0062] In the modeling section (15a), data that has undergone two 3x3 convolutions (12) is used as input, and on one side of the input, the array shape is changed through Reshape to enable the inner product, and on the other side, the channels are transformed through a 1x1 convolution (13) layer and then Reshape, and then a Softmax function is applied to obtain the probability for each class.

[0063] The result obtained from the modeling unit (15a) is used twice in the transform unit (15b) to find dependencies between channels and create a new attention feature map.

[0064] The extracted attention feature map and the input data that has undergone the two 3x3 convolutions (12) described above pass through a fusion (15c) section that applies attention to each pixel location using element-wise addition operations.

[0065] 1.2. Noise removal unit (20)

[0066] As shown in FIG. 1, the reflexance map having color components among the outputs separated through the decomnet configured in the image separation unit (10) is used as an input value for the noise removal unit.

[0067] Low-light images represent the most challenging environments due to the camera's characteristics of recording light, such as low light and backlighting, and high ISO values ​​are used to capture the image.

[0068] ISO amplifies the light signal reaching the camera sensor, and as the signal is amplified, more noise is generated compared to images with lower ISO.

[0069] Accordingly, the noise removal unit (20) is composed of a Denoise-net, which is an artificial intelligence neural network that performs machine learning using a data set consisting of a color information image and a color information image with improved noise as training data, in order to improve and output the noise of a reflexance map having separated color components.

[0070] FIG. 3 is a block diagram of a denoisenet artificial intelligence neural network configured in a noise removal unit (20).

[0071] As mentioned above, the noise removal unit (20) includes a CSB block (11) to indicate the importance of each position with normalized weights after measuring the positional relationship of each layer, and additionally includes a Selective Kernel Feature Fusion block (16) (hereinafter referred to as the SKFF block).

[0072] The noise removal unit (20) is formed with a U-Net structure, and the SKFF block (16) described above is used in the decoder part, and the CSB block (11) is used in the encoder part.

[0073] The input of the SKFF block (16) includes the output feature map from the encoder in which the CSB block (11) is used and the output feature map that has undergone convolution in the previous step, so it can achieve more effective performance than the existing neural network structure.

[0074] A reflexance map having a color component with improved noise is output through the noise removal unit (20).

[0075] 1.3. First composite part (30)

[0076] As shown in FIG. 1, the reflexance map output through the noise removal unit (20) has a color component with improved noise. An illumination map having the improved color component and the illuminance component separated by the image separation unit (10) becomes the input value of the first synthesis unit (30).

[0077] The data synthesized and output through the first synthesis unit (30) becomes the input value of the illumination improvement unit (40).

[0078] In order to improve the illuminance component, instead of just extracting and improving the features of the illumination map having the illuminance component, a reflexance map having the color component is synthesized into the illumination map to extract and combine more features, and the synthesis process is performed through the first synthesis unit (30).

[0079] The first synthesis unit (30) may be composed of a concatenation module that synthesizes each image.

[0080] 1.4. Illumination improvement section (40)

[0081] As shown in FIG. 1, the illuminance improvement unit (40) uses the output value obtained from the first synthesis unit (30) described above as an input value to restore the illuminance that was previously generated at a low level to a normal level.

[0082] The illumination improvement unit (40) is composed of RelightNet, an artificial intelligence neural network that machine learns using a dataset consisting of an image and an image with improved illumination as training data to output an image map with improved illumination. At this time, the image with improved illumination, which is the true value in the training data, is composed of an image that has only grayscale. Accordingly, the illumination improvement unit (40) of the present invention is trained to output an input image as an image with improved illumination.

[0083] The output is structured to have only a grey scale, not RGB color.

[0084] The structure of the artificial intelligence neural network RelightNet also includes a CSB block (11) and an SKFF block (16), similar to the artificial intelligence neural network DeNoiseNet of the noise removal unit (20) described above.

[0085] An image synthesized through the first synthesis unit (30) is input into the illumination improvement unit (40), and an illumination map with improved illumination is output through the RelightNet artificial intelligence neural network.

[0086] 1.5. Second composite part (50)

[0087] As shown in FIG. 1, the process involves combining an output value containing a grayscale with improved illumination through an illumination improvement unit (40) and a reflexance map having a color component with improved noise through a noise removal unit (20).

[0088] According to the Retinex theory used when separating the low-light image described in the image separation unit (10), the second synthesis unit (50) multiplies the reflex tens map having a color component with improved noise and the improved illumination map having an illuminance component.

[0089] The low-light image finally input through the corresponding product outputs a final improved image through the low-light image improvement system (100) according to the present invention. This can be implemented as a concatenation module that synthesizes two images, similar to the first synthesis unit described above.

[0090] 2. Low light image improvement method (200) according to the present invention

[0091] The present invention improves low-light images using an artificial intelligence neural network. In the present invention, a low-light image improvement method (200) is described in detail step by step as follows with reference to FIG. 8, using a low-light image as an input value and outputting a final improved image through a low-light image improvement system (100).

[0092] 2.1. Image separation step (S10)

[0093] A low-light image is input into DICOMNET, an artificial neural network trained to separate and output a low-light image into a reflexance map containing color components and an illumination map containing illuminance components.

[0094] The input image separates the illuminance and color components of each pixel according to Retinex theory.

[0095] The Retinex theory used in the image separation step (S10) is <Equation 1>.

[0096] <Formula 1>

[0097]

[0098] is a value representing the amount of light reflected at a specific pixel location in a low-light image, which becomes the value of a reflexance map having a color component, and becomes the brightness or color value at a specific pixel location.

[0099] represents the lighting intensity in a low-light image and is a value indicating the amount of light entering the corresponding pixel location from the surrounding environment, becoming the value of an illumination map with an illuminance component. It is a value representing the lighting conditions at a specific pixel location in the input image.

[0100] Therefore, the value corresponding to the reflexance map containing color components and the value corresponding to the illumination map having illuminance components The product of corresponds to the low-light image It can be expressed as a value.

[0101] The Retinex theory is used in the same way when obtaining the final improved image as described in 1.5. Second Synthesis Section (50).

[0102] The decomnet that separates images according to the Retinex theory uses a CSB block (11) in addition to the conventionally designed 3x3 convolution (12) layer.

[0103] The CSB block (11) performs inference by focusing on key image information using the spatial attention block (14) and the GCB block (15) connected in parallel thereto.

[0104] 2.2. Noise removal step (S20)

[0105] The reflexance map having color components separated through the above image separation step (S10) has noise removed through the noise removal step (S20).

[0106] A reflexance map having color components is used as input, and a CSB block (11) composed of a spatial attention block (14) and a GCB block (15) connected in parallel with it is used to effectively extract various information by considering both spatial attention and global context for the input.

[0107] Using the SKFF block (16), the output feature map obtained through the CSB block (11) and the convolution feature map performed in the previous step are combined and output.

[0108] 2.3. Illumination Restoration Step (S30)

[0109] The illumination recovery step (S30) takes as input the sum of the illumination map having the illumination component separated through the image separation step (S10) above and the reflexance map having the color component improved in the noise removal step (S20) above.

[0110] The above combined input values ​​are effectively extracted by considering spatial attention and global context together with the input using a CSB block (11) composed of a spatial attention block (14) and a GCB block (15) connected in parallel therewith, and

[0111] An illuminance recovery step (S30) is performed by combining the output feature map obtained through the CSB block (11) using the SKFF block (16) with the convolution feature map performed in the previous step.

[0112] The output of the illumination recovery step (S30) is structured to have only a grey scale, not RGB color.

[0113] 2.4. Final improved image output step (S40)

[0114] The process involves combining an output value containing a grayscale with improved illumination through an illumination recovery step (S30) and a reflexance map having color components with improved noise through a noise removal step (S20).

[0115] Combine using the Retinex theory described in the image separation step (S10) above.

[0116] <Formula 1>

[0117]

[0118] represents the value of a reflexance map having a color component with improved noise through a noise removal step (S20), and

[0119] represents the value of an illumination map including a grayscale with improved illumination through the illumination recovery step (S30).

[0120] Therefore, the value corresponding to the reflexance map having color components with improved noise. and a value corresponding to an illumination map having an illuminance component including a grayscale with improved illuminance The product of is the final improved image It can be expressed as a value. As a result, the final improved image is output through the low-light image improvement method (200).

[0121] The image processing results according to the low-light image improvement system and method of the present invention described above are described through Tables 1 and 2 below.

[0122] FLOPs Parameter PSNR Comparative Example: RM (16-channel) 188.35 16.04 17.6131 Example: CSB + SKFF 39.56 14.82 18.1001

[0123] Table 1 compares FLOPs and Parameters for the conventional technology and the present invention, where RM(16channel) refers to a Residual module composed of 16 channels used in the past, and CSB + SKFF refers to the addition of the CSB block, which is the core of the present invention.

[0124] The ptflops used in the above comparison is a Python-based open source library and is a metric indicating how many petaflops of operations can be performed per second.

[0125] The image used for the measurement was based on the LOL dataset (Low Light Image Quality Benchmark and Challenge), with dimensions of 3x600x400, where 3 represents the channels with RGB values, 600 represents the height, and 400 represents the width.

[0126] The PSNR (Peak Signal to noise ratio) in Table 1 is a quantitative indicator used for image comparison, representing the maximum signal-to-noise ratio.

[0127] The datasets used for training were the LOL dataset and the eval data from LSRW (Low Light Scene Recognition with Warmth). Using eval data refers to data used to evaluate the performance of the model after training.

[0128] Based on the above, as shown in Table 1, it can be confirmed that the amount of operations (FLOPs) of the present invention has been significantly reduced compared to conventional technology, and the signal-to-noise ratio has been improved with fewer parameters.

[0129] Model name PSNR SSIM FLOPs (G) Test1 Test2 Test3 AVG. Input - - - 79.3% 65.4% 70.7% 71.8% R2R-net [4] 18.2183 0.5936 NA 88.8% 84.3% 82.5% 85.2% Retinex-net [1] 17.2483 0.4917 129.93 83.5% 72.1% 66.6% 74.1% Zero-DCE [5] 16.6285 0.4733 19.15 93.0% 86.2% 77.3% 85.5% RUAS [2] 14.5173 0.4162 0.87 92.6% 79.7% 77.1% 83.2% MIRNet_v2 [6] 19.52 0.6158 515.42 94.5% 89.0% 83.0% 88.8% DCC-Net [3] 19.0255 0.6145 271.8 93.6% 87.9% 80.7% 87.4% A4R-Net (the present invention) 18.2968 0.5944 39.56 93.3% 85.7% 83.4% 87.5%

[0130] Table 2 is a table comparing the results of conventionally proposed low-illumination improvement methods with the present invention. Bold text from the second column to the last column of Table 2 indicates the highest performance value, and NA (Not available) indicates that measurement is not possible.

[0131] The descriptions for each list in Table 2 are as follows.

[0132] The Model name in column 1 is the name of the method used in each paper described above, and the method of 6 papers from [1] to [6] was used, and the method in the last row refers to the present invention.

[0133] The PSNR (Peak Signal to Noise Ratio) in column 2 is a quantitative indicator used in image comparison, which relates to the maximum signal-to-noise ratio.

[0134] The 3rd column SSIM (Structural Similarity Index Measure) is used as an indicator to evaluate differences in human visual image quality.

[0135] FLOPs (G) in column 4 represents the amount of computation for processing one image.

[0136] Each of Test1, Test2, and Test3 in columns 5 through 7 assumes three situations to compare the values ​​of the present invention with those of conventional methods. Test1 assumes a case where shadows are cast on the face due to backlighting; Test2 assumes a case where there is little ambient light indoors, such as in a parking lot; and Test3 assumes a case where there is little ambient light outdoors in the evening. The % results in the table represent the results of face recognition using YOLOv4, which is described below. "input" in the first row refers to the original image. According to this, it can be seen that the present invention has significantly reduced the amount of computation compared to a high recognition rate [6].

[0137] The test mentioned above shows the probability of detecting a face in an enhanced image using the face recognition software YOLOv4 (You Only Look Once version 4).

[0138] As shown in Table 2, the present invention shows values ​​similar to conventional methods in terms of PSNR and SISIM indices, but can be seen to be significantly improved compared to conventional methods [6] in terms of FLOPs (G) for computational amount.

[0139] The conventional methods mentioned above are the methods proposed in the following papers.

[0140] [1] Wei, C.; Wang, W.; Yang, W.; Liu, J. Deep Retinex Decomposition for Low-Light Enhancement 2018.

[0141] [2] Liu, R.; Ma, L.; Zhang, J.; Fan, X.; Luo, Z. Retinex-Inspired Unrolling with Cooperative Prior Architecture Search for Low-Light Image Enhancement. In Proceedings of the 2021 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR); IEEE: Nashville, TN, USA, June 2021; pp. 10556-10565.

[0142] [3] Zhang, Z.; Zheng, H.; Hong, R.; Xu, M.; Yan, S.; Wang, M. Deep Color Consistent Network for Low-Light Image Enhancement.; 2022; pp. 1899-1908.

[0143] [4] Hai, J.; Xuan, Z.; Yang, R.; Hao, Y.; Zou, F.; Lin, F.; Han, S. R2RNet: Low-Light Image Enhancement via Real-Low to Real-Normal Network. Journal of Visual Communication and Image Representation 2023, 90, 103712, doi:10.1016 / j.jvcir.2022.103712.

[0144] [5] Li, C.; Guo, C.; Chen, CL Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation. IEEE Trans. Pattern Anal. Mach. Intel. 2021, 1-1, doi:10.1109 / TPAMI.2021.3063604.

[0145] [6] Zamir, SW; Arora, A.; Khan, S.; Hayat, M.; Khan, F.S.; Yang, M.-H.; Shao, L. Learning Enriched Features for Fast Image Restoration and Enhancement 2022.

[0146] The above detailed description is illustrative of the present invention. Furthermore, the foregoing describes preferred embodiments of the present invention, and the present invention may be used in various other combinations, modifications, and environments. That is, modifications or alterations are possible within the scope of the concept of the invention disclosed herein, the scope equivalent to the disclosed content, and / or the scope of the art or knowledge. The described embodiments describe the best state for implementing the technical concept of the present invention, and various modifications required for specific fields of application and uses of the present invention are possible. Accordingly, the above detailed description of the invention is not intended to limit the present invention to the disclosed embodiments. Additionally, the appended claims should be interpreted as including other embodiments. Explanation of the symbols

[0147] 10 : Image separation section 11: Context Spatial Block (CSB Block) 12 : 3x3 convolution 13 : 1x1 convolution 14: Spatial Attention Block 14a : Global Average Pooling 14b : Global Max Pooling 15 : Global Context Block (GCB Block) 15a : Modeling 15b : Transform section ( Transform) 15c : Fusion Section (Fusion) 16: Selective Kernel Feature Fusion Block (SKFF Block) 20 : Noise Removal Section 30: 1st Synthetic Part 40 : Illumination improvement section 50: 2nd Synthetic Part 100: Low-light image enhancement system 200: Low-light image improvement method S10: Image separation step S20: Noise removal step S30: Illumination recovery step S40: Final improved image output stage

Claims

Claim 1 A system for improving low-light images comprises: a Decom-Net unit that receives a low-light image as an input value and separates the input low-light image into a Reflectance Map and an Illumination Map for output; a Denoise-Net unit that takes the Reflectance Map output from the Decom-Net unit as an input and outputs the result of prioritizing the reduction of noise in the Reflectance Map, which is in a state prior to the improvement of the illumination component; and a system that receives as an input value a concatenation of the Illumination Map output from the Decom-Net unit and the Reflectance Map from which noise has been preemptively removed through the Denoise-Net unit, and outputs the result of restoring the illumination component of the Illumination Map to a normal level by referring to the feature information of the Reflectance Map from which noise has been removed. A low-light image enhancement system comprising: a Relight-Net unit; and an output unit that outputs a final enhanced image by combining an illumination map in which the illuminance has been restored in the Relight-Net unit and a reflexance map in which the noise has been improved in the Denoise-Net unit. Claim 2 A low-light image enhancement system according to claim 1, wherein each of the above-mentioned Decom-Net unit, above-mentioned Denoise-Net unit, and above-mentioned Relight-Net unit comprises a Context Spatial Block unit that extracts features by applying spatial weights to an input feature map and extracts and combines features of the overall image. Claim 3 In claim 2, the Context Spatial Block unit is configured to integrate each feature information by connecting in parallel a Spatial Attention unit that weights spatial information of an input feature map and a Global Context Block unit that weights each pixel by reflecting the overall correlation of the input feature map. Claim 4 A low-light image enhancement system according to claim 2, comprising: a Denoise-Net unit and a Relight-Net unit, and a Selective Kernel Feature Fusion unit that combines and outputs an output feature map obtained through the context spatial block unit and an output feature map of a convolution performed in a previous step. Claim 5 In claim 1, the above-mentioned DecomNet unit is composed of an artificial intelligence neural network machine-learned using a predetermined dataset consisting of {low-light image, reflectance map, illumination map} as training data to receive a low-light image as an input value and output the input low-light image by separating it into a reflectance map and an illumination map; the above-mentioned Denoise-Net unit is composed of an artificial intelligence neural network machine-learned using a predetermined dataset consisting of {color information image, color information image with improved noise} as training data to output a reflection map separated and output by the above-mentioned DecomNet unit with reduced noise; and the above-mentioned RelightNet unit is composed of an artificial intelligence neural network machine-learned using a predetermined dataset consisting of {image, image with improved illumination} as training data to output an image map with improved illumination from the illumination map separated and output by the above-mentioned DecomNet unit and the reflection map with reduced noise, which is the output of the above-mentioned Denoise-Net unit. Low-light image enhancement system. Claim 6 A method for improving a low-light image, comprising: an image separation step in which a low-light image is input and separated into a reflection map and an illumination map; a noise removal step in which noise in the separated reflection map is preemptively removed prior to the illumination recovery step; an illumination recovery step in which the low illumination is restored to a normal level using the combination of the separated illumination map and the reflection map with noise improved through the noise removal step as an input value; and a final improved image output step in which the illumination map with restored illumination and the reflection map with improved noise are combined to output a final improved image. Claim 7 A low-illumination image improvement method according to claim 6, wherein the image separation step comprises inputting an input image into a DICOMNET, which is an artificial neural network trained to separate and output a reflexance map having a color component and an illumination map having an illuminance component, and performing an illuminance-color separation procedure to separate the illuminance component and the color component of each pixel of the input image. Claim 8 In claim 7, the illuminance color separation procedure comprises an image information concentration inference step in which inference is performed by focusing on key image information using a global context block and a spatial attention unit connected in parallel with it in the dicomnet; a low-light image improvement method. Claim 9 A low-light image enhancement method according to claim 6, wherein the noise removal step performs an image information concentration inference step in which the replicity map separated in the image separation step is input and inference is performed by focusing on key image information using a global context block and spatial attention connected in parallel therewith, and an image information combination step in which the output feature map obtained through the image information concentration inference step and the convolution feature map performed in the previous step are combined and output using selective kernel feature fusion. Claim 10 A low-light image enhancement method according to claim 6, wherein the illumination recovery step comprises an image information concentration inference step in which the illumination map separated in the image separation step and the reflexance map improved in the noise removal step are combined as input values, and inference is performed by focusing on key image information using a global context block and spatial attention connected in parallel therewith, and an image information combination step in which the output feature map obtained through the image information concentration inference step and the convolution feature map performed in the previous step are combined and output using selective kernel feature fusion. Claim 11 In claim 6, the final improved image output step comprises an image synthesis procedure that combines the illumination map with restored illuminance and the reflexance map with improved noise into a single image, and outputs the combined single image as the final improved image; a low-light image improvement method.

Images