RAW domain low light image enhancement method and system emulating a traditional isp pipeline

By mimicking the traditional ISP pipeline method, black level saturation normalization and denoising decoder processing are performed on low-light image RAW data. Combined with a color correction decoder, the problems of poor noise and color restoration in existing technologies are solved, achieving efficient denoising and color restoration of low-light images and significantly improving the output image quality.

CN116739916BActive Publication Date: 2026-02-27NANKAI UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310517528.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-05-09
Publication Date
2026-02-27
Estimated Expiration
2043-05-09

AI Technical Summary

Technical Problem

Existing low-light image enhancement algorithms have poor denoising performance in both the RGB and RAW domains, especially in extremely dark environments where noise is a significant factor. Furthermore, multi-stage enhancement algorithms fail to effectively handle noise and color restoration, resulting in loss of detail.

Method used

By adopting a method similar to the traditional ISP pipeline, black level saturation normalization is performed on the RAW image data under low light conditions. Denoising is then performed through a RAW data encoder and a denoising decoder, and color space conversion is performed using a color correction decoder. Backpropagation training is then combined with color reconstruction and denoising loss to ensure the accuracy of denoising and color restoration.

Benefits of technology

It achieves efficient noise reduction and color restoration of images in extremely dark environments, and the output RGB images have rich texture details and accurate colors, improving image quality under low light conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116739916B_ABST
    Figure CN116739916B_ABST
Patent Text Reader

Abstract

The application discloses a RAW domain low-light image enhancement method and system imitating a traditional ISP pipeline, and comprises the following steps: acquiring image RAW data under low-light conditions and performing black level saturation standardization on the image RAW data; inputting the data after the standardization processing into a RAW data encoder to obtain a feature set under different scales; inputting the feature set into a network denoising decoder to obtain a denoised feature set under different scales; feeding back the denoised feature set to the RAW data encoder, re-encoding the data after the standardization processing in combination with the feature set, obtaining the feature set, inputting the feature set into a color correction encoder, and calculating an RGB image after low-light enhancement. The method makes the overall structure more conducive to the task from the RAW domain to the RGB domain, thereby improving the level of completing the RAW domain low-light enhancement.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a RAW domain low-light image enhancement method and system imitating a traditional ISP pipeline. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] Low-light images often have low visibility and low signal-to-noise ratio. Not only does this result in the inability to extract effective information therefrom, but it often also has a non-negligible impact on some downstream computer vision tasks. For example, night-time license plate recognition and face recognition do not achieve good results, which is caused by high noise and low visibility in low-light images. Methods of changing camera parameters to enhance do not work in some situations. For example, increasing exposure time causes blurring and ghosting in photos; increasing sensitivity causes obvious noise. Changing the camera hardware requires a high cost. Therefore, a method of low-light image enhancement through software has important research significance.

[0004] With the development of deep learning, low-light image enhancement algorithms based on convolutional neural networks have achieved good results. Existing low-light enhancement algorithms based on deep learning are generally divided into two categories: RGB domain low-light enhancement algorithms and RAW domain low-light enhancement algorithms. Among them, low-light enhancement algorithms based on RGB images cannot achieve good denoising effects, because the noise is destroyed after passing through the traditional ISP (image signal processing) pipeline, greatly increasing the difficulty of denoising. The traditional ISP pipeline usually performs denoising in the RAW domain, and then converts the image RAW data into an RGB image that can be understood by the human eye through a series of transformations such as color correction matrix and gamma correction. These two steps irreversibly affect the noise. Again, in extremely dark environments, the signal-to-noise ratio of the image is extremely low, and the signal is often covered by noise. At this time, the RAW domain denoising of the ISP pipeline not only cannot remove the noise well, but also destroys the original noise distribution. RGB domain low-light enhancement algorithms cannot handle noise well, so this method is not suitable for image enhancement tasks in extremely dark environments.

[0005] Low-light enhancement algorithms based on image RAW data can handle noise well. This is because image RAW data has strong originality and high bit depth. 1) Originality means that the noise distribution in image RAW data is original, simple, and traceable, and such noise is often easier for neural networks to remove. 2) High bit depth means that the data range of image RAW data is large, making it more sensitive to noise. Compared to RGB images, RAW data can more easily distinguish noise and signal in extremely low signal-to-noise ratio situations.

[0006] The existing RAW domain low light enhancement algorithm is generally divided into two categories: single-stage enhancement and multi-stage enhancement. The single-stage enhancement algorithm does not distinguish between the two sub-tasks of denoising and color space conversion, so that the two tasks interfere with each other, and finally cannot achieve good results. The multi-stage enhancement algorithm considers this problem, although it divides the RAW domain enhancement algorithm into two sub-tasks of denoising and enhancement, which seems to be more consistent with the traditional ISP pipeline, but it does not consider the error accumulation between neural networks, resulting in the loss of details in the final result. There is no design of operation module that meets the denoising and color space conversion, resulting in that the noise in part of the scene cannot be effectively removed, and the color cannot be effectively restored; finally, a better result is also not achieved. SUMMARY

[0007] In order to solve the above problems, the present application provides a RAW domain low light image enhancement method and system imitating a traditional ISP pipeline, which can efficiently perform low light enhancement on the input and output an RGB image with rich texture detail information and more accurate color by inputting an image of any size.

[0008] In some embodiments, the following technical solutions are adopted:

[0009] A RAW domain low light image enhancement method imitating a traditional ISP pipeline, comprising:

[0010] obtaining image RAW data under low light conditions , and performing black level saturation normalization on the image RAW data

[0011] inputting the data after the normalization processing into a RAW data encoder to obtain a feature set under different scales , inputting the feature set into a network denoising decoder to obtain a denoised feature set under different scales ;

[0012] feeding back the denoised feature set to the RAW data encoder in combination with the feature set , re-encoding the data after the normalization processing to obtain a feature set ;

[0013] inputting the feature set into a color correction decoder to calculate a low light enhanced RGB image.

[0014] Further, the image RAW data is normalized by black level saturation, specifically:

[0015] normalizing the image RAW data under low light conditions by black level saturation A PixelUnshuffle operation is performed to obtain ;

[0016] Subtracting the corresponding black level value and scaling the theoretical maximum value of the data to 1 to obtain ; ;

[0017] An enhancement coefficient is calculated through the exposure time of the image , and the data after standardization processing is obtained by enhancing .

[0018] In some other embodiments, the following technical solutions are adopted:

[0019] A RAW domain low-light image enhancement system imitating a traditional ISP pipeline, comprising:

[0020] A data acquisition module is configured to acquire image RAW data under low-light conditions , and to perform black level saturation standardization on the image RAW data;

[0021] A denoising module is configured to input the data after standardization processing into a RAW data encoder to obtain a feature set at different scales , and to input the feature set into a network denoising decoder to obtain a denoised feature set at different scales ;

[0022] An encoding module is configured to feed back the denoised feature set to the RAW data encoder in combination with the feature set to re-encode the data after standardization processing to obtain a feature set ;

[0023] A color correction module is configured to input the feature set into a color correction decoder to calculate a low-light enhanced RGB image.

[0024] In some other embodiments, the following technical solutions are adopted:

[0025] A terminal device comprising a processor and a memory, the processor being configured to implement instructions; the memory is configured to store a plurality of instructions, the instructions being adapted to be loaded and executed by the processor to implement the RAW domain low-light image enhancement method imitating the traditional ISP pipeline.

[0026] In some other embodiments, the following technical solutions are adopted:

[0027] A computer readable storage medium, wherein a plurality of instructions are stored, the instructions being adapted to be loaded and executed by the processor of a terminal device to implement the RAW domain low-light image enhancement method imitating the traditional ISP pipeline.

[0028] Compared with the prior art, the present application has the following advantages:

[0029] (1) The present application can guide the denoising by normalizing the black level saturation of the RAW data of the image under low light conditions, thereby improving the model effect. The channel-independent denoising module in the RAW data encoder and the network denoising decoder of the present application is more consistent with the intensity distribution of the RAW domain noise, making the RAW domain denoising more effective.

[0030] (2) The matrix color space transformation module in the color correction decoder of the present application can effectively handle the color space conversion problem, making the color more accurate.

[0031] (3) The present application calculates the color reconstruction loss and the denoising loss according to the denoised image RAW data and the low light enhanced RGB image, respectively, for backpropagation training of the RAW data encoder, the network denoising decoder and the color correction decoder; it can more effectively preserve the details on the basis of completing the denoising.

[0032] (4) The neural network architecture of the present application imitates the traditional image signal processing pipeline, making the overall structure more conducive to the task of RAW domain to RGB domain, thereby improving the level of completing the RAW domain low light enhancement.

[0033] Other features and advantages of the additional aspects of the present application will be partially given in the following description, partially will become apparent from the following description, or will be understood by practicing the present application. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 The RAW domain low light image enhancement method flowchart of the present application embodiment imitates the traditional ISP pipeline;

[0035] Figure 2 The channel-independent denoising module schematic diagram of the present application embodiment;

[0036] Figure 3 The feature fusion module schematic diagram of the present application embodiment;

[0037] Figure 4 The matrix color transformation module schematic diagram of the present application embodiment;

[0038] Figure 5 The four sets of output results comparison schematic diagram of the present application embodiment method and other existing methods. DETAILED DESCRIPTION

[0039] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0040] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0041] Example 1

[0042] In one or more embodiments, a RAW domain low-light image enhancement method that mimics a conventional image signal processing pipeline is disclosed, combining... Figure 1 Specifically, it includes the following process:

[0043] (1) Input RAW image data under low light conditions The black-level saturation normalization is applied to obtain the neural network input. .

[0044] In this embodiment, the process of performing black level saturation normalization is as follows:

[0045] (1.1) RAW image data under low light conditions Perform the PixelUnshuffle operation, which will change the original PixelUnshuffle value. The CCD or CMOS sensor data are arranged according to different color sensors. The format is as follows. This indicates the number of sensors forming a unit array, where each channel contains only one color sensor. Among them, for Bayer data For X-Trans data, .

[0046] (1.2) will Subtract the black level value corresponding to the device. Then scale the theoretical maximum value of the data to 1 to obtain... .

[0047] (1.3) Calculate the enhancement factor using the image exposure time. ,right Enhancement .

[0048] (2) The black level saturation normalization process Input RAW data encoder Obtain feature sets at different scales ,in Indicates the number of network layers.

[0049] feature set Input to network denoising decoder In this process, denoising feature sets at different scales are obtained. Network denoising decoder It will also output the denoised RAW image data. It is used to monitor and evaluate the denoising results.

[0050] Specifically, the implementation method of this step is as follows:

[0051] (2.1) will After RAW data encoder The feature extraction module obtains features representing RAW spatial information. Then it goes through the RAW data encoder The first channel-independent denoising module obtains .

[0052] Among them, RAW data encoder Includes a feature extraction module, Channel-independent noise reduction module Each downsampling layer and A feature fusion module, as an example. The function of the feature fusion module is to connect two features along the channel dimension.

[0053] The feature extraction module consists of two It consists of grouped convolutions and activation functions. The first group of convolutions has 32 output channels. The second group of convolutions also has 32 output channels. The specific structure of the channel-independent denoising module is as follows: Figure 2 This indicates that the kernel of each depthwise convolution layer is... Pointwise convolution kernel is The number of channels in layers 1 through 4 are 32, 64, 128, and 256 respectively, with a stride of 1 for each layer. The convolutional kernels in the downsampling layers are all... The step size is 2, the input channels are 32, 64, and 128 respectively, and the output channels are 64, 128, and 256 respectively.

[0054] (2.2) Let the feature obtained in the previous step be... It is downsampled and then passed through a RAW data encoder. The The feature is obtained by a channel-independent denoising module. , i=1,2,…,L-1.

[0055] (2.3) Repeat (2.2) Each time, they obtained Thus, the feature set is obtained. .

[0056] (2.4) After noise reduction decoder The first channel-independent denoising module obtains .

[0057] Among them, the noise reduction decoder Includes a spatial reconstruction module, Channel-independent noise reduction module It consists of several upsampling layers. A feature fusion module, as an example. The spatial reconstruction module consists of two convolutional layers and activation functions. The first is the convolutional kernel. The first layer has a stride of 1 and 32 channels; the second layer has 32 input channels and 4 or 9 output channels. It consists of pointwise convolutions. The specific structure of the channel-independent denoising module is as follows: Figure 2 This indicates that the kernel of each depthwise convolution layer is... The pointwise convolution kernel is The number of channels in layers 1 through 4 are 32, 64, 128, and 256, respectively, with a stride of 1 for each layer. The upsampling layers are transposed convolutions, with all kernels being... The step size is 2, the input channels are 64, 128, and 256 respectively, and the output channels are 32, 64, and 128 respectively.

[0058] (2.5) Let the feature obtained in the previous step be Upsample it to obtain , i=1,2,…,L.

[0059] (2.6) and The feature fusion module obtains .

[0060] Feature fusion module, such as Figure 3 As shown, The kernel of each depthwise convolution is... The step size is 1, and the number of input channels for layers 1 to 3 are 32, 64, and 128, respectively.

[0061] (2.7) After noise reduction decoder The The feature is obtained by a channel-independent denoising module. .

[0062] (2.8) Repeat (2.5) to (2.7) Each time, they obtained Thus, the feature set is obtained. .

[0063] (2.9) and The summation operation is performed, and the sum is then processed by the denoising decoder. The structural reconstruction module outputs... . Compared with standard results Calculate loss Used for backpropagation training of RAW data encoders and noise reduction decoder .

[0064] (3) Denoising feature set Feedback to the RAW data encoder It uses feature sets The guidance Re-encode to obtain the feature set .

[0065] Specifically, the implementation process of this step is as follows:

[0066] (3.1) will be the features obtained in (2.1) The result obtained in (2.8) Perform feature fusion to obtain It passes through a RAW data encoder. The first channel-independent denoising module obtains .

[0067] (3.2) Let the feature obtained in the previous step be... Downsample it and Perform feature fusion to obtain .after After RAW data encoder The The feature is obtained by a channel-independent denoising module. .

[0068] (3.3) Repeat (3.2) Each time, they obtained Thus, the feature set is obtained. .

[0069] As an example, Feature fusion module, such as Figure 3 As shown, the kernel of each depthwise convolution is... The step size is 1, and the number of input channels for layers 1 to 4 are 32, 64, 128, and 256, respectively.

[0070] (4) Feature set Input to color correction decoder In the middle, color correction decoder Calculate the RGB image after low-light enhancement .

[0071] Specifically, the implementation method of this step is as follows:

[0072] (4.1) After color correction decoder The first color correction module obtained .

[0073] Among them, the color correction decoder Includes a spatial reconstruction module, A matrix-based color space transformation module Each upsampling layer and A feature fusion module, as an example. The spatial reconstruction module consists of two convolutional layers and activation functions. The first is the convolutional kernel. The first layer has a stride of 1 and 32 channels; the second layer has 32 input channels and 4 or 9 output channels. It consists of pointwise convolutions. The specific structure of the matrix-based color space transformation module is as follows: Figure 4 This indicates that the kernel of each depthwise convolution layer is... Pointwise convolution kernel is The number of channels in layers 1 through 4 are 32, 64, 128, and 256, respectively, with a stride of 1 for each layer. The upsampling layers are transposed convolutions, with all kernels being... The step size is 2 for all input channels, 64, 128, and 256 respectively, and 32, 64, and 128 for output channels respectively. The feature fusion module is as follows: Figure 3 As shown. The kernel for each depthwise convolution is... The step size is 1, and the number of input channels for layers 1 to 3 are 32, 64, and 128, respectively.

[0074] (4.2) Let the feature obtained in the previous step be... Upsample it to obtain i = 1, 2, ..., L.

[0075] (4.3) and After color correction decoder characteristics fusion module .

[0076] (4.4) through the color correction decoder of the first color correction module to obtain characteristics .

[0077] (4.5) Repeat 4.2) ~ 4.4) times, respectively, to obtain .

[0078] (4.6) through the structure reconstruction module of the color correction decoder , output . and the standard results loss , used for back propagation training RAW data encoder and color correction decoder .

[0079] RAW data encoder , denoising decoder and color correction decoder are trained by back propagation algorithm. Repeat iteration until the iteration reaches the preset threshold to determine the completion of training.

[0080] In order to verify the effectiveness of the method of the embodiment, the data set used in the embodiment is the Sony data set of SID and the Fuji data set of SID. Among them, the data set used for training of Sony data set has 1865 pairs, the data set used for verification has 234 pairs, and the data set used for testing has 598 pairs. The data set used for training of Fuji data set has 1655 pairs, the data set used for verification has 218 pairs, and the data set used for testing has 524 pairs. The preset threshold of the number of iterations for Sony data set is 932,500 times; the preset threshold of the number of iterations for Fuji data set is 827,500 times.

[0081] The method proposed in the embodiment is compared with five methods, including SID (Learning to see in the dark), EEMEFN (EEMEFN: Low-Light Image Enhancement via Edge-Enhanced Multi-Exposure Fusion Network), LDC (Learning to Restore Low-Light Images via Decomposition-and-Enhancement), RRENet (Towards Low Light Enhancement with RAW Images), and R2R+ISP, wherein the R2R+ISP is from the inspiration in Towards Low Light Enhancement with RAW Images, the image RAW data is denoised through a network, and then an image signal processing pipeline (ISP pipeline) is used to obtain an RGB image. The obtained results are as shown in Figure 5 Figure 5 In the table, Input is input, GT is a data set annotation, and Our is the method of the embodiment, and there are 8 columns. As can be seen from the output results, the image denoised and enhanced by the method is clearer, the edges and texture information of the target are more abundant, and has a better visual effect.

[0082] Embodiment Two

[0083] In one or more embodiments, a RAW domain low-light image enhancement system imitating a traditional ISP pipeline is disclosed, comprising:

[0084] A data acquisition module is configured to acquire image RAW data under low-light conditions , and to normalize the image RAW data according to a black level saturation standard;

[0085] A denoising module is configured to input the data after normalization into a RAW data encoder to obtain a feature set at different scales , and to input the feature set into a network denoising decoder to obtain a denoised feature set at different scales ;

[0086] An encoding module is configured to feed back the denoised feature set to the RAW data encoder in combination with the feature set to re-encode the data after normalization to obtain a feature set ;

[0087] A color correction module is configured to input the feature set ​The input is fed into a color correction decoder, and a low-light enhanced RGB image is calculated.

[0088] It should be noted that the specific implementation of each module has been described in detail in Embodiment One, and will not be described in detail here.

[0089] Embodiment Three

[0090] In one or more embodiments, a terminal device is disclosed, comprising a server, the server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the RAW domain low-light image enhancement method imitating a traditional ISP pipeline in Embodiment One when executing the program. For brevity, this will not be described in detail here.

[0091] It should be understood that in the embodiments, the processor can be a central processing unit (CPU), and the processor can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), ready programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0092] The memory can include read-only memory and random access memory, and provide instructions and data to the processor, and a portion of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0093] In the implementation process, each step of the above method can be completed by integrated logic circuits of hardware in the processor or instructions in the form of software.

[0094] Embodiment Four

[0095] In one or more embodiments, a computer-readable storage medium is disclosed, wherein a plurality of instructions are stored, the instructions being adapted to be loaded and executed by a processor of a terminal device to implement the RAW domain low-light image enhancement method imitating a traditional ISP pipeline in Embodiment One.

[0096] The above describes the specific embodiments of the application in conjunction with the accompanying drawings, but is not a limitation on the scope of protection of the application. Those skilled in the art should understand that various modifications or variations made by those skilled in the art on the basis of the technical solutions of the application without inventive labor are still within the scope of protection of the application.

Claims

1. A method for enhancing low-light images in the RAW domain, mimicking a traditional ISP pipeline, characterized in that: include: Acquiring RAW image data under low-light conditions And perform black level saturation normalization on the RAW image data; The standardized data is input into the RAW data encoder to obtain feature sets at different scales. The feature set is input into the network denoising decoder to obtain denoised feature sets at different scales. The standardized data is input into the RAW data encoder to obtain feature sets at different scales. Specifically: The standardized data is input into the feature extraction module of the RAW data encoder to obtain features representing the RAW spatial information. The first feature is then obtained through the first channel-independent denoising module of the RAW data encoder. ; The i-th feature obtained from the previous step is downsampled, and then processed by the RAW data encoder. The channel-independent denoising module obtains the first... Features Finally, the feature set is obtained. , i = 1, 2, ..., L-1; Denoising feature set Feedback is sent to the RAW data encoder and combined with the feature set. The standardized data is re-encoded to obtain the feature set. ; feature set The image is input into the color correction decoder to calculate the low-light enhanced RGB image, specifically: Re-encoded feature set In The first color correction module after passing through the color correction decoder obtains ; Features obtained from the previous one Upsampling was performed to obtain ; With the re-encoded feature set In The feature fusion module of the color correction decoder obtains ; After the color correction decoder Each color correction module obtains features This process is repeated. Each time, they obtained i = 1, 2, ..., L; The structure reconstruction module of the color correction decoder outputs a low-light enhanced RGB image; the loss is calculated between the image and the standard result and used for backpropagation to train the RAW data encoder and color correction decoder.

2. The RAW domain low-light image enhancement method as described in claim 1, which imitates a traditional ISP pipeline, is characterized in that... Black level saturation normalization is performed on the RAW image data, specifically as follows: RAW image data under low light conditions Perform the PixelUnshuffle operation to obtain ; Will Subtract the corresponding black level value, and scale the theoretical maximum value of the data to 1 to obtain ; The enhancement factor is calculated using the image's exposure time. ,right Enhancement is performed to obtain standardized data.

3. The RAW domain low-light image enhancement method as described in claim 1, which imitates a traditional ISP pipeline, is characterized in that... The feature set is input into the network denoising decoder to obtain denoised feature sets at different scales. Specifically: Features of RAW data encoder output The features are obtained after passing through the first channel-independent denoising module of the denoising decoder. ; Let the feature obtained in the previous step be... Upsample it to obtain ; and The feature fusion module obtains ; After the network noise reduction decoder... The feature is obtained by a channel-independent denoising module. Repeat the process. Each time, they obtained ; obtain feature set ; i = 1, 2, ..., L.

4. The RAW domain low-light image enhancement method as described in claim 3, which imitates a traditional ISP pipeline, is characterized in that... feature Features of RAW spatial information output by the feature extraction module of the RAW data encoder The summation operation is performed, and the sum is passed through the structure reconstruction module of the network denoising decoder to output the denoised image RAW data. This data is compared with the standard result to calculate the loss, which is used for backpropagation to train the RAW data encoder and the network denoising decoder.

5. A method for enhancing low-light images in the RAW domain, mimicking a traditional ISP pipeline, as described in claim 1, characterized in that... Denoising feature set Feedback is sent to the RAW data encoder and combined with the feature set. The standardized data is re-encoded to obtain the feature set. Specifically: The features of the RAW spatial information output by the feature extraction module of the RAW data encoder. With the denoising feature set Perform feature fusion to obtain The first channel-independent denoising module of the RAW data encoder yields the following result: ; Features obtained from the previous one Perform downsampling, then compare with the denoised feature set. Perform feature fusion to obtain ; After the RAW data encoder The feature is obtained by a channel-independent denoising module. Repeat the process. Each time, they obtained ; obtain feature set ; i = 1, 2, ..., L.

6. A RAW domain low-light image enhancement system modeled after a traditional ISP pipeline, characterized in that, include: The data acquisition module is used to acquire RAW image data under low-light conditions. And perform black level saturation normalization on the RAW image data; The denoising module is used to input the standardized data into the RAW data encoder to obtain feature sets at different scales. The feature set is input into the network denoising decoder to obtain denoised feature sets at different scales. The standardized data is input into the RAW data encoder to obtain feature sets at different scales. Specifically: The standardized data is input into the feature extraction module of the RAW data encoder to obtain features representing the RAW spatial information. The first feature is then obtained through the first channel-independent denoising module of the RAW data encoder. ; The i-th feature obtained from the previous step is downsampled, and then processed by the RAW data encoder. The channel-independent denoising module obtains the first... Features Finally, the feature set is obtained. , i = 1, 2, ..., L-1; The encoding module is used to denoise the feature set. Feedback is sent to the RAW data encoder and combined with the feature set. The standardized data is re-encoded to obtain the feature set. ; The color correction module is used to adjust the feature set. The image is input into the color correction decoder to calculate the low-light enhanced RGB image, specifically: Re-encoded feature set In The first color correction module after passing through the color correction decoder obtains ; Features obtained from the previous one Upsampling was performed to obtain ; With the re-encoded feature set In The feature fusion module of the color correction decoder obtains ; After the color correction decoder Each color correction module obtains features This process is repeated. Each time, they obtained i = 1, 2, ..., L; The structure reconstruction module of the color correction decoder outputs a low-light enhanced RGB image; the loss is calculated between the image and the standard result and used for backpropagation to train the RAW data encoder and color correction decoder.

7. A terminal device comprising a processor and a memory, the processor for implementing instructions; the memory for storing multiple instructions, characterized in that, The instructions are adapted to be loaded by a processor and executed as described in any one of claims 1-5, the RAW domain low-light image enhancement method mimicking a conventional ISP pipeline.

8. A computer-readable storage medium storing a plurality of instructions, characterized in that, The instructions are adapted to be loaded by the processor of the terminal device and executed as described in any one of claims 1-5, the RAW domain low-light image enhancement method mimicking a conventional ISP pipeline.

Citation Information

Patent Citations

  • Color space and decoder for video

    CN106031143A

  • Image denoising method, image denoising device, storage medium and electronic equipment

    CN111768351A