Method and system for enhancing low-light RAW image based on diffusion model, equipment and medium

Abstract

The invention relates to the technical field of computational graphics and computer vision, and discloses a method and system for enhancing a low-light RAW image based on a diffusion model, and the method comprises the steps: collecting paired low-light RAW images and normal light sRGB images in various real static scenes, and synthesizing a low-light image data set through a paired data synthesis strategy; giving a pair of low light RAW image and reference sRGB image as input images; constructing a diffusion model, wherein the diffusion model comprises an input module, a self-adaptive illumination correction module and an output module; training a diffusion model by using a loss function to generate a trained diffusion model; and collecting a low-light RAW image and inputting the low-light RAW image into the trained diffusion model to generate an enhanced image. The system comprises a data synthesis unit, an image input unit, a diffusion model construction unit, a training unit and a generation unit. The invention further discloses electronic equipment and a computer readable storage medium. The present invention enhances a low light RAW image by using a diffusion model and outputs it as an sRGB image.

Description

Technical Field

[0001] The present invention relates to the fields of computational graphics and computer vision technology, and in particular to a method, system, device, and medium for enhancing a low-light RAW image based on a diffusion model. The low-light RAW image is enhanced by using a diffusion model and outputted as an sRGB image. This is a method for synthesizing a paired low-light enhancement data set under conditions close to real dark light. Background Art

[0002] Restoring high-quality clear images from low-light images is a challenging task because it requires improving global and local contrast, suppressing amplified noise, and retaining details to make them more suitable for human visual perception and subsequent visual tasks, which is crucial for application scenarios such as night photography and surveillance security. Traditional methods include histogram equalization and Retinex theory, which can improve image brightness and contrast to a certain extent, but often perform poorly in complex scenes. In recent years, low-light image enhancement methods based on deep learning have made significant progress in brightness enhancement, detail recovery, and noise suppression by virtue of their powerful feature learning capabilities and data-driven advantages.

[0003] Generally, deep learning based methods can be divided into two categories according to the input format: sRGB image based methods and RAW image based methods. Compared with the widely studied sRGB based methods, RAW based methods have attracted increasing attention due to their ability to utilize more informative low-intensity RAW signals.

[0004] At the same time, qualified datasets are essential for advancing deep learning-based low-light RAW enhancement methods. Existing datasets usually collect paired images in a dark environment by changing camera parameters, where a normal-light reference image is obtained by extending the exposure time. However, due to the influence of ambient light sources and brightness fluctuations during data acquisition, these datasets only provide a rough illumination level of the captured low-light RAW images. In addition, in truly extremely dark conditions, such as environments with illumination as low as 0.0001Lux, it is impossible to obtain a well-exposed reference image through long exposure, which results in residual noise and motion blur.

[0005] In this paper, we propose a diffusion model-based framework that leverages the generative power and intrinsic denoising properties of the diffusion model to recover visually pleasing results from RAW image inputs with extremely low signal-to-noise ratios, where an adaptive brightness correction module (AICM) and color consistency loss are introduced to ensure accurate exposure correction and color restoration. On the other hand, we propose a paired data synthesis method that is able to synthesize well-calibrated extremely low-light RAW images in different illumination ranges, and together with high-quality sRGB reference images, form a paired dataset for training and testing low-light RAW image enhancement methods. Experiments show that our proposed diffusion model-based method surpasses existing research, and extensive experiments conducted on our proposed synthetic datasets and public datasets demonstrate the effectiveness of our method. We believe that this work provides new ideas for solving problems in the field of low-light image enhancement.

[0006] The present invention provides a method, system, device, and medium for enhancing low-light RAW images based on a diffusion model. The low-light RAW images are enhanced by using a diffusion model and output as sRGB images. This is a method for synthesizing a paired low-light enhancement data set under conditions close to actual dark light. Summary of the invention

[0007] The present invention provides a method, system, device, and medium for enhancing low-light RAW images based on a diffusion model. The low-light RAW images are enhanced by using a diffusion model and output as sRGB images. This is a method for synthesizing a paired low-light enhancement data set under conditions close to actual dark light.

[0008] The present invention is implemented by the following technical solution: A method for enhancing low-light RAW images based on a diffusion model, comprising the following steps:

[0009] Step S1, collecting pairs of low-light RAW images and normal-light sRGB images in various real static scenes, and synthesizing a low-light image dataset through a paired data synthesis strategy;

[0010] Step S2, a pair of low-light RAW images and a reference sRGB image are given in the data set as input images;

[0011] Step S3, constructing a diffusion model, wherein the diffusion model includes an input module, an adaptive illumination correction module and an output module;

[0012] Step S4, training the diffusion model using a loss function in the data set to generate a trained diffusion model;

[0013] Step S5: collect a low-light RAW image and input it into the trained diffusion model to generate an enhanced image.

[0014] In order to better implement the present invention, further, step S1 includes:

[0015] A professional camera is used to collect paired low-light RAW images and normal-light sRGB images in various scenes by adjusting the shooting parameters. A low-light image dataset is synthesized by processing the illumination information of the low-light images and adding modeled Gaussian and Poisson noise.

[0016] In order to better implement the present invention, further, step S3 includes:

[0017] The input module includes an encoder, and the output module includes a decoder.

[0018] In order to better implement the present invention, further, the encoder converts the input low-light RAW image and normal-light sRGB image into latent space features;

[0019] Then the encoded sRGB image features are input into the diffusion model to perform a forward diffusion process, while the encoded low-light RAW image features are sent to the adaptive illumination correction module to generate contrast-improved features;

[0020] In the back diffusion, the improved low-light RAW image features generated by the adaptive illumination correction module are used as a guide, and the reconstructed normal-light sRGB image features are generated by splicing the noise grayscale features, which are replaced with randomly sampled Gaussian noise during the inference process; finally, the reconstructed normal-light sRGB image features are sent to the decoder to obtain the final normal-light sRGB image restoration result.

[0021] In order to better implement the present invention, further, the structure of the adaptive illumination correction module in step S3 includes: the adaptive illumination correction module includes a convolution layer for embedding input features, four 3*3 cascaded convolution layers, an adaptive average pool, an Araw layer with a magnification factor of 1*1*c obtained after the mean pooling layer, a convolution layer for embedding output features, a feature multiplication layer and a feature addition layer;

[0022] The convolution layer for embedding input features is sequentially connected to a first cascade convolution layer, a feature multiplication layer, a fourth cascade convolution layer, a feature addition layer and a convolution layer for embedding output features; the first cascade convolution layer and the feature multiplication layer are sequentially connected in series to a second cascade convolution layer, a third cascade convolution layer, an adaptive average pool and an Araw layer.

[0023] In order to better implement the present invention, further, step S4 includes:

[0024] First, the encoder, decoder and adaptive illumination correction module are trained in one stage, then the parameters are frozen and the diffusion model is trained in the second stage.

[0025] In order to better realize the present invention, further, it includes:

[0026] The loss function used in the training encoder and decoder is as follows:

[0027]

[0028] Among them, I is the RAW image, sRGB image and the corresponding grayscale image, ε(·) is the encoder, D(·) is the decoder, ||·|| 2 is the l1 norm;

[0029] The loss function used by the adaptive illumination correction module is as follows:

[0030]

[0031] in, is the input feature, is the enlarged feature, L and R are the illumination map and reflection map respectively, ||·|| 1 is the l1 norm.

[0032] The one-stage training loss function is:

[0033]

[0034] The loss function used in the diffusion model is as follows:

[0035] Among them, ||·|| 1 They represent the l1 norm respectively;

[0036] The color consistency constraint loss function is set in the diffusion model, expressed as:

[0037]

[0038] in, and The color histograms of the generated sRGB feature map and the target sRGB feature map are respectively, and τ is 0.001;

[0039] The two-stage training loss function is:

[0040] The present invention also provides a system for enhancing low-light RAW images based on a diffusion model, comprising a data synthesis unit, an image input unit, a diffusion model construction unit, a training unit and a generation unit, wherein:

[0041] It includes a data synthesis unit, which is used to collect pairs of low-light RAW images and normal-light sRGB images in various real static scenes, and synthesize a low-light image dataset through a paired data synthesis strategy;

[0042] An image input unit, configured to give a pair of low-light RAW images and a reference sRGB image in the dataset as input images;

[0043] A diffusion model construction unit, used to construct a diffusion model, wherein the diffusion model includes an input module, an adaptive illumination correction module and an output module;

[0044] A training unit, configured to train the diffusion model using a loss function in the data set to generate a trained diffusion model;

[0045] The generation unit is used to collect a low-light RAW image and input it into the trained diffusion model to generate an enhanced image.

[0046] The present invention also provides an electronic device, which includes a processor and a memory; the processor includes the system for enhancing low-light RAW images based on a diffusion model as described in the second aspect above.

[0047] The present invention also provides a computer-readable storage medium, which includes instructions; when the instructions are executed on the electronic device described in the third aspect, the electronic device executes the method described in the first aspect.

[0048] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0049] (1) The present invention provides a method, system, device, and medium for enhancing low-light RAW images based on a diffusion model, which utilizes the generation capability and intrinsic denoising characteristics of the diffusion model to restore visually pleasing results from RAW image inputs with extremely low signal-to-noise ratios, wherein an adaptive brightness correction module (AICM) and color consistency loss are introduced to ensure accurate exposure correction and color restoration;

[0050] (2) The present invention provides a method, system, device, and medium for enhancing low-light RAW images based on a diffusion model, which can synthesize well-calibrated extremely low-light RAW images under different illumination ranges and form a paired data set together with a high-quality sRGB reference image for training and testing low-light RAW image enhancement methods;

[0051] (3) The present invention provides a method, system, device, and medium for enhancing low-light RAW images based on a diffusion model. Experiments have shown that the method based on the diffusion model proposed by us surpasses existing research, and a large number of experiments conducted on synthetic datasets and public datasets proposed by us have proved the effectiveness of our method. We believe that this work provides new ideas for solving problems in the field of low-light image enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The present invention is further described in conjunction with the following drawings and embodiments, and all the concepts and innovations of the present invention should be regarded as the disclosed contents and the protection scope of the present invention.

[0053] Figure 1 A method for enhancing low-light RAW images based on a diffusion model and a method flow chart in a system, device, and medium provided in an embodiment of the present application;

[0054] Figure 2 A schematic diagram of a data synthesis process in a method, system, device, and medium for enhancing low-light RAW images based on a diffusion model provided in an embodiment of the present application;

[0055] Figure 3 A method and system, device, and medium data synthesis process diagram b for enhancing low-light RAW images based on a diffusion model provided in an embodiment of the present application;

[0056] Figure 4 A method for enhancing low-light RAW images based on a diffusion model, a system, a device, and a diffusion model structure schematic diagram in a medium provided in an embodiment of the present application;

[0057] Figure 5 A structural schematic diagram of an adaptive illumination correction module in a method, system, device, and medium for enhancing low-light RAW images based on a diffusion model provided in an embodiment of the present application;

[0058] Figure 6 A schematic diagram showing the effects of different methods on a synthetic data set on a Canon EOSR in a method, system, device, and medium for enhancing low-light RAW images based on a diffusion model provided in an embodiment of the present application;

[0059] Figure 7 A method, system, device, and medium for enhancing low-light RAW images based on a diffusion model provided in an embodiment of the present application are provided, and a schematic diagram showing the effects of different methods on a synthetic data set on a Sony a7RIII. DETAILED DESCRIPTION

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. It should be understood that the described embodiments are only part of the embodiments of the present invention, not all of the embodiments, and therefore should not be regarded as limiting the scope of protection. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technical personnel in this field without making creative work are within the scope of protection of the present invention.

[0061] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "disposed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0062] Embodiment 1:

[0063] The present embodiment provides a method, system, device, and medium for enhancing low-light RAW images based on a diffusion model. Figure 1 As shown in the figure, in this paper, we first propose a low-light RAW image enhancement framework based on a diffusion model, which exploits the generative power of the diffusion model to achieve visually pleasing sRGB image results. Given a pair of low-light RAW images and a reference sRGB image, we first adopt a RAW encoder and an sRGB encoder to convert the input image into a feature representation in the latent space. Then, we design an adaptive illumination correction module (AICM) to improve the contrast of the encoded RAW features to avoid exposure bias in the subsequent diffusion process, which is different from the previous method that uses the exposure information of the reference image for pre-amplification, aiming to achieve better restoration results and avoid exposure bias in the subsequent diffusion process. Subsequently, the encoded sRGB features are used as the input of the diffusion model and restored under the guidance of the amplified RAW features, where a color consistency loss is further proposed so that the diffusion model generates reconstructed sRGB features with an accurate color map. Finally, the restored features are fed into the sRGB decoder for reconstruction to obtain the final restored image.

[0064] like Figure 2 - Figure 3As shown, we also propose a synthesis method for paired low-light image datasets to generate high-quality images that are close to the distribution of real low-light scenes. In order to generate low-light RAW images with an accurate illumination range, we first use a professional micro-single camera to capture standard illumination low-light RAW images under three accurate illumination ranges (0.01-0.1lux, 0.001-0.01lux, and 0.0001-0.001lux) in an optical laboratory with a controllable light source and a professional illuminance meter. Subsequently, in order to make the synthesized images close to real images, three synthesis steps are performed in sequence: 1) paired low-light RAW images and clear sRGB images are collected in various scenes by adjusting the shooting parameters using the above-mentioned professional camera; 2) the illumination information of the low-light image is processed to produce an image with the same illumination level as the standard data obtained in the optical laboratory; 3) modeled Gaussian and Poisson noise are added to approach the real dark light environment.

[0065] Embodiment 2:

[0066] This embodiment is further optimized on the basis of embodiment 1. Figure 2 - Figure 3 As shown, we first used Sony α7RIII and Canon EOSR to capture standard low-light RAW images with illumination ranges of 0.01-0.1lux, 0.001-0.01lux, and 0.0001-0.001lux in an optical laboratory with a controllable light source and a professional illuminance meter (PHOTO-2000μ). st . Figure 2 (a) shows the optical laboratory scene, the collected standard RAW images, and the camera parameter settings used for image capture. The RAW images are visualized after a simple demosaicing process. Figure 3 (b) shows our proposed paired data synthesis strategy. We collected paired low-light RAW images and normal-light sRGB images in various real static scenes by changing the camera parameters. To ensure the consistency of the content, we fixed the camera on a tripod and controlled the shooting through a remote application to control the exposure setting and capture. Since the lighting information in the RAW image is linearly related to the photon intensity, we use the standard data I st Adjust the exposure of captured low-light images cap Finally, the three illumination ranges are obtained in accordance with the standard data I st Aligned composite image I syn . The standard data I st And the composite image I syn After ISP, the synthetic data and standard data are aligned by calculating the matching degree of the histogram distribution of the Y channel in the YUV space, and finally the correction value η is used to fine-tune the illumination synthesis.

[0067] The other parts of this embodiment are the same as those of Embodiment 1, and thus will not be described in detail.

[0068] Embodiment 3:

[0069] This embodiment is further optimized on the basis of the above-mentioned embodiment 1 or 2. The network is trained on the data set synthesized by the above-mentioned synthesis method. The two data subsets contain a large number of RAW-sRGB image pairs taken and synthesized by two different cameras, which can ensure that the model fully learns the mapping from low-light RAW images to sRGB images.

[0070] The other parts of this embodiment are the same as any of the above embodiments 1-2, so they are not repeated here.

[0071] Embodiment 4:

[0072] This embodiment is further optimized based on any one of the above embodiments 1-3. Figure 4 As shown, in the training phase of the model, the encoder is first used to convert the input noisy low-light RAW image I raw and the reference sRGB image I rgb Convert to latent space feature representation and Then, the encoded sRGB features As the input of the diffusion model to perform the forward diffusion process, in the forward diffusion process, given x 0 Add Gaussian noise related to time t to obtain the noise grayscale feature x at time t t . x0 is actually the same as the previous stage in terms of content. It is just that in the diffusion model, it is customary to use x plus a subscript to represent the characteristic state at a certain moment. At time 0, it means x 0 , no noise is added at this time.

[0073] At the same time, the encoded RAW features is sent to the proposed adaptive illumination correction module (AICM) to generate contrast-improved features, denoted as like Figure 5 As shown in Figure 2, it aims to avoid exposure bias in the diffusion process. In the Adaptive Illumination Correction Module (AICM), we first use a convolutional layer to embed the input features Then, a cascade of convolutional layers and adaptive average pooling layers is used to estimate the per-channel amplification factor A. raw , which is used to improve the contrast of the embedded features. Finally, the refined features are reconstructed using a convolutional layer with residual content to generate the enlarged features Improved RAW features during back diffusion As a guide, the noise grayscale feature x at time t is concatenated with the concatenate module. t Stitched together, from the noise tensor x t Generate reconstructed sRGB features Replace it with randomly sampled Gaussian noise during inference Finally, the reconstructed sRGB features are sent to the sRGB decoder to obtain the final sRGB image restoration result.

[0074] The other parts of this embodiment are the same as any one of the above embodiments 1-3, so they are not repeated here.

[0075] Embodiment 5:

[0076] This embodiment is further optimized on the basis of any one of the above embodiments 1-4. The network adopts a two-stage training method. First, the encoder-decoder architecture and the adaptive illumination correction module are trained. Then, the parameters are frozen and the diffusion model and the color consistency module in the diffusion model are trained. The color consistency module is mainly reflected in the diffusion model network through the loss function, which is equivalent to proposing a new loss function constraint, which is mainly reflected in Figure 4 middle and loss of content.

[0077] The loss function used by the encoder-decoder architecture is as follows:

[0078]

[0079] Among them, I is the RAW image, sRGB image and the corresponding grayscale image, ε(·) is the encoder, D(·) is the decoder, ||·|| 2 is the l1 norm.

[0080] The loss function used by the adaptive illumination correction module is as follows:

[0081]

[0082] in, is the input feature, is the enlarged feature, L and R are the illumination map and reflection map respectively, ||·|| 1 is the l1 norm.

[0083] The loss function of one-stage training is:

[0084]

[0085] The loss function used by the diffusion model is as follows:

[0086]

[0087] where ||·|| 1 They represent the l1 norm respectively.

[0088] The color consistency constraint loss function is set in the diffusion model. The color consistency module needs to satisfy the following constraints:

[0089]

[0090] in and The color histograms of the generated sRGB feature map and the target sRGB feature map are respectively, and τ is 0.001.

[0091] The loss function for the two-stage training is:

[0092] The other parts of this embodiment are the same as any one of the above embodiments 1-4, so they are not repeated here.

[0093] Embodiment 6:

[0094] The present invention also provides a system for enhancing low-light RAW images based on a diffusion model matching the method.

[0095] Figure 6 , Figure 7 The results of different state-of-the-art methods and our model on our synthetic dataset are shown in Figure 1. (a) is the input low-light RAW image, (b)-(f) are the sRGB images predicted by previous state-of-the-art methods, (g) is the effect of our proposed method, and (h) is the reference target sRGB image.

[0096] The present invention also provides an electronic device, which includes a processor and a memory; the processor includes the above-mentioned system for enhancing low-light RAW images based on a diffusion model.

[0097] Embodiment 7:

[0098] The present invention also provides a computer-readable storage medium, which includes instructions; when the instructions are executed on the electronic device described in the above embodiment, the electronic device executes the method described in the above embodiment. Optionally, the computer-readable storage medium may be a memory.

[0099] The processor involved in the embodiments of this application may be a chip. For example, it may be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD), or other integrated chips.

[0100] The memory involved in the embodiments of this application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include but not be limited to these and any other suitable types of memory.

[0101] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0102] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0103] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and modules described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0104] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0105] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one device or distributed on multiple devices. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0106] In addition, each functional module in each embodiment of the present application may be integrated into one device, or each module may exist physically separately, or two or more modules may be integrated into one device.

[0107] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing a computer program instruction on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (Digital Subscriber Line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or may contain one or more servers, data centers and other data storage devices that can be integrated with a medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0108] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for enhancing low-light RAW images based on a diffusion model, characterized in that: The following steps are involved: Step S1, collecting pairs of low-light RAW images and normal-light sRGB images in various real static scenes, and synthesizing a low-light image dataset through a paired data synthesis strategy; Step S2, giving a pair of low-light RAW images and a reference sRGB image in the data set as input images; Step S3, constructing a diffusion model, wherein the diffusion model includes an input module, an adaptive illumination correction module and an output module; Step S4, training the diffusion model using a loss function in the data set to generate a trained diffusion model; Step S5: collect a low-light RAW image and input it into the trained diffusion model to generate an enhanced image.

2. The method for enhancing low-light RAW images based on a diffusion model according to claim 1, characterized in that: The step S1 comprises: A professional camera is used to collect paired low-light RAW images and normal-light sRGB images in various scenes by adjusting the shooting parameters. A low-light image dataset is synthesized by processing the illumination information of the low-light images and adding modeled Gaussian and Poisson noise.

3. The method for enhancing low-light RAW images based on a diffusion model according to claim 1, characterized in that: The step S3 comprises: The input module includes an encoder, and the output module includes a decoder.

4. The method for enhancing low-light RAW images based on a diffusion model according to claim 3, characterized in that: include: The encoder converts the input low-light RAW image and normal-light sRGB image into latent space features; Then the encoded sRGB image features are input into the diffusion model to perform a forward diffusion process, while the encoded low-light RAW image features are sent to the adaptive illumination correction module to generate contrast-improved features; In the back diffusion, the improved low-light RAW image features generated by the adaptive illumination correction module are used as a guide, and the reconstructed normal-light sRGB image features are generated by splicing the noise grayscale features, which are replaced with randomly sampled Gaussian noise during the inference process; finally, the reconstructed normal-light sRGB image features are sent to the decoder to obtain the final normal-light sRGB image restoration result.

5. The method for enhancing low-light RAW images based on a diffusion model according to claim 1, characterized in that: The structure of the adaptive illumination correction module in step S3 includes: The adaptive illumination correction module includes a convolution layer for embedding input features, four 3*3 cascaded convolution layers, an adaptive average pool, an Araw layer with a magnification factor of 1*1*c obtained after the mean pooling layer, a convolution layer for embedding output features, a feature multiplication layer and a feature addition layer; The convolution layer for embedding input features is sequentially connected to a first cascade convolution layer, a feature multiplication layer, a fourth cascade convolution layer, a feature addition layer and a convolution layer for embedding output features; the first cascade convolution layer and the feature multiplication layer are sequentially connected in series to a second cascade convolution layer, a third cascade convolution layer, an adaptive average pool and an Araw layer.

6. The method for enhancing low-light RAW images based on a diffusion model according to claim 1, characterized in that: The step S4 comprises: First, the encoder, decoder and adaptive illumination correction module are trained in one stage, then the parameters are frozen and the diffusion model is trained in the second stage.

7. The method for enhancing low-light RAW images based on a diffusion model according to claim 6, characterized in that: include: The loss function used in the training encoder and decoder is as follows: Where I is the RAW image, sRGB image and the corresponding grayscale image, ε(·) is the encoder, D(·) is the decoder, and ||·||2 is the l1 norm; The loss function used by the adaptive illumination correction module is as follows: in, is the input feature, is the enlarged feature, L and R are the illumination map and reflectance map respectively, and ||·||1 is the l1 norm. The one-stage training loss function is: The loss function used in the diffusion model is as follows: Among them, ||·||1 represents the l1 norm; The color consistency constraint loss function is set in the diffusion model, expressed as: in, and The color histograms of the generated sRGB feature map and the target sRGB feature map are respectively, and τ is 0.001; The two-stage training loss function is:

8. A system for enhancing low-light RAW images based on a diffusion model, characterized in that: It includes a data synthesis unit, an image input unit, a diffusion model building unit, a training unit and a generation unit, wherein: It includes a data synthesis unit, which is used to collect pairs of low-light RAW images and normal-light sRGB images in various real static scenes, and synthesize a low-light image dataset through a paired data synthesis strategy; An image input unit, configured to give a pair of low-light RAW images and a reference sRGB image in the dataset as input images; A diffusion model construction unit, used to construct a diffusion model, wherein the diffusion model includes an input module, an adaptive illumination correction module and an output module; The training unit is used to train the diffusion model using a loss function in the data set to generate a trained diffusion model; the generation unit is used to collect a low-light RAW image and input it into the trained diffusion model to generate an enhanced image.

9. An electronic device, characterized in that: The system comprises a processor and a memory; the processor is used to run the system for enhancing low-light RAW images based on a diffusion model as claimed in claim 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes instructions; when the instructions are executed on the electronic device as claimed in claim 9, the electronic device is caused to execute the method as claimed in any one of claims 1 to 7.

Images