Raw domain low-illumination image enhancement method and system

Abstract

The application provides a RAW domain low-illumination image enhancement method and system, relates to the technical field of image processing, and can effectively avoid amplification of noise in the subsequent enhancement process, reduce noise and color distortion of the RAW domain enhanced image by separating the content signal graph and the noise residual graph of the RAW domain low-illumination image. Through the content signal graph, the light rays of the camera light path are simulated and light integration is carried out, so that the physical level detail reconstruction can be carried out on the region with serious structure loss in the RAW domain low-illumination image, and the global light consistency and edge sharpness better than the traditional two-dimensional convolution network are realized. By introducing the target double-flow recurrent neural network, the optimization conflict between denoising and color restoration in the low-illumination enhancement can be solved, the progressive collaborative improvement of the image quality can be realized, and the high harmony of the final enhancement result in the three dimensions of brightness, detail and color is ensured.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for enhancing low-light images in the RAW domain. Background Technology

[0002] In machine vision applications such as defect detection in intelligent manufacturing production lines, monitoring of automated warehousing equipment, and nighttime security, the digital images acquired by image sensors are often subject to problems such as extremely low signal-to-noise ratio, poor contrast, and severe loss of detail due to the limited exposure time caused by low-light environments or high-speed motion. These images belong to the RAW domain and are low-light images.

[0003] Existing RAW domain low-light image enhancement techniques have the following significant drawbacks: (1) Irreversible information loss in traditional image signal processing (ISP): Traditional ISPs typically process sRGB images obtained after processing by the camera's built-in ISP module in the sRGB color space. Because the camera's built-in ISP module performs non-linear compression operations such as white balance and gamma correction on low-light RAW images, the dynamic range of the resulting sRGB images is limited. Under extreme low-light conditions, forcibly increasing the brightness of sRGB images will lead to severe noise amplification and color distortion.

[0004] (2) Domain blurring effect caused by heterogeneous task coupling: Existing end-to-end deep learning RAW domain low-light image enhancement methods usually forcibly couple two heterogeneous tasks, noise removal and color space conversion, requiring the neural network to directly learn the mapping from the noisy RAW domain to the clean sRGB domain. This cross-domain black box mapping is prone to causing obvious domain blurring effect and systematic color shift.

[0005] (3) Optimization conflict between structure and color restoration: Existing RAW domain low-light image enhancement methods often enhance image brightness and detail separately, and then simply stitch and fuse the enhanced brightness features and detail features in the final stage, lacking interpretable modeling of the deep coupling mechanism between the two. Under extreme low light conditions, there is a strong competition between noise reduction and smoothing and color and texture preservation, which often leads to enhancement results that show increased brightness but grayish colors or vibrant colors but severe edge artifacts. Summary of the Invention

[0006] This invention provides a method and system for enhancing low-light images in the RAW domain, in order to overcome the deficiencies existing in related technologies.

[0007] This invention provides a method for enhancing low-light images in the RAW domain, comprising: Acquire a low-light image in the RAW domain and extract the content signal map of the low-light image in the RAW domain; Based on the content signal map, simulate the camera's optical path to project light rays and perform light integration to determine the initial illuminance enhancement feature map; Based on the target channel attention module and the target space attention module, the color channel features of the RAW domain low-light image are adaptively modulated to obtain an initial color recovery feature map. Based on the target dual-stream recurrent neural network, the initial illuminance enhancement feature map and the initial color restoration feature map are iteratively fused and updated to obtain the target illuminance enhancement feature map and the target color restoration feature map. Based on the target illuminance enhancement feature map and the target color restoration feature map, the RAW domain enhanced image is determined.

[0008] According to the present invention, a method for enhancing low-light images in the RAW domain includes extracting the content signal map of the low-light image in the RAW domain, comprising: The RAW domain low-light image is input into a two-branch decoupled neural network to obtain the content signal map output by the content branch network and the noise residual map output by the noise branch network in the two-branch decoupled neural network. The dual-branch decoupled neural network is obtained by supervising training based on normal illumination denoised images of RAW domain low-light image samples and corresponding RAW domain normal illumination image samples, combined with content reconstruction constraints, noise statistical consistency constraints and total variation regularization constraints.

[0009] According to the present invention, a method for enhancing low-light images in the RAW domain includes determining an initial illumination enhancement feature map by simulating the projection of light rays from a camera optical path and performing ray integration based on the content signal map. Based on the content signal map, the camera optical path projection light is simulated to obtain the light line direction vector, and based on the gradient information of the content signal map, the camera optical path projection light is sampled in layers to obtain each sampling point on the camera optical path projection light. The position information of each sampling point on the camera's optical path projection ray and the ray's line-of-sight vector are input into the target neural radiation field model. The target neural radiation field model performs ray integration to obtain the initial illumination enhancement feature map output by the target neural radiation field model.

[0010] According to the RAW domain low-light image enhancement method provided by the present invention, the method involves iteratively fusing and updating the initial illumination enhancement feature map and the initial color restoration feature map based on a target dual-stream recurrent neural network to obtain a target illumination enhancement feature map and a target color restoration feature map, including: For the current iteration, the previous illuminance enhancement feature map and the previous color restoration feature map from the previous iteration are respectively input into the illuminance enhancement branch and the color restoration branch of the target dual-stream recurrent neural network; The illuminance enhancement branch is used to fuse the previous illuminance enhancement feature map with the previous color hidden state feature of the previous iteration to obtain the intermediate illuminance feature, and to enhance the intermediate illuminance feature and restore the structural details to obtain the current illuminance enhancement feature map. Based on the current illuminance enhancement feature map, the current illuminance hidden state feature is determined. The color recovery branch is used to fuse the previous color recovery feature map with the previous illuminance latent state feature of the previous iteration to obtain intermediate color features, and to perform color correction and color recovery on the intermediate color features to obtain the current color recovery feature map. Based on the current color recovery feature map, the current color latent state feature is determined. If the current illuminance enhancement feature map, the current color restoration feature map, the current illuminance latent state feature map, and the current color latent state feature map all converge, or reach a preset iteration round, then the current illuminance enhancement feature map and the current color restoration feature map are respectively used as the target illuminance enhancement feature map and the target color restoration feature map; otherwise, the next iteration round is used as the current iteration round, and the above process is iterated.

[0011] According to the RAW domain low-light image enhancement method provided by the present invention, the target dual-stream recurrent neural network is trained based on the following steps: Determine the low-light denoised images of the RAW domain normal illumination image samples and the corresponding RAW domain low illumination image samples; Based on the content signal map of the low-light denoised image, the initial neural radiation field model is applied to obtain the first illumination enhancement feature map. Based on the initial channel attention module and the initial spatial attention module, the color channel features of the RAW domain low-light image sample are adaptively modulated to obtain the first color restoration feature map. The first illuminance enhancement feature map and the first color restoration feature map are input into the initial two-stream recurrent neural network to obtain the predicted illuminance enhancement feature map and the predicted color restoration feature map output by the initial two-stream recurrent neural network. Based on the predicted illuminance enhancement feature map and the predicted color restoration feature map, the predicted RAW domain enhanced image is determined. Based on the predicted RAW domain enhanced image and the RAW domain normal illumination image sample, the initial neural radiation field model, the initial channel attention module, the initial spatial attention module and the initial dual-stream recurrent neural network are iteratively trained to obtain the target neural radiation field model, the target channel attention module, the target spatial attention module and the target dual-stream recurrent neural network.

[0012] According to the present invention, a method for enhancing low-light images in the RAW domain includes adaptively modulating the color channel features of the low-light image in the RAW domain based on a target channel attention module and a target spatial attention module to obtain an initial color restoration feature map, comprising: The color channel features of the RAW domain low-light image are subjected to a two-dimensional discrete Fourier transform to obtain the frequency domain features. Based on the target channel attention module and the target space attention module, the frequency domain features are adaptively modulated in different frequency sub-bands in the frequency domain to obtain modulated frequency domain features. The modulation frequency domain features are subjected to inverse Fourier transform to obtain the initial color recovery feature map.

[0013] According to a method for enhancing low-light images in the RAW domain provided by the present invention, the step of determining the enhanced RAW domain image based on the target illumination enhancement feature map and the target color restoration feature map includes: The RAW domain enhanced image is subjected to nonlinear processing to obtain the target RGB enhanced image.

[0014] The present invention also provides a RAW domain low-light image enhancement system, comprising: The image acquisition module is used to acquire a low-light image in the RAW domain and extract the content signal map of the low-light image in the RAW domain; The illuminance enhancement module is used to simulate the projection of light rays from the camera's optical path and perform light ray integration based on the content signal map to determine the initial illuminance enhancement feature map; The color restoration module is used to adaptively modulate the color channel features of the RAW domain low-light image based on the target channel attention module and the target space attention module to obtain an initial color restoration feature map. The fusion update module is used to iteratively fuse and update the initial illuminance enhancement feature map and the initial color restoration feature map based on the target dual-stream recurrent neural network to obtain the target illuminance enhancement feature map and the target color restoration feature map, and to determine the RAW domain enhanced image based on the target illuminance enhancement feature map and the target color restoration feature map.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the RAW domain low-light image enhancement method as described above.

[0016] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the RAW domain low-light image enhancement method as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the RAW domain low-light image enhancement method as described above.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The RAW domain low-light image enhancement method and system provided by this invention effectively avoids noise amplification during subsequent enhancement processes by separating the content signal map and noise residual map of the RAW domain low-light image, thus reducing noise and color distortion in the enhanced RAW domain image. By simulating the projection of light rays through the camera's optical path and performing ray integration using the content signal map, physical-level detail reconstruction can be performed on areas with severe structural deficiencies in the RAW domain low-light image, achieving global illumination consistency and edge sharpness superior to traditional two-dimensional convolutional networks. By introducing a target dual-stream recurrent neural network, the optimization conflict between denoising and color restoration in low-light enhancement can be resolved, enabling a progressive and synergistic improvement in image quality and ensuring a high degree of harmony in brightness, detail, and color in the final enhancement result. Furthermore, this method decouples the two heterogeneous tasks of noise elimination and color space conversion, avoiding direct mapping from the noisy RAW domain to the clean sRGB domain, thus preventing domain blurring effects and systematic color shifts. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is one of the flowcharts illustrating the RAW domain low-light image enhancement method provided by the present invention.

[0021] Figure 2 This is the second schematic diagram of the RAW domain low-light image enhancement method provided by the present invention.

[0022] Figure 3 This is a schematic diagram of the training process of the dual-branch decoupled neural network in the RAW domain low-light image enhancement method provided by the present invention.

[0023] Figure 4 This is a schematic diagram of the process of determining the initial illuminance enhancement feature map using the content signal map in this invention.

[0024] Figure 5 This is a schematic diagram of the processing flow of the target dual-stream recurrent neural network provided by the present invention.

[0025] Figure 6 This is a schematic diagram of the structure of the RAW domain low-light image enhancement system provided by the present invention.

[0026] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0028] Figure 1 This is a flowchart illustrating a low-light image enhancement method in the RAW domain provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes: S1, acquire a low-light image in the RAW domain, and extract the content signal map of the low-light image in the RAW domain; S2, based on the content signal map, simulate the camera optical path to project light rays and perform light integration to determine the initial illuminance enhancement feature map; S3, based on the target channel attention module and the target space attention module, adaptively modulate the color channel features of the RAW domain low-light image to obtain the initial color recovery feature map; S4. Based on the target dual-stream recurrent neural network, the initial illuminance enhancement feature map and the initial color restoration feature map are iteratively fused and updated to obtain the target illuminance enhancement feature map and the target color restoration feature map. Based on the target illuminance enhancement feature map and the target color restoration feature map, the RAW domain enhanced image is determined.

[0029] Specifically, the RAW domain low-light image enhancement method provided in this embodiment of the invention is executed by a RAW domain low-light image enhancement system. This system can be a heterogeneous computing acceleration module, which can be configured in a smartphone, digital camera, security monitoring equipment, autonomous driving vehicle computing platform, or industrial vision inspection terminal.

[0030] First, step S1 is executed to acquire a RAW domain low-light image. This RAW domain low-light image refers to the raw data directly output from the image sensor, without previous non-linear processing such as mosaicking and color correction. It is typically arranged in a Bayer array, with each pixel containing only one color component among red, green, or blue. In extremely low-light environments, the signal-to-noise ratio of this RAW domain low-light image is extremely low, and details are severely masked by physical noise.

[0031] After acquiring the RAW domain low-light image, preprocessing operations such as black level correction, normalization, and Bayer packing can be performed on the RAW domain low-light image.

[0032] Because image sensors receive very few photons in low-light environments, structural information and noise are highly coupled in low-light RAW domain images. Therefore, in this embodiment of the invention, a Poisson-Gaussian hybrid physical degradation model can be constructed using the physical mechanism of sensor photoelectric conversion. ; in, For low-light images in the RAW domain, This is a content signal map of a low-light image in the RAW domain. It is the Poisson noise term representing photon shot noise in low-light images in the RAW domain. This is a noise residual map of a low-light image in the RAW domain. It is a Gaussian distribution term representing the circuit readout noise of the image sensor in a low-light image in the RAW domain. This represents the variance of the circuit readout noise of the image sensor in low-light images in the RAW domain.

[0033] By employing the Poisson-Gaussian mixture physics degradation model, the content signal map and noise residual map of RAW domain low-light images can be extracted, enabling precise separation of content signal and noise from the imaging mechanism perspective. The content signal map refers to the clean image features separated from the RAW domain low-light image after noise removal, preserving the scene's topological structure information. The noise residual map corresponds to various random interferences generated during the imaging process, mainly including photon shot noise and image sensor circuit readout noise. Specifically, image sensor circuit readout noise refers to the random level fluctuations introduced by the image sensor's circuitry itself when converting photoelectrons into voltage and performing analog-to-digital conversion.

[0034] Then, step S2 is executed to expand the content signal map from a two-dimensional space to a three-dimensional voxel space. Specifically, the content signal map is first used to simulate the camera's projected light rays, obtaining the ray direction vector. Multiple camera projected light rays can be generated, each starting at the origin and ending at a pixel in the content signal map. Here, the origin is typically the camera's optical center or the origin of the pinhole model. The camera projected light ray can be represented as: ; in, Projecting light into the camera's optical path The coordinates of a three-dimensional point in space that is t away from the origin. Let d be the coordinates of the origin, and d be the ray projected from the camera's optical path. The line-of-sight vector, t is the ray projected from the camera's optical path. Distance along the direction of projection.

[0035] To achieve accurate reconstruction of structural details, gradient information from the content signal map is used to perform layered sampling of the camera's projected light rays, obtaining each sampling point on the projected light rays. For example, the gradient at different locations on the projected light rays can be used to perform layered sampling, with dense sampling at locations with lower gradients and a smaller dynamic integration step size, and sparse sampling at locations with higher gradients and a larger dynamic integration step size.

[0036] Here, the sampling points on the projected light rays of the camera's optical path, together with the two-dimensional content signal map, constitute a three-dimensional voxel space.

[0037] By utilizing the position information of each sampling point on the camera's optical path projection ray and the ray's line-of-sight direction vector, the voxel density and radiance features of each sampling point on each camera's optical path projection ray are extracted. Then, using the voxel density and radiance features of each sampling point on each camera's optical path projection ray, ray integration is performed in three-dimensional voxel space to separate the geometric structure features and radiance features.

[0038] The radiance characteristics of the rays projected by each camera's optical path can be obtained by integrating the rays using the following volumetric rendering equation: ; ; in, Projecting light into the camera's optical path The radiance characteristics, where M is the light ray projected from the camera's optical path. The number of sampling points on the surface Projecting light into the camera's optical path The location information of the m-th sampling point for Cumulative transmittance at that location for Three-dimensional voxel density at the location, for The radiance characteristics at that location. Projecting light into the camera's optical path The location information of the s-th sampling point for Three-dimensional voxel density at the location, Projecting light into the camera's optical path The dynamic integration step size at the s-th sampling point.

[0039] By stitching together the radiance characteristics of the light rays projected from all camera optical paths, the initial illuminance enhancement feature map can be obtained. The initial illuminance enhancement feature map is a high-quality grayscale illuminance map with a high dynamic range.

[0040] Next, step S3 is executed. Addressing the color response imbalance caused by low-light scenes, the target channel attention module and the target spatial attention module are used to adaptively modulate the color channel features of the RAW domain low-light image, resulting in an initial color restoration feature map. The target channel attention module is a functional component that calculates the weights of different frequency sub-bands in the frequency domain using a channel attention mechanism to suppress high-frequency noise energy and balance the color channel response. The target spatial attention module is a functional component that calculates the weights at different locations in the frequency domain using a spatial attention mechanism to accurately locate and preserve color boundaries and texture edge features. The initial color restoration feature map refers to the intermediate features that, after frequency domain filtering and enhancement processing, have eliminated color cast and preliminarily restored natural color attributes.

[0041] Finally, step S4 is executed, inputting the initial illuminance enhancement feature map and the initial color restoration feature map into the target two-stream recurrent neural network. The target two-stream recurrent neural network is a recursive processing architecture that includes an illuminance enhancement branch and a color restoration branch, and these branches can exchange information across branches through hidden states. The initial illuminance enhancement feature map and the initial color restoration feature map are iteratively fused and updated through the illuminance enhancement branch and the color restoration branch to obtain the target illuminance enhancement feature map and the target color restoration feature map. In each iteration, the illuminance enhancement branch can fuse the previous illuminance enhancement feature map obtained in the previous iteration with the previous color hidden state feature obtained by the color restoration branch in the previous iteration to obtain intermediate illuminance features. Illuminance enhancement and structural detail restoration are then performed on the intermediate illuminance features to obtain the current illuminance enhancement feature map. Spatial information compression and high-dimensional feature abstraction are then performed on the current illuminance enhancement feature map to obtain the current illuminance hidden state features.

[0042] The color restoration branch can fuse the previous color restoration feature map obtained in the previous iteration with the previous illuminance enhancement feature map obtained in the previous iteration from the illuminance enhancement branch to obtain intermediate color features. Color correction and restoration are then performed on these intermediate color features to obtain the current color restoration feature map. Spatial information compression and high-dimensional feature abstraction are then applied to the current color restoration feature map to obtain the final color restoration feature map.

[0043] When the iteration termination condition is met, the current illuminance enhancement feature map and the current color restoration feature map are used as the target illuminance enhancement feature map and the target color restoration feature map, respectively; otherwise, the next iteration round is used as the current iteration round, and the above process is repeated until the iteration termination condition is met.

[0044] Finally, by combining the target illumination enhancement feature map and the target color restoration feature map, the RAW domain enhanced image can be obtained. First, the target illumination enhancement feature map and the target color restoration feature map can be concatenated along the channel dimension to construct a multi-channel composite feature tensor. Then, the composite feature tensor is input into a fusion convolutional layer. This fusion convolutional layer performs cross-channel linear combination and non-linear mapping on the composite feature tensor, compressing and reducing the number of channels to the target number of channels, thus obtaining the RAW domain enhanced image. The convolution kernel of the fusion convolutional layer can be a 1×1 kernel. The number of channels in the RAW domain enhanced image is the target number of channels, and the RAW domain enhanced image can contain complete structural details and accurate color information.

[0045] The RAW domain low-light image enhancement method provided in this embodiment of the invention effectively avoids noise amplification during subsequent enhancement processes by separating the content signal map and noise residual map of the RAW domain low-light image, thereby reducing noise and color distortion in the enhanced RAW domain image. By simulating the projection of light rays through the camera's optical path and performing ray integration using the content signal map, physical-level detail reconstruction can be performed on areas with severe structural deficiencies in the RAW domain low-light image, achieving global illumination consistency and edge sharpness superior to traditional two-dimensional convolutional networks. By introducing a target dual-stream recurrent neural network, the optimization conflict between denoising and color restoration in low-light enhancement can be resolved, enabling a progressive and synergistic improvement in image quality and ensuring a high degree of harmony in brightness, detail, and color in the final enhancement result. Furthermore, this method decouples the two heterogeneous tasks of noise elimination and color space conversion, avoiding direct mapping from the noisy RAW domain to the clean sRGB domain, thus preventing domain blurring effects and systematic color shifts.

[0046] Based on the above embodiments, the extraction of the content signal map of the RAW domain low-light image includes: The RAW domain low-light image is input into a two-branch decoupled neural network to obtain the content signal map output by the content branch network and the noise residual map output by the noise branch network in the two-branch decoupled neural network. The dual-branch decoupled neural network is obtained by supervising training based on normal illumination denoised images of RAW domain low-light image samples and corresponding RAW domain normal illumination image samples, combined with content reconstruction constraints, noise statistical consistency constraints and total variation regularization constraints.

[0047] Specifically, in this embodiment of the invention, a two-branch decoupling neural network can be introduced. The two-branch decoupling neural network uses a Poisson-Gaussian mixture physical degradation model to extract the content signal map and noise residual map of the low-light image in the RAW domain.

[0048] A dual-branch decoupled neural network is an end-to-end deep learning architecture specifically designed to decouple coupled content signals from noise signals. It comprises parallel content branch networks and noise branch networks, each responsible for processing heterogeneous physical signals. These two branches remain independent during feature extraction. Through this parallel processing mechanism, the system can initially separate complex noise and weak content signals, which are difficult to handle using traditional methods, at the physical level.

[0049] Content branch networks typically employ a U-Net architecture with a symmetrical structure. An encoder extracts multi-scale features, and a decoder reconstructs a clean content signal map. Noise branch networks, on the other hand, are lightweight neural networks specifically designed to fit the residual distribution. They do not focus on the geometric structure of the image but rather on capturing the random fluctuations of the imaging sensor.

[0050] The RAW domain low-light image is simultaneously fed into the content branch network and the noise branch network. The content branch network uses deep convolutional layers to learn structural priors in the low-light background under a large receptive field, removing granular interference and outputting a high-fidelity content signal map. At the same time, the noise branch network extracts random noise distribution features that are unrelated to the content signal intensity from the content signal map and outputs a noise residual map.

[0051] The dual-branch decoupling neural network can be obtained by supervising the training of the initial dual-branch decoupling neural network using RAW domain low-light image samples and corresponding RAW domain normal-light image samples with normal-light denoised images, combined with content reconstruction constraints, noise statistical consistency constraints and total variation regularization constraints.

[0052] RAW domain low-light image samples are RAW domain low-light images captured by the image sensor in extremely dark environments. RAW domain normal-light image samples can be RAW domain normal-light images captured by long exposure under the same scene. Normal-light denoising images are obtained by denoising RAW domain normal-light image samples and using them as the ground truth (GT) as the standard answer for supervised learning. It is a completely noise-free and detailed ideal image obtained by preprocessing images captured under normal exposure conditions.

[0053] Low-light image samples in the RAW domain are input into an initial two-branch decoupled neural network. The content branch network in the initial two-branch decoupled neural network outputs a predicted content signal map, and the noise branch network in the initial two-branch decoupled neural network outputs a predicted noise residual map.

[0054] Content reconstruction constraints refer to a loss mechanism that forces the predicted content signal map to approximate a normally illuminated denoised image at both the pixel and perceptual levels; this is typically represented by Mean Squared Error (MSE) loss. Noise statistical consistency constraints utilize probability distribution metrics to require the predicted noise residual map to conform to the true physical distribution of the image sensor; this is usually a KL divergence constraint. Total variation regularization constraints suppress unnatural artifacts by calculating the total variation of the predicted content signal map, ensuring local smoothness in the predicted content signal map.

[0055] Here, the first training loss can be calculated by weighted summation of the content reconstruction constraint, the noise statistical consistency constraint, and the total variation regularization constraint. This first training loss can be expressed as: ; in, This is the first training loss. For denoising images under normal illumination, To predict the content signal graph, Denotes the KL divergence function. To predict the noise residual map, Poisson noise, To represent the Gaussian distribution term of the image sensor's circuit readout noise in low-light image samples in the RAW domain, This represents the variance of the circuit readout noise of the image sensor in low-light image samples in the RAW domain. It is a total variation function. and These are the weight coefficients for the noise statistical consistency constraint and the total variation regularization constraint, respectively.

[0056] Using the first training loss, the parameters of the initial two-branch decoupled neural network are continuously optimized through the backpropagation algorithm until the first training loss converges or the training ends after reaching the first specified number of iterations, thereby obtaining a two-branch decoupled neural network that can accurately decouple real noise.

[0057] In this embodiment of the invention, a dual-branch collaborative architecture is used to achieve dedicated processing of content and noise, which avoids the detail blurring problem that is easily caused by a single network during denoising, ensuring that the feature maps obtained in subsequent enhancement steps have extremely high fidelity. Moreover, by introducing various constraints during training, the initial dual-branch decoupled neural network can be freed from its dependence on blind data fitting, thereby improving the generalization ability of the initial dual-branch decoupled neural network to different low-light environments and sensor characteristics.

[0058] Based on the above embodiments, the step of simulating the projection of light rays from the camera's optical path and performing ray integration based on the content signal map, separating the geometric structural features and radiance features in the three-dimensional voxel space, and determining the initial illuminance enhancement feature map includes: Based on the content signal map, the camera optical path projection light is simulated to obtain the light line direction vector, and based on the gradient information of the content signal map, the camera optical path projection light is sampled in layers to obtain each sampling point on the camera optical path projection light. The position information of each sampling point on the camera's optical path projection ray and the ray's line-of-sight vector are input into the target neural radiation field model. The target neural radiation field model performs ray integration to obtain the initial illumination enhancement feature map output by the target neural radiation field model.

[0059] Specifically, in the process of determining the initial illumination enhancement feature map, a target neural radiance field (NeRF) model can be introduced. The target neural radiance field model takes the position information of each sampling point on the camera optical path projection ray and the ray line direction vector as input. The ray line direction vector can be obtained by simulating the camera optical path projection ray through the content signal map. Each sampling point on the camera optical path projection ray can be obtained by performing layered sampling of the camera optical path projection ray through the gradient information of the content signal map.

[0060] The target neural radiation field model uses the input query to obtain the three-dimensional voxel density and radiance features of each sampling point on the projected light rays of each camera optical path. It then uses a volumetric rendering equation to integrate and accumulate the radiance features of each sampling point on the projected light rays of each camera optical path, obtaining and outputting an initial illumination enhancement feature map. This target neural radiation field model can be implemented using a multilayer perceptron (MLP).

[0061] In this embodiment of the invention, by operating from two-dimensional space to three-dimensional space and back to two-dimensional space, physical-level illumination compensation can be achieved for the content signal map, improving the accuracy and determination efficiency of the initial illumination enhancement feature map. This method introduces a target neural radiation field model, which can perform high-density sampling on structurally blurred, low-gradient regions in low-light images, achieving a physical-level dynamic range expansion of illumination that is impossible with traditional 2D convolutional networks.

[0062] Based on the above embodiments, the iterative fusion and update of the initial illuminance enhancement feature map and the initial color recovery feature map based on the target dual-stream recurrent neural network to obtain the target illuminance enhancement feature map and the target color recovery feature map includes: For the current iteration, the previous illuminance enhancement feature map and the previous color restoration feature map from the previous iteration are respectively input into the illuminance enhancement branch and the color restoration branch of the target dual-stream recurrent neural network; The illuminance enhancement branch is used to fuse the previous illuminance enhancement feature map with the previous color hidden state feature of the previous iteration to obtain the intermediate illuminance feature, and to enhance the intermediate illuminance feature and restore the structural details to obtain the current illuminance enhancement feature map. Based on the current illuminance enhancement feature map, the current illuminance hidden state feature is determined. The color recovery branch is used to fuse the previous color recovery feature map with the previous illuminance latent state feature of the previous iteration to obtain intermediate color features, and to perform color correction and color recovery on the intermediate color features to obtain the current color recovery feature map. Based on the current color recovery feature map, the current color latent state feature is determined. If the current illuminance enhancement feature map, the current color restoration feature map, the current illuminance latent state feature map, and the current color latent state feature map all converge, or reach a preset iteration round, then the current illuminance enhancement feature map and the current color restoration feature map are respectively used as the target illuminance enhancement feature map and the target color restoration feature map; otherwise, the next iteration round is used as the current iteration round, and the above process is iterated.

[0063] Specifically, after receiving the initial illumination enhancement feature map and the initial color restoration feature map, the target dual-stream recurrent neural network begins the iterative process. The same operations are performed in each iteration. Taking the current iteration as an example, this current iteration can be any iteration within the overall iterative process.

[0064] The previous illuminance enhancement feature map and the previous color restoration feature map from the previous iteration can be input into the illuminance enhancement branch and the color restoration branch of the target two-stream recurrent neural network, respectively. In particular, if the current iteration is the first iteration, since there are no iterations before the first iteration, both the previous illuminance enhancement feature map and the previous color restoration feature map are empty, and the initial illuminance enhancement feature map is the current illuminance enhancement feature map; similarly, the previous color restoration feature map and the previous illuminance hidden state feature are also empty, and the initial color restoration feature map is the current color restoration feature map.

[0065] If the current iteration is not the first iteration, the illuminance enhancement branch can fuse the previous illuminance enhancement feature map with the previous color latent state feature through a fusion processor to obtain intermediate illuminance features, which can be represented as: ; in, It is a characteristic of intermediate illuminance. Let t be the fusion unit, t be the current iteration round, and t-1 be the previous iteration round. This is the feature map of the previous illumination enhancement. This is a feature of the previous color's hidden state.

[0066] The illuminance enhancement branch, through the main branch, can enhance the illuminance features of intermediate illuminance areas and restore structural details to obtain the current illuminance enhancement feature map. Here, the main branch can be implemented by a Convolutional Gated Recurrent Unit (ConvGRU) or a U-Net network.

[0067] The illuminance enhancement branch can extract the current illuminance hidden state features from the current illuminance enhancement feature map through a shared hidden state feature extractor. Here, the shared latent state feature extractor is implemented through a global average pooling (GAP) layer and a fully connected layer (FC).

[0068] The color restoration branch can fuse the previous color restoration feature map with the previous illumination latent state feature from the previous iteration using a fusion processor to obtain intermediate color features, which can be represented as: ; in, It is a mid-color feature. This is the feature map for restoring the previous color. This represents the hidden state characteristics of the previous illuminance.

[0069] The color restoration branch, through the main branch, can perform color correction and color restoration on intermediate color features to obtain the current color restoration feature map. Here, the main branch can also be implemented by a convolutional recurrent neural network (ConvGRU) or a U-Net network.

[0070] The color restoration branch can extract the current color hidden state features from the current color restoration feature map through a shared hidden state feature extractor. Here, the shared hidden state feature extractor can also be implemented using a global average pooling layer and a fully connected layer.

[0071] If the current illuminance enhancement feature map, the current color restoration feature map, the current illuminance latent state feature map, and the current color latent state feature map all converge, or if the preset number of iterations is reached, the current illuminance enhancement feature map and the current color restoration feature map can be directly used as the target illuminance enhancement feature map and the target color restoration feature map, respectively.

[0072] Otherwise, the next iteration is taken as the current iteration, and the above process is repeated until the iteration termination condition is met, that is, the current illuminance enhancement feature map, the current color restoration feature map, the current illuminance latent state feature, and the current color latent state feature all converge, or the preset iteration number is reached, the iteration process ends, and the current illuminance enhancement feature map and the current color restoration feature map of the current iteration are taken as the target illuminance enhancement feature map and the target color restoration feature map, respectively.

[0073] It is understandable that the convergence of the current illuminance enhancement feature map, the current color restoration feature map, the current illuminance latent state feature, and the current color latent state feature means that the structural similarity between the current illuminance enhancement feature map and the previous illuminance enhancement feature map is greater than a first threshold, the mean square error between the current color restoration feature map and the previous color restoration feature map is less than a second threshold, the change in error between the current illuminance latent state feature and the previous illuminance latent state feature is less than a third threshold, and the change in error between the current color latent state feature and the previous color latent state feature is less than a fourth threshold. Here, the first, second, third, and fourth thresholds can all be set as needed.

[0074] In this embodiment of the invention, the implicit state cross-feedback mechanism between the illumination enhancement branch and the color restoration branch can be used to achieve iterative fusion and update of the initial illumination enhancement feature map and the initial color restoration feature map, thereby overcoming the optimization conflict between noise reduction smoothing and color texture preservation in low illumination enhancement.

[0075] Based on the above embodiments, the adaptive modulation of the color channel features of the RAW domain low-light image based on the target channel attention module and the target space attention module to obtain an initial color restoration feature map includes: The color channel features of the RAW domain low-light image are subjected to a two-dimensional discrete Fourier transform to obtain the frequency domain features. Based on the target channel attention module and the target space attention module, the frequency domain features are adaptively modulated in different frequency sub-bands in the frequency domain to obtain modulated frequency domain features. The modulation frequency domain features are subjected to inverse Fourier transform to obtain the initial color recovery feature map.

[0076] Specifically, in the process of adaptively modulating the color channel features of a low-light image in the RAW domain, the color channel features of the low-light image in the RAW domain can first be subjected to a two-dimensional discrete Fourier transform to convert them into the frequency domain, thus obtaining the frequency domain features.

[0077] Subsequently, using the target channel attention module and the target space attention module, the frequency domain features are adaptively modulated in different frequency sub-bands in the frequency domain to obtain the modulated frequency domain features. These modulated frequency domain features can be expressed as: ; in, For modulation frequency domain characteristics, For frequency domain characteristics, The frequency coordinates are two-dimensional in the frequency domain. Represents the spatial frequency variable in the horizontal direction. Represents the spatial frequency variable in the vertical direction. The channel attention weights are generated by the target channel attention module to adaptively suppress high-frequency noise bands. This refers to the spatial attention weights generated by the target spatial attention module using the frequency domain amplitude spectrum, which are used for precise localization and to preserve boundary information of color transitions. This frequency domain amplitude spectrum can be the complex modulus of the frequency domain features.

[0078] The target channel attention module can be implemented using a multilayer perceptron, while the target space attention module can be implemented using convolutional layers or preset operators.

[0079] Finally, by performing an inverse Fourier transform on the modulation frequency domain features, the initial color recovery feature map can be obtained.

[0080] In this embodiment of the invention, by using a dual attention mechanism in the frequency domain to separate color boundaries and noise, the industry problem of red and blue channel response imbalance caused by image sensors in low-light scenes can be accurately solved.

[0081] Based on the above embodiments, the target dual-stream recurrent neural network is trained using the following steps: Determine the low-light denoised images of the RAW domain normal illumination image samples and the corresponding RAW domain low illumination image samples; Based on the content signal map of the low-light denoised image, the initial neural radiation field model is applied to obtain the first illumination enhancement feature map. Based on the initial channel attention module and the initial spatial attention module, the color channel features of the RAW domain low-light image sample are adaptively modulated to obtain the first color restoration feature map. The first illuminance enhancement feature map and the first color restoration feature map are input into the initial two-stream recurrent neural network to obtain the predicted illuminance enhancement feature map and the predicted color restoration feature map output by the initial two-stream recurrent neural network. Based on the predicted illuminance enhancement feature map and the predicted color restoration feature map, the predicted RAW domain enhanced image is determined. Based on the predicted RAW domain enhanced image and the RAW domain normal illumination image sample, the initial neural radiation field model, the initial channel attention module, the initial spatial attention module and the initial dual-stream recurrent neural network are iteratively trained to obtain the target neural radiation field model, the target channel attention module, the target spatial attention module and the target dual-stream recurrent neural network.

[0082] Specifically, the target dual-stream recurrent neural network can be jointly trained end-to-end with the target neural radiation field model, the target channel attention module, and the target spatial attention module. Furthermore, during the joint training process, the two-branch decoupled neural network can be fine-tuned; that is, the target dual-stream recurrent neural network can be jointly trained end-to-end with the two-branch decoupled neural network, the target neural radiation field model, the target channel attention module, and the target spatial attention module.

[0083] During end-to-end joint training, the low-light denoised images of the RAW domain normal illumination image samples and the corresponding RAW domain low-light image samples are first determined. These low-light denoised images can be obtained by denoising the RAW domain low-light image samples, for example, by inputting the RAW domain low-light image samples into a two-branch decoupled neural network, from which the content signal map is obtained.

[0084] Using the content signal map of the low-light denoised image, the projected light rays of the camera's optical path are simulated to obtain the light ray direction vector corresponding to the low-light denoised image. Then, using the gradient information of the content signal map of the low-light denoised image, the simulated projected light rays of the camera's optical path are sampled layer by layer to obtain each sampling point on the projected light rays. The position information of each sampling point on the simulated projected light rays of the camera's optical path and the light ray direction vector corresponding to the low-light denoised image are input into the initial neural radiation field model to obtain the first illumination enhancement feature map.

[0085] Using the initial channel attention module and the initial spatial attention module, the color channel features of the RAW domain low-light image samples are adaptively modulated to obtain the first color recovery feature map.

[0086] The first illumination enhancement feature map and the first color restoration feature map are input into the initial two-stream recurrent neural network. The initial two-stream recurrent neural network is used to obtain and output the predicted illumination enhancement feature map and the predicted color restoration feature map. The predicted RAW domain enhanced image is obtained by fusing the predicted illumination enhancement feature map and the predicted color restoration feature map.

[0087] Subsequently, using the predicted RAW domain enhanced image and RAW domain normal illumination image samples, a second training loss can be calculated. This second training loss can include a data fidelity term, a priori regularization loss term, and a total variation loss term, which can be expressed as: ; in, For the second training loss, This is a data fidelity term used to measure the difference between the predicted RAW domain enhanced image and the RAW domain normal illumination image sample. This is a priori regularization loss term used to constrain the noise residual map separated by the two-branch decoupled neural network to conform to a pre-defined physical statistical law. The total variational loss term is used to constrain the smoothness of the content signal graph separated by the two-branch decoupled neural network and suppress abrupt changes in local noise. and These are the weight coefficients for the prior regularization loss term and the total variation loss term, respectively.

[0088] Using the second training loss, the initial neural radiation field model, initial multilayer perceptron, initial channel attention module, initial spatial attention module, and initial two-stream recurrent neural network are simultaneously optimized through the backpropagation algorithm until the second training loss converges or the training ends after reaching the second specified number of iterations, thereby obtaining the target neural radiation field model, target channel attention module, target spatial attention module, and target two-stream recurrent neural network.

[0089] In this embodiment of the invention, by jointly training the target neural radiation field model, the target channel attention module, the target spatial attention module, and the target dual-stream recurrent neural network, the final RAW domain enhanced image can be made more accurate.

[0090] Based on the above embodiments, the step of determining the RAW domain enhanced image based on the target illumination enhancement feature map and the target color restoration feature map includes: The RAW domain enhanced image is subjected to nonlinear processing to obtain the target RGB enhanced image.

[0091] Specifically, after determining the RAW domain enhanced image, non-linear processing can be performed on it, such as adaptive demosaicing, color correction, and mapping, to obtain the target RGB enhanced image. Color correction and mapping can include white balance correction, color restoration, and gamma mapping. This target RGB enhanced image is a high-fidelity, low-noise RGB enhanced image, which can be used for downstream monitoring interface display or as a detection target in defect detection algorithms.

[0092] Figure 2 This is a schematic diagram illustrating the complete process of the RAW domain low-light image enhancement method provided in this embodiment of the invention, as shown below. Figure 2 As shown, the method includes: Acquire low-light images in the RAW domain; Preprocessing of low-light images in the RAW domain; The RAW domain low-light image is input into a two-branch decoupled neural network to obtain the content signal map and noise residual map of the RAW domain low-light image; The initial illuminance enhancement feature map is determined using the content signal map; By using the target channel attention module and the target space attention module, the color channel features of the RAW domain low-light image are adaptively modulated to obtain the initial color restoration feature map. By utilizing the illuminance enhancement branch and color restoration branch in the target dual-stream recurrent neural network, and employing a hidden state feedback cross-feedback mechanism, the initial illuminance enhancement feature map and the initial color restoration feature map are iteratively fused and updated to obtain the target illuminance enhancement feature map and the target color restoration feature map. The target illumination enhancement feature map and the target color restoration feature map are fused to determine the RAW domain enhanced image; The target RGB enhanced image is obtained by performing nonlinear processing on the RAW domain enhanced image.

[0093] Figure 3 This is a schematic diagram illustrating the training process of the dual-branch decoupled neural network in the RAW domain low-light image enhancement method provided in this embodiment of the invention, as shown below. Figure 3 As shown, the training process of a two-branch decoupled neural network includes: This process acquires low-light RAW image samples and corresponding normal-light RAW image samples with denoised normal-light images. Furthermore, it extracts metadata from the low-light RAW images, including camera parameters, exposure information, white balance, and noise characteristics. This metadata helps the Poisson-Gaussian mixture physical degradation model more accurately match the real physical noise distribution of the image sensor in extreme low-light scenes, thus achieving high-quality physical-level decoupling of the content signal from the noise. Noise characteristics can include photon shot noise and circuit readout noise.

[0094] Low-light image samples in the RAW domain are input into an initial two-branch decoupled neural network. The content branch network in the initial two-branch decoupled neural network outputs a predicted content signal map, and the noise branch network in the initial two-branch decoupled neural network outputs a predicted noise residual map.

[0095] By using the predicted content signal map and the predicted noise residual map, combined with content reconstruction constraints, noise statistical consistency constraints and total variation regularization constraints, the initial two-branch decoupling neural network is trained under supervision, thereby obtaining the two-branch decoupling neural network.

[0096] Figure 4 This is a schematic diagram of the process for determining the initial illuminance enhancement feature map using the content signal map in an embodiment of the present invention, such as... Figure 4 As shown, the determination process includes: Determine the content signal map of a low-light image in the RAW domain; By using the content signal map to simulate the projection of light rays from the camera's optical path, the direction vector of the light rays' line of sight is obtained; Using the gradient information of the content signal map, the camera optical path projection rays are sampled in layers to obtain each sampling point on the camera optical path projection rays; The position information of each sampling point on the camera's optical path projection ray and the ray's line-of-sight vector are input into the target neural radiation field model to obtain the initial illumination enhancement feature map output by the target neural radiation field model.

[0097] Figure 5 This is a schematic diagram of the processing flow of the target dual-stream recurrent neural network in an embodiment of the present invention, as shown below. Figure 5 As shown, the processing flow includes: The initial illuminance enhancement feature map and the initial color restoration feature map are respectively input into the illuminance enhancement branch and the color restoration branch of the target two-stream recurrent neural network; The illuminance enhancement branch and the color restoration branch perform the following iterative process: For the current iteration, the previous illuminance enhancement feature map and the previous color restoration feature map from the previous iteration can be input into the illuminance enhancement branch and the color restoration branch of the target dual-stream recurrent neural network, respectively. The illuminance enhancement branch first fuses the previous illuminance enhancement feature map with the previous color latent state feature through the fusion unit to obtain the intermediate illuminance feature. Then, through the main branch, the intermediate illuminance feature is enhanced with illuminance and structural details are restored to obtain the current illuminance enhancement feature map. Finally, the current illuminance latent state feature is extracted from the current illuminance enhancement feature map through the shared latent state feature extractor. The color restoration branch first fuses the previous color restoration feature map with the previous illuminance latent state feature through a fusion unit to obtain intermediate color features. Then, through the main branch, color correction and color restoration are performed on the intermediate color features to obtain the current color restoration feature map. Finally, the shared latent state feature extractor extracts the current color latent state features from the current color restoration feature map. The next iteration is taken as the current iteration. Through the hidden state cross-feedback mechanism between the illuminance enhancement branch and the color restoration branch, the current illuminance enhancement feature map and the current color restoration feature map are iterated smaller and smaller until the iteration termination condition is met. The current illuminance enhancement feature map and the current color restoration feature map are taken as the target illuminance enhancement feature map and the target color restoration feature map, respectively.

[0098] like Figure 6 As shown, based on the above embodiments, this embodiment of the invention provides a RAW domain low-light image enhancement system, comprising: Image acquisition module 61 is used to acquire a low-light image in the RAW domain and extract the content signal map of the low-light image in the RAW domain; Illumination enhancement module 62 is used to simulate the projection of light rays from the camera optical path and perform light ray integration based on the content signal map to determine the initial illumination enhancement feature map; Color restoration module 63 is used to adaptively modulate the color channel features of the RAW domain low-light image based on the target channel attention module and the target space attention module to obtain an initial color restoration feature map. The fusion update module 64 is used to iteratively fuse and update the initial illuminance enhancement feature map and the initial color restoration feature map based on the target dual-stream recurrent neural network to obtain the target illuminance enhancement feature map and the target color restoration feature map, and to determine the RAW domain enhanced image based on the target illuminance enhancement feature map and the target color restoration feature map.

[0099] Specifically, the functions of each module in the RAW domain low-light image enhancement system provided in this embodiment correspond one-to-one with the operation flow of each step in the above method embodiment, and the achieved effects are also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment.

[0100] Figure 7An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute the RAW domain low-light image enhancement method provided in the above embodiments.

[0101] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0102] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the RAW domain low-light image enhancement method provided in the above embodiments.

[0103] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the RAW domain low-light image enhancement method provided in the above embodiments. This computer-readable storage medium can be either a non-transitory computer-readable storage medium or a transient computer-readable storage medium, and is not specifically limited herein.

[0104] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0105] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for enhancing low-light images in the RAW domain, characterized in that, include: Acquire a low-light image in the RAW domain and extract the content signal map of the low-light image in the RAW domain; Based on the content signal map, simulate the camera's optical path to project light rays and perform light integration to determine the initial illuminance enhancement feature map; Based on the target channel attention module and the target space attention module, the color channel features of the RAW domain low-light image are adaptively modulated to obtain an initial color recovery feature map. Based on the target dual-stream recurrent neural network, the initial illuminance enhancement feature map and the initial color restoration feature map are iteratively fused and updated to obtain the target illuminance enhancement feature map and the target color restoration feature map. Based on the target illuminance enhancement feature map and the target color restoration feature map, the RAW domain enhanced image is determined. The target channel attention module and the target space attention module adaptively modulate the color channel features of the RAW domain low-light image to obtain an initial color restoration feature map, including: The color channel features of the RAW domain low-light image are subjected to a two-dimensional discrete Fourier transform to obtain the frequency domain features. Based on the target channel attention module and the target space attention module, the frequency domain features are adaptively modulated in different frequency sub-bands in the frequency domain to obtain modulated frequency domain features. The modulation frequency domain features are subjected to inverse Fourier transform to obtain the initial color recovery feature map; The step of determining the RAW domain enhanced image based on the target illumination enhancement feature map and the target color restoration feature map includes: The target illumination enhancement feature map and the target color restoration feature map are concatenated along the channel dimension to obtain a composite feature tensor; The composite feature tensor is input into a fusion convolutional layer, which compresses and reduces the number of channels of the composite feature tensor to the target number of channels, thereby obtaining the RAW domain enhanced image.

2. The RAW domain low-light image enhancement method according to claim 1, characterized in that, The extraction of the content signal map from the RAW domain low-light image includes: The RAW domain low-light image is input into a two-branch decoupled neural network to obtain the content signal map output by the content branch network and the noise residual map output by the noise branch network in the two-branch decoupled neural network. The dual-branch decoupled neural network is obtained by supervising training based on normal illumination denoised images of RAW domain low-light image samples and corresponding RAW domain normal illumination image samples, combined with content reconstruction constraints, noise statistical consistency constraints and total variation regularization constraints.

3. The RAW domain low-light image enhancement method according to claim 1, characterized in that, The step of simulating the projection of light rays from the camera's optical path and performing ray integration based on the content signal map to determine the initial illumination enhancement feature map includes: Based on the content signal map, the camera optical path projection light is simulated to obtain the light line direction vector, and based on the gradient information of the content signal map, the camera optical path projection light is sampled in layers to obtain each sampling point on the camera optical path projection light. The position information of each sampling point on the camera's optical path projection ray and the ray's line-of-sight vector are input into the target neural radiation field model. The target neural radiation field model performs ray integration to obtain the initial illumination enhancement feature map output by the target neural radiation field model.

4. The RAW domain low-light image enhancement method according to claim 3, characterized in that, The method based on a target dual-stream recurrent neural network iteratively fuses and updates the initial illuminance enhancement feature map and the initial color restoration feature map to obtain a target illuminance enhancement feature map and a target color restoration feature map, including: For the current iteration, the previous illuminance enhancement feature map and the previous color restoration feature map from the previous iteration are respectively input into the illuminance enhancement branch and the color restoration branch of the target dual-stream recurrent neural network; The illuminance enhancement branch is used to fuse the previous illuminance enhancement feature map with the previous color hidden state feature of the previous iteration to obtain the intermediate illuminance feature, and to enhance the intermediate illuminance feature and restore the structural details to obtain the current illuminance enhancement feature map. Based on the current illuminance enhancement feature map, the current illuminance hidden state feature is determined. The color recovery branch is used to fuse the previous color recovery feature map with the previous illuminance latent state feature of the previous iteration to obtain intermediate color features, and to perform color correction and color recovery on the intermediate color features to obtain the current color recovery feature map. Based on the current color recovery feature map, the current color latent state feature is determined. If the current illuminance enhancement feature map, the current color restoration feature map, the current illuminance latent state feature map, and the current color latent state feature map all converge, or reach a preset iteration round, then the current illuminance enhancement feature map and the current color restoration feature map are respectively used as the target illuminance enhancement feature map and the target color restoration feature map; otherwise, the next iteration round is used as the current iteration round, and the above process is iterated.

5. The RAW domain low-light image enhancement method according to claim 4, characterized in that, The target dual-stream recurrent neural network is trained based on the following steps: Determine the low-light denoised images of the RAW domain normal illumination image samples and the corresponding RAW domain low illumination image samples; Based on the content signal map of the low-light denoised image, the initial neural radiation field model is applied to obtain the first illumination enhancement feature map. Based on the initial channel attention module and the initial spatial attention module, the color channel features of the RAW domain low-light image sample are adaptively modulated to obtain the first color restoration feature map. The first illuminance enhancement feature map and the first color restoration feature map are input into the initial two-stream recurrent neural network to obtain the predicted illuminance enhancement feature map and the predicted color restoration feature map output by the initial two-stream recurrent neural network. Based on the predicted illuminance enhancement feature map and the predicted color restoration feature map, the predicted RAW domain enhanced image is determined. Based on the predicted RAW domain enhanced image and the RAW domain normal illumination image sample, the initial neural radiation field model, the initial channel attention module, the initial spatial attention module and the initial dual-stream recurrent neural network are iteratively trained to obtain the target neural radiation field model, the target channel attention module, the target spatial attention module and the target dual-stream recurrent neural network.

6. The RAW domain low-light image enhancement method according to any one of claims 1-5, characterized in that, The step of determining the RAW domain enhanced image based on the target illumination enhancement feature map and the target color restoration feature map includes: The RAW domain enhanced image is subjected to nonlinear processing to obtain the target RGB enhanced image.

7. A RAW domain low-light image enhancement system, characterized in that, include: The image acquisition module is used to acquire a low-light image in the RAW domain and extract the content signal map of the low-light image in the RAW domain; The illuminance enhancement module is used to simulate the projection of light rays from the camera's optical path and perform light ray integration based on the content signal map to determine the initial illuminance enhancement feature map; The color restoration module is used to adaptively modulate the color channel features of the RAW domain low-light image based on the target channel attention module and the target space attention module to obtain an initial color restoration feature map. The fusion update module is used to iteratively fuse and update the initial illuminance enhancement feature map and the initial color recovery feature map based on the target dual-stream recurrent neural network to obtain the target illuminance enhancement feature map and the target color recovery feature map, and to determine the RAW domain enhanced image based on the target illuminance enhancement feature map and the target color recovery feature map; The color restoration module is specifically used for: The color channel features of the RAW domain low-light image are subjected to a two-dimensional discrete Fourier transform to obtain the frequency domain features. Based on the target channel attention module and the target space attention module, the frequency domain features are adaptively modulated in different frequency sub-bands in the frequency domain to obtain modulated frequency domain features. The modulation frequency domain features are subjected to inverse Fourier transform to obtain the initial color recovery feature map; The fusion update module is specifically used for: The target illumination enhancement feature map and the target color restoration feature map are concatenated along the channel dimension to obtain a composite feature tensor; The composite feature tensor is input into a fusion convolutional layer, which compresses and reduces the number of channels of the composite feature tensor to the target number of channels, thereby obtaining the RAW domain enhanced image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the RAW domain low-light image enhancement method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the RAW domain low-light image enhancement method as described in any one of claims 1-6.

Images