Low-light image enhancement method based on dark channel prior and multi-domain feature fusion

Abstract

The invention discloses an automatic driving low-light image enhancement method based on dark channel prior and multi-domain feature fusion, and the method comprises the steps: firstly, rapidly estimating an initial illumination component of a low-light image through an improved dark channel prior model which carries out the self-adaptive optimization of a large-area sky, a vehicle lamp and other regions in an automatic driving scene, and secondly, carrying out the self-adaptive optimization of the initial illumination component of the low-light image; a multi-domain feature fusion network is constructed, the network carries out feature extraction and enhancement on an image in a spatial domain and a frequency domain at the same time, a spatial domain sub-network focuses on local detail recovery and noise suppression, a frequency domain sub-network globally processes the image through Fourier transform and effectively separates noise and useful information, and finally, the image is extracted and enhanced. And designing an adaptive fusion module, carrying out weighted fusion on physical prior information obtained by dark channel prior and deep features extracted by the multi-domain deep learning network, and reconstructing an enhanced image with high quality and high dynamic range.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a low-light image enhancement method based on dark channel prior and multi-domain feature fusion. Background Technology

[0002] The environmental perception module of autonomous driving technology relies heavily on visual information captured by onboard cameras. However, under low-light conditions such as nighttime, tunnels, dawn, and dusk, the acquired images suffer from problems such as low brightness, poor contrast, overexposure, significant noise, blurred details, and color distortion. These issues severely limit the performance of subsequent visual tasks such as object detection and semantic segmentation, posing a threat to driving safety.

[0003] Existing low-light image enhancement methods mainly include those based on traditional image processing and those based on deep learning. Traditional image processing methods, such as histogram equalization, homomorphic filtering, and methods based on Retinex theory (e.g., MSRCR), while computationally efficient, often rely on manually designed priors and are prone to problems like halos, color distortion, and over-enhancement when dealing with complex natural scenes. Deep learning-based methods have made significant progress by training end-to-end neural networks (e.g., CNN, GAN) to learn mapping relationships from large amounts of data. However, these methods have the following shortcomings: (1) It heavily relies on the quality and quantity of training data, has limited generalization ability, and may fail in unseen harsh low-light scenes; (2) Most of them are "black box" models, lacking interpretable physical prior guidance, which may lead to processing results that do not conform to physical laws; (3) While enhancing image brightness, it is easy to amplify the inherent noise in the image and reduce signal-to-noise ratio; (4) Most methods operate only in the spatial domain, making it difficult to globally separate the structural information and noise / degradation factors of an image.

[0004] Therefore, there is an urgent need in this field for a low-light image enhancement technology that can combine the advantages of physical priors and data-driven approaches, and can process collaboratively across multiple domains, thereby improving image brightness and contrast while effectively preserving details, suppressing noise, and ensuring color fidelity. Summary of the Invention

[0005] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0006] In view of the problems existing in the low-light image enhancement methods based on dark channel prior and multi-domain feature fusion, the present invention is proposed.

[0007] Therefore, the purpose of this invention is to provide a low-light image enhancement method based on the fusion of dark channel prior and multi-domain features. This method improves the environmental perception robustness of autonomous driving systems under adverse lighting conditions by fusing improved dark channel prior with multi-domain (spatial domain, frequency domain) deep learning features.

[0008] To address the aforementioned technical problems, this invention provides the following technical solution: a low-light image enhancement method based on dark channel prior and multi-domain feature fusion, comprising the following steps: Step 1: Input autonomous driving scene images under low light conditions and perform standardized preprocessing; Step 2: Enhance the input image using an improved dark channel prior model, utilizing the dark channel image I. dark (x) and the target dark channel J dark (x), the illumination transmission parameter t0(x) is estimated through the neural network U1, and then combined with the road feature map M0(x) and the preliminary enhanced image J0(x) to generate the enhanced image J1(x); Step 3: Input the preprocessed low-light image into the spatial domain enhancement subnetwork and the frequency domain enhancement subnetwork in parallel for feature extraction and enhancement, and obtain the spatial domain enhancement feature map and the frequency domain enhancement feature map respectively; Step 4: Using the enhanced image J1(x) obtained in Step 2 as a physical prior, compare it with the spatial domain enhanced feature map and frequency domain enhanced feature map obtained in Step 3. Figure 1 The image is then input into the adaptive fusion module, where it undergoes feature weighted fusion through an attention mechanism to generate the final high dynamic range enhanced image. Step 5: Output the enhanced high dynamic range image for subsequent autonomous driving vision tasks.

[0009] As a preferred embodiment of the low-light image enhancement method based on dark channel prior and multi-domain feature fusion described in this invention, wherein: in step 2, a low-quality nighttime road image I(x) is input, and its dark channel image I is calculated. dark (x), the formula is as follows: ; Where Ω(x) represents a local window centered at pixel x, I c Represents the color channels of an image; Using dark channel image I dark (x) and the target dark channel J dark (x), the illumination transmission parameter t0(x) is estimated using neural network U1: ; in, It is the estimated target dark channel generated by network U0; Using the illumination transmission parameter t0(x) and the original image I(x), a preliminary enhanced image J0(x) is generated through the enhancement network U3: ; Simultaneously, the road scene parsing network U2 is used to extract road feature maps M0(x) from the original image, including lane lines, traffic signs, and pedestrian areas: ; By combining the road feature map M0(x) and the initial enhanced image J0(x), the enhancement result is further optimized using the residual network U4: ; Where J1(x) is the final enhanced image.

[0010] As a preferred embodiment of the low-light image enhancement method based on dark channel prior and multi-domain feature fusion described in this invention, wherein: in step 3, the spatial domain enhancement subnetwork is a convolutional neural network with an encoder-decoder structure, used to extract and enhance local detail features of the image and perform noise suppression; the frequency domain enhancement subnetwork first transforms the image to the frequency domain through fast Fourier transform, processes the frequency domain coefficients using frequency domain filtering or feature enhancement modules, and then inversely transforms it back to the spatial domain to obtain a frequency domain enhanced feature map, focusing on global noise suppression and structural information enhancement.

[0011] As a preferred embodiment of the low-light image enhancement method based on dark channel prior and multi-domain feature fusion described in this invention, step 3 specifically includes: 1) Spatial domain feature extraction: The input low-light image is processed through a spatial domain branch. This branch adopts a channel-based processing strategy. One part uses depthwise separable convolution to extract local features, and the other part uses a self-attention mechanism to obtain global features. The intermediate features are downsampled by average pooling and then restored to their original size. 2) Frequency domain feature extraction: The spatial domain features are transformed into the frequency domain using Fast Fourier Transform. ; Where X(u,v) represents the frequency domain feature and x(m,n) represents the spatial domain feature; 3) Frequency domain component separation and processing: The frequency domain features are decomposed into amplitude and phase components: Amplitude component calculation: Amplitude component calculation: Where R(u,v) and I(u,v) represent the real and imaginary parts of the frequency domain features, respectively. Then, the amplitude and phase components are transformed by 1×1 convolution to preserve the original structural information. 4) Multi-domain feature fusion: The processed amplitude and phase components are then transformed back into the spatial domain using an inverse fast Fourier transform: ; Where X′(u,v) is the processed frequency domain feature, x′(m,n) is the converted spatial domain feature, M is the output feature map processed from the spatial domain branch and finally converted back to the spatial form, and N is the output feature map processed from the frequency domain branch and finally converted back to the spatial form.

[0012] As a preferred embodiment of the low-light image enhancement method based on dark channel prior and multi-domain feature fusion described in this invention, step 4 specifically includes: 1) The illumination map J1(x) obtained in step 2 is convolved and adjusted, and then used as the guiding signal; 2) Concatenate the spatial domain enhanced feature map and the frequency domain enhanced feature map; 3) The channel attention submodule in the fusion module calculates the importance weight of each feature channel, and the spatial attention submodule (J1(x) guided) calculates the importance weight of different spatial locations in the image; 4) The stitched feature maps are weighted and fused, and the final enhanced image is reconstructed through a convolutional layer.

[0013] As a preferred embodiment of the low-light image enhancement method based on dark channel prior and multi-domain feature fusion described in this invention, step 5 further includes training and inference: the entire network is trained end-to-end using a combined loss function, and during online inference, the low-light image is input into the network, and the enhancement result is output in real time.

[0014] As a preferred embodiment of the low-light image enhancement method based on dark channel prior and multi-domain feature fusion described in this invention, the combined loss function includes L1 Loss, Perceptual Loss, and GAN Loss.

[0015] The beneficial effects of this invention are: This invention provides a stable initial estimate that conforms to physical laws through an improved dark channel prior, guiding the deep learning model to generate more reasonable augmentation results and improving the model's generalization ability and interpretability.

[0016] Spatial domain networks are responsible for local detail restoration, while frequency domain networks are responsible for global noise suppression and structure enhancement. The two complement each other, overcoming the limitations of single-domain processing and achieving a more comprehensive improvement in image quality.

[0017] The improved dark channel prior effectively handles the bright areas in automotive scenes, avoiding common exposure and color distortion problems.

[0018] It can significantly improve the overall brightness and contrast of images while maintaining edge and texture details, effectively suppressing noise, ensuring color authenticity, and providing more reliable input for downstream perception tasks. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This invention relates to a low-light image enhancement model based on dark channel prior constraints and a low-light image enhancement model based on a combination of frequency and spatial domains.

[0020] Figure 2 This is a histogram illustrating the overall performance of the low-light image enhancement method based on dark channel prior and multi-domain feature fusion of the present invention on different datasets compared with other state-of-the-art methods in publicly available data.

[0021] Figure 3 This diagram illustrates the results at different stages of the low-light image enhancement method based on dark channel prior and multi-domain feature fusion in this invention when dealing with nighttime road problems. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Secondly, the present invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of the present invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not according to the usual scale. Furthermore, the schematic diagrams are merely examples and should not limit the scope of protection of the present invention. In addition, actual fabrication should include three-dimensional spatial dimensions of length, width, and depth.

[0026] Reference Figures 1-3 This paper presents a low-light image enhancement method based on dark channel prior and multi-domain feature fusion, including the following steps: Step 1: Input autonomous driving scene images under low light conditions and perform standardized preprocessing; Step 2: Enhance the input image using an improved dark channel prior model, utilizing the dark channel image I. dark (x) and the target dark channel J dark (x), the illumination transmission parameter t0(x) is estimated through the neural network U1, and then combined with the road feature map M0(x) and the preliminary enhanced image J0(x) to generate the enhanced image J1(x); Step 3: Input the preprocessed low-light image into the spatial domain enhancement subnetwork and the frequency domain enhancement subnetwork in parallel for feature extraction and enhancement, and obtain the spatial domain enhancement feature map and the frequency domain enhancement feature map respectively; Step 4: Using the enhanced image J1(x) obtained in Step 2 as a physical prior, compare it with the spatial domain enhanced feature map and frequency domain enhanced feature map obtained in Step 3. Figure 1 The image is then input into the adaptive fusion module, where it undergoes feature weighted fusion through an attention mechanism to generate the final high dynamic range enhanced image. Step 5: Output the enhanced high dynamic range image for subsequent autonomous driving vision tasks.

[0027] Specifically, in step 2, a low-quality nighttime road image I(x) is input, and its dark channel image I is calculated. dark (x), the formula is as follows: ; Where Ω(x) represents a local window centered at pixel x, I c Represents the color channels of an image; Using dark channel image I dark (x) and the target dark channel Jdark (x), the illumination transmission parameter t0(x) is estimated using neural network U1: ; in, It is the estimated target dark channel generated by network U0; Using the illumination transmission parameter t0(x) and the original image I(x), a preliminary enhanced image J0(x) is generated through the enhancement network U3: ; Simultaneously, the road scene parsing network U2 is used to extract road feature maps M0(x) from the original image, including lane lines, traffic signs, and pedestrian areas: ; By combining the road feature map M0(x) and the initial enhanced image J0(x), the enhancement result is further optimized using the residual network U4: ; Where J1(x) is the final enhanced image.

[0028] In step 3, the spatial domain enhancement subnetwork is a convolutional neural network with an encoder-decoder structure, used to extract and enhance local detail features of the image and suppress noise. The frequency domain enhancement subnetwork first transforms the image to the frequency domain through a fast Fourier transform, processes the frequency domain coefficients using frequency domain filtering or feature enhancement modules, and then inversely transforms it back to the spatial domain to obtain a frequency domain enhanced feature map, focusing on global noise suppression and structural information enhancement.

[0029] Specifically, step 3 includes: 1) Spatial domain feature extraction: The input low-light image is processed through a spatial domain branch. This branch adopts a channel-based processing strategy. One part uses depthwise separable convolution to extract local features, and the other part uses a self-attention mechanism to obtain global features. The intermediate features are downsampled by average pooling and then restored to their original size. 2) Frequency domain feature extraction: The spatial domain features are transformed into the frequency domain using Fast Fourier Transform. ; Where X(u,v) represents the frequency domain feature and x(m,n) represents the spatial domain feature; 3) Frequency domain component separation and processing: The frequency domain features are decomposed into amplitude and phase components: Amplitude component calculation: Amplitude component calculation: Where R(u,v) and I(u,v) represent the real and imaginary parts of the frequency domain features, respectively. Then, the amplitude and phase components are transformed by 1×1 convolution to preserve the original structural information. 4) Multi-domain feature fusion: The processed amplitude and phase components are then transformed back into the spatial domain using an inverse fast Fourier transform: ; Where X′(u,v) is the processed frequency domain feature, x′(m,n) is the converted spatial domain feature, M is the output feature map processed from the spatial domain branch and finally converted back to the spatial form, and N is the output feature map processed from the frequency domain branch and finally converted back to the spatial form.

[0030] Specifically, step 4 includes: 1) The illumination map J1(x) obtained in step 2 is convolved and adjusted, and then used as the guiding signal; 2) Concatenate the spatial domain enhanced feature map and the frequency domain enhanced feature map; 3) The channel attention submodule in the fusion module calculates the importance weight of each feature channel, and the spatial attention submodule (J1(x) guided) calculates the importance weight of different spatial locations in the image; 4) The stitched feature maps are weighted and fused, and the final enhanced image is reconstructed through a convolutional layer.

[0031] Furthermore, step 5 also includes training and inference: the entire network is trained end-to-end using a combined loss function, and during online inference, a low-light image is input into the network, and the enhancement result is output in real time; the combined loss function includes L1 Loss, Perceptual Loss, and GAN Loss.

[0032] This invention provides a stable initial estimate that conforms to physical laws through an improved dark channel prior, guiding the deep learning model to generate more reasonable augmentation results and improving the model's generalization ability and interpretability.

[0033] Spatial domain networks are responsible for local detail restoration, while frequency domain networks are responsible for global noise suppression and structure enhancement. The two complement each other, overcoming the limitations of single-domain processing and achieving a more comprehensive improvement in image quality.

[0034] The improved dark channel prior effectively handles the bright areas in automotive scenes, avoiding common exposure and color distortion problems.

[0035] It can significantly improve the overall brightness and contrast of images while maintaining edge and texture details, effectively suppressing noise, ensuring color authenticity, and providing more reliable input for downstream perception tasks.

[0036] Specific implementation examples are shown in the table below:

[0037] Experiments show that the method of the present invention outperforms the comparative methods in terms of subjective visual quality and objective indicators (such as PSNR, SSIM, LOE), and can effectively improve the performance of the target detection model in low light environment.

[0038] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A low-light image enhancement method based on dark channel prior and multi-domain feature fusion, characterized in that, Includes the following steps: Step 1: Input autonomous driving scene images under low light conditions and perform standardized preprocessing; Step 2: Enhance the input image using an improved dark channel prior model, utilizing the dark channel image I. dark (x) and the target dark channel J dark (x), the illumination transmission parameter t0(x) is estimated through the neural network U1, and then combined with the road feature map M0(x) and the preliminary enhanced image J0(x) to generate the enhanced image J1(x); Step 3: Input the preprocessed low-light image into the spatial domain enhancement subnetwork and the frequency domain enhancement subnetwork in parallel for feature extraction and enhancement, and obtain the spatial domain enhancement feature map and the frequency domain enhancement feature map respectively; Step 4: The enhanced image J1(x) obtained in Step 2 is used as a physical prior guide and is input into the adaptive fusion module along with the spatial domain enhanced feature map and frequency domain enhanced feature map obtained in Step 3. Feature weighted fusion is performed through the attention mechanism to generate the final high dynamic range enhanced image. Step 5: Output the enhanced high dynamic range image for subsequent autonomous driving vision tasks.

2. The low-light image enhancement method based on dark channel prior and multi-domain feature fusion according to claim 1, characterized in that: In step 2, a low-quality nighttime road image I(x) is input, and its dark channel image I is calculated. dark (x), the formula is as follows: ； Where Ω(x) represents a local window centered at pixel x, I c Represents the color channels of an image; Using dark channel image I dark (x) and the target dark channel J dark (x), the illumination transmission parameter t0(x) is estimated using neural network U1: ； in, It is the estimated target dark channel generated by network U0; Using the illumination transmission parameter t0(x) and the original image I(x), a preliminary enhanced image J0(x) is generated through the enhancement network U3: ； Simultaneously, the road scene parsing network U2 is used to extract road feature maps M0(x) from the original image, including lane lines, traffic signs, and pedestrian areas: ； By combining the road feature map M0(x) and the initial enhanced image J0(x), the enhancement result is further optimized using the residual network U4: ； Where J1(x) is the final enhanced image.

3. The low-light image enhancement method based on dark channel prior and multi-domain feature fusion according to claim 2, characterized in that: In step 3, the spatial domain enhancement subnetwork is a convolutional neural network with an encoder-decoder structure, used to extract and enhance local detail features of the image and suppress noise. The frequency domain enhancement subnetwork first transforms the image to the frequency domain through a fast Fourier transform, processes the frequency domain coefficients using frequency domain filtering or feature enhancement modules, and then inversely transforms it back to the spatial domain to obtain a frequency domain enhanced feature map, focusing on global noise suppression and structural information enhancement.

4. The low-light image enhancement method based on dark channel prior and multi-domain feature fusion according to claim 1, characterized in that: Step 3 specifically includes: 1) Spatial domain feature extraction: The input low-light image is processed through a spatial domain branch. This branch adopts a channel-based processing strategy. One part uses depthwise separable convolution to extract local features, and the other part uses a self-attention mechanism to obtain global features. The intermediate features are downsampled by average pooling and then restored to their original size. 2) Frequency domain feature extraction: The spatial domain features are transformed into the frequency domain using Fast Fourier Transform. ； Where X(u,v) represents the frequency domain feature and x(m,n) represents the spatial domain feature; 3) Frequency domain component separation and processing: The frequency domain features are decomposed into amplitude and phase components: Amplitude component calculation: ; Amplitude component calculation: ; Where R(u,v) and I(u,v) represent the real and imaginary parts of the frequency domain features, respectively. Then, the amplitude and phase components are transformed by 1×1 convolution to preserve the original structural information. 4) Multi-domain feature fusion: The processed amplitude and phase components are then transformed back into the spatial domain using an inverse fast Fourier transform: ； Where X′(u,v) is the processed frequency domain feature, x′(m,n) is the converted spatial domain feature, M is the output feature map processed from the spatial domain branch and finally converted back to the spatial form, and N is the output feature map processed from the frequency domain branch and finally converted back to the spatial form.

5. The low-light image enhancement method based on dark channel prior and multi-domain feature fusion according to claim 1, characterized in that: Step 4 specifically includes: 1) The illumination map J1(x) obtained in step 2 is convolved and adjusted, and then used as the guiding signal; 2) Concatenate the spatial domain enhanced feature map and the frequency domain enhanced feature map; 3) The channel attention submodule in the fusion module calculates the importance weight of each feature channel, and the spatial attention submodule (J1(x) guided) calculates the importance weight of different spatial locations in the image; 4) The stitched feature maps are weighted and fused, and the final enhanced image is reconstructed through a convolutional layer.

6. The low-light image enhancement method based on dark channel prior and multi-domain feature fusion according to claim 1, characterized in that: Step 5 also includes training and inference: the entire network is trained end-to-end using a combined loss function, and during online inference, low-light images are input into the network, and the enhancement results are output in real time.

7. The low-light image enhancement method based on dark channel prior and multi-domain feature fusion according to claim 6, characterized in that: The combined loss functions include L1 Loss, Perceptual Loss, and GAN Loss.

Images