A low-illumination image self-adaptive enhancement method based on physical prior guidance and space-time decoupling

Abstract

This invention discloses an adaptive enhancement method for low-light images based on physical prior guidance and spatiotemporal decoupling, belonging to the fields of computer vision and autonomous driving perception technology. The method first converts the original low-light image to YUV space to extract the brightness channel and performs inverse mapping. A pseudo-dehazing physical model is used to inversely solve for the transmittance map as the structural prior matrix. Subsequently, a mask decomposition network decouples the original image features into a reflection component containing local texture and an initial illumination component containing global brightness distribution. This invention constructs a dual-branch parallel enhancement architecture: the first branch uses an enhanced Transformer network for detail restoration and introduces a color histogram matching loss function to ensure color fidelity; the second branch uses an adaptive Gamma U-Net to predict pixel-level gamma parameter matrices for brightness compensation and suppresses overexposure through a regularized loss function with a range penalty term. Finally, the enhanced image is reconstructed based on Retinex theory.

Description

[0001] This invention relates to the fields of computer vision, image processing, and autonomous driving perception technology. Specifically, it relates to a low-light image adaptive enhancement method and system that combines decoupled degraded physical model priors with deep neural network features. It is applicable to complex lighting perception scenarios such as autonomous driving (e.g., nighttime or tunnel scenes), intelligent security, and machine vision. Background Technology

[0002] In machine vision applications such as autonomous driving and all-weather security monitoring, camera sensors are often in environments with extremely low light or drastic changes in lighting. These low-light images are usually accompanied by severe degradation problems such as decreased signal-to-noise ratio, color distortion, extremely low contrast, and local overexposure (such as oncoming headlights).

[0003] Existing low-light image enhancement techniques mainly suffer from the following bottlenecks: Traditional physical / mathematical modeling methods (such as Retinex theory, histogram equalization, etc.): Although they have a certain degree of interpretability, they are prone to amplifying noise and producing artifacts under extremely low light conditions, and their computational complexity is high, making them difficult to adapt to dynamically changing lighting scenarios.

[0004] Purely data-driven deep learning methods (such as end-to-end networks based on GANs or general CNNs): These methods fit massive amounts of data, which improves visual perception, but due to the lack of guidance from physical degradation processes, the networks often produce "illusion" artifacts or color shifts when dealing with extremely dark areas. Furthermore, existing large models based on complex Transformers have a huge number of parameters and high inference latency, failing to meet the millisecond-level inference real-time requirements of edge computing devices such as autonomous driving.

[0005] Therefore, there is an urgent need for a method that can combine the physical laws of atmospheric scattering with the advantages of deep neural network feature extraction, so as to achieve ultra-fast inference while ensuring image enhancement quality and restoring true colors, and effectively improve the accuracy of downstream advanced perception tasks (such as object detection and depth estimation). Summary of the Invention

[0006] To address the problems of noise amplification, color distortion, local overexposure, and high computational latency in existing low-light image enhancement technologies, this invention provides an adaptive low-light image enhancement method based on physical prior guidance and spatiotemporal decoupling.

[0007] The specific technical solution for achieving the objective of this invention is as follows: 1. A low-light image adaptive enhancement method based on physical prior guidance and spatiotemporal decoupling, comprising the following steps: Step 1: Obtain the original low-light image, convert it to the YUV color space to obtain the luminance channel, perform inversion mapping on the luminance channel, and use a pseudo-dehazing physical model to extract the transmittance map of the inverted mapped image as the structure prior matrix.

[0008] Step 2: Input the original low-light image and the structure prior matrix into the mask decomposition network to decouple and separate the original image features into a reflection component containing local texture and an illumination component containing global brightness distribution; Step 3: Construct a dual-branch parallel enhancement architecture. The first branch uses a lightweight deep attention network to perform detail and color restoration on the reflection component, and introduces a color histogram matching loss function during training. The second branch uses a parameter prediction network to generate pixel-level adaptive adjustment parameters for the illumination component and perform nonlinear brightness compensation, and introduces a regularization loss function with a range penalty term during training. Step 4: Perform pixel-by-pixel product fusion of the enhanced reflection component and the compensated illumination component to reconstruct and output the enhanced high-illuminance image.

[0009] The beneficial effects of this invention are: Driven by both physical laws and deep learning: For the first time, the "brightness inversion-pseudo-dehazing" physical prior is used to guide the deep learning network, allowing the network to obtain structural references and avoiding noise amplification and artifact "illusions" in extremely dark areas by pure black box models.

[0010] Solving color distortion and local overexposure: By introducing a unique color histogram matching loss ( The learnable pixel-level Gamma adaptive constraint not only ensures the color fidelity of key targets such as traffic signs, but also adaptively suppresses local overexposed areas.

[0011] Extreme edge inference speed: For automotive edge computing hardware, the LayerNorm in Transformer is replaced with BatchNorm and fused with convolution during the inference stage, with a single frame processing time as low as 0.19 seconds, perfectly adapted to real-time perception.

[0012] Significantly improves downstream sensing accuracy: Experiments show that nighttime images preprocessed by the method of this invention have significantly improved accuracy in target detection (such as YOLOv5) and depth estimation tasks. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the overall process of the adaptive enhancement method of the present invention; Figure 2 This is a schematic diagram illustrating the calculation process of the physical degradation prior extraction module of the present invention; Figure 3This is a schematic diagram of the internal structure and separation process of the Mask Decomposition Network (MDN) of the present invention; Figure 4 This is a structural diagram of the reflection component enhancement Transformer (RET) of the present invention; Figure 5 This is a schematic diagram of the processing of the illumination component adaptive Gamma U-Net (G-UNet) of the present invention; Figure 6 This is a schematic diagram of the image reconstruction and fusion operation based on Retinex theory in this invention. Detailed Implementation

[0014] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, so as to more clearly illustrate the advantages and features of the present invention and enable those skilled in the art to more easily understand the essence of the present invention. The description of specific embodiments is intended to further clarify the scope of protection of the present invention and provide a basis for defining the claims.

[0015] refer to Figures 1-6 A low-light image adaptive enhancement method based on physical prior guidance and spatiotemporal decoupling, combined with Figure 1 The diagram shown illustrates the overall workflow of the adaptive enhancement method. This method mainly includes four core stages: physical degradation prior extraction, mask feature decoupling, dual-branch parallel enhancement, and image fusion reconstruction. The specific steps are as follows: Step 1: Extracting Physical Degradation Priors (Constructing the Physical Driving Layer) Receive raw low-light RGB images captured by the camera : in These represent the height and width of the image, respectively, and 3 represents the RGB channels.

[0016] Furthermore, using a standard color conversion matrix to... Convert from RGB to YUV color space, extract only the luminance channel that represents the intensity of ambient light. : Furthermore, based on the atmospheric physics scattering model, the light attenuation law in low-light images and the light scattering law in daytime haze images exhibit mathematical duality. A pixel inversion operation is performed on the brightness channel to generate a pseudo-haze image. The specific reversal formula is as follows: in, These are the pixel coordinates of the image. Extremely dark areas in the original image in the pseudo-fog map The dense fog region appears as a high-brightness area, while the bright area appears as a fog-free area. This is the basis for the pseudo-fog map. Treating it as a standard haze image containing atmospheric degradation, and then substituting it into a classic atmospheric scattering physics model: in For an ideal fog-free, high-contrast image; Defined in pixels local window centered Calculate the local dark channel features of the pseudo-fog map. Extracting dark channel features reveals the lowest ambient light reflectance in local areas: Using Dark Channel Prior (DCP) or Contrast Enhancement Turbulence Mitigation (CETM) algorithms to treat pseudo-fog maps Process and select The top 0.1% of pixels at medium brightness The mean value in the data is used as the estimated value of ambient light. This allows for the inverse solution of the atmospheric transmittance map. This transmittance map accurately reflects the depth of field and the degree of light attenuation in different areas of the scene. Its values ​​are normalized to... Intervals, serving as global structural prior matrices representing image edges and depth. ; in, To retain some depth of the defogging adjustment factor, The closer the value is to 0, the more it represents a region in the original image that is in a "deep abyss" with extremely little light; the closer the value is to 1, the more it represents a region in the bright light of oncoming headlights, streetlights, etc. This matrix serves as a priori information. It is then input into the next layer of the network, providing clear spatial attention guidance for the next layer.

[0017] Combination Figure 2 The diagram shown illustrates the computational process of the physical degradation prior extraction module. The above steps involve converting the input RGB image to YUV, extracting brightness, and reversing the process to generate a pseudo-haze image. Finally, the global structure prior matrix is ​​calculated by combining the atmospheric scattering model with the dark channel prior. The entire process provides clear physical environment guidance and spatial attention guidance for the next layer of the network.

[0018] Step 2: Mask Decomposition Network (MDN) (Feature Decoupling Layer) Original low-light RGB image Compared with the structural prior matrix obtained in step 1 By concatenating along the channel dimension, a fused input feature tensor is constructed. .at this time It acts as a strong supervisory spatial attention weight, guiding the network to focus on severely degraded areas.

[0019] Furthermore, combined Figure 3 The diagram shown illustrates the internal structure and separation process of a mask decomposition network (MDN). The input is fed into a lightweight encoder-decoder structure mask decomposition network. After passing through two consecutive filter layers, the size is... A convolutional layer with a stride of 1, supplemented by the LeakyReLU activation function, extracts shallow representations containing physical edge information. : Furthermore, at the end of the network, it is divided into two parallel lines. Convolutional prediction head, supplemented by a Sigmoid activation function Map it back to the corresponding physical space. Output 1 is the reflection component. : It represents the inherent color and high-frequency texture details of an object, unaffected by ambient light. Output 2 is the initial illumination component. : Characterizes the low-frequency spatial distribution of ambient light.

[0020] in, This represents the convolution operation. and These represent the weights and biases of the corresponding convolutional layers.

[0021] Step 3: Dual-branch parallel enhanced architecture; Branch 1: Enhanced Transformer (RET) for the reflection graph R Combination Figure 4 The diagram shown illustrates the Transformer structure that enhances the reflection component, which in turn enhances the reflection image. The data is divided into non-overlapping local blocks and input into a network containing four Swing Transformer components. A global window multi-head self-attention mechanism is used to repair the lost continuous texture in extremely dark areas. The calculation formula is as follows: in, These are the query matrix, key matrix, and value matrix, respectively; d is the channel dimension; and B is the relative position encoding bias matrix. Furthermore, to meet the real-time requirements of edge devices, BatchNorm is used to replace the traditional LayerNorm, and the normalization parameters are incorporated into the convolution weights during the inference phase. The feature labeling formula during training is as follows: During the inference phase, the parameters of BatchNorm are fused into the weights W and biases b of the previous convolutional layer using reparameterization techniques. The new equivalent convolution parameter calculation formula is as follows: After the above mathematical transformation, when the network performs forward propagation on the edge device, the computation time of the normalization layer is completely "erased". Furthermore, to prevent severe color distortion after nighttime image enhancement, a color histogram matching loss function is introduced into the training loss of branch one. Let the enhanced reflection map be... The reference image is as follows For RGB channels , The difference in probability distribution across the RGB three channels before and after augmentation using KL divergence constraints is given by the following formula: Where C represents the color channel of the image. The number of intervals (buckets) into which the color histogram is divided. This represents the normalized probability distribution of the intensity range of the i-th pixel in the color channel C of the normal lighting reference image. This represents the normalized color histogram value of the enhanced image output by the network in the i-th interval of channel C.

[0022] Branch 2: Combination Figure 5 The diagram shown illustrates the adaptive Gamma U-Net processing procedure for illumination components. The adaptive Gamma U-Net (G-UNet) constructs a U-Net with an adaptive parameter layer to process the initial illumination map. This network embeds a learnable Gamma layer after the upsampling operation of the decoder. Instead of directly outputting pixel brightness values, the network outputs a gamma parameter matrix with the same spatial resolution as the image at the end of the decoder. Each element in the matrix represents the optimal gamma adjustment coefficient for the corresponding pixel. .

[0023] Furthermore, based on the network output , for illumination diagram Perform pixel-by-pixel nonlinear exponential mapping compensation to obtain the enhanced illumination map. The pixel-level brightness compensation formula is: in, The initial lighting map obtained by decoupling is in coordinates The normalized luminance value at that location, its range is .

[0024] Furthermore, if not Imposing constraints makes the network prone to outputting extreme values, leading to tearing and overexposure in the highlight areas of the image. To prevent the network from overstretching the brightness and causing local overexposure, a weighted squared bias loss function with smoothing range constraints is introduced during training. : in, The desired gamma target smoothing center value is typically set at... Used to brighten dark areas; This represents the total number of pixels in the image that are involved in the calculation. The gamma parameter of the i-th pixel predicted by the network; The slope parameter is used to control the strictness of the Sigmoid constraint; The weighting coefficient for the squared deviation term of the center offset; The weighting coefficient for the logical boundary penalty term; and The theoretical maximum and minimum values ​​of the gamma parameters are defined for the network, when the network predicts... When the value exceeds this range, the Sigmoid term in the latter half of the formula increases rapidly, resulting in a huge loss penalty, which forces the network to reduce the loss through backpropagation. Converging within a safe range, this formula establishes the "adaptive boundary" for the network when adjusting brightness.

[0025] Step 4: Re-integrate the enhanced independent features into a high-quality image that conforms to the logic of human vision and machine vision.

[0026] Combination Figure 6 The diagram shown illustrates image reconstruction and fusion operations based on Retinex theory. The final image is the product of the object's reflectivity and the ambient illumination. The enhanced reflectance map of branch one is obtained. Adaptive illumination map of branch 2 output Performing element-wise multiplication operation pixel by pixel yields the final high-illumination image. The reconstruction formula is: At this time, it was obtained It has effectively eliminated noise in dark areas, restored true colors, and perfectly suppressed overexposed areas such as streetlights and car lights.

[0027] Furthermore, the output The data is directly input into the subsequent machine vision sensing module. Because the physical prior in step 1 greatly preserves the edge fidelity of the depth, and step 3... It maintained the consistency of color semantics. This method can significantly improve the feature response values ​​of pre-trained YOLO series object detectors or binocular depth matching networks (such as HitNet) in low-light environments. Based on the above theoretical mechanism and feature preservation advantages, the images enhanced by this method can significantly reduce the false negative rate of pedestrians in tunnels and nighttime environments without additional lighting equipment, thus comprehensively improving the robustness of autonomous driving sensing systems.

Claims

1. A low-light image adaptive enhancement method based on physical prior guidance and spatiotemporal decoupling, characterized in that, Includes the following steps: Step 1: Obtain the original low-light image, convert it to the YUV color space to obtain the luminance channel and perform inversion mapping. Use the pseudo-dehazing physical model to extract the transmittance map of the inverted mapped image as the structure prior matrix. Step 2: Input the original low-light image and the structure prior matrix into the mask decomposition network to decouple the original image features into a reflection component containing local texture and an initial illumination component containing global brightness distribution; Step 3: Construct a dual-branch parallel enhancement architecture: The first branch uses an enhanced Transformer network to perform detail and color restoration on the reflection components, outputting an enhanced reflection map, and introducing a color histogram matching loss function during training; The second branch uses an adaptive Gamma U-Net to predict pixel-level gamma parameter matrices for the initial illumination components and perform nonlinear brightness compensation, outputting an enhanced illumination map, and introducing a regularized loss function with a range penalty term during training; Step 4: Perform pixel-by-pixel product fusion of the enhanced reflectance map and the enhanced illumination map to reconstruct and output the enhanced high-illuminance image.

2. The low-light image adaptive enhancement method based on physical prior guidance and spatiotemporal decoupling according to claim 1, characterized in that, Step 1, extracting the structural prior matrix, specifically involves: Step 1-1: Receive the raw low-light RGB image captured by the camera. : Using the standard color conversion matrix Convert from RGB to YUV color space, extract only the luminance channel that represents the intensity of ambient light. : Steps 1-2: Perform pixel inversion on the brightness channel to generate a pseudo-fog map. The specific reversal formula is as follows: in, These are the pixel coordinates of the image; Steps 1-3: Define in pixels local window centered Calculate the local dark channel features of the pseudo-fog map. : Select The top 0.1% of pixels at medium brightness The mean value in the range is used as the estimated value A for ambient light; Steps 1-4: The pseudo-fog map Treating it as a standard haze image containing atmospheric degradation, and then substituting it into a classic atmospheric scattering physics model: Ambient light is estimated using Dark Channel Prior (DCP) or Contrast Enhanced Turbulence Mitigation (CETM) algorithms. This allows for the inverse solution of the atmospheric transmittance map. ; Steps 1-5: Normalize the transmittance map values ​​to The interval is used to obtain the global structural prior matrix representing the image edges and depth. : in, To retain some depth of the dehazing adjustment factor, 。 3. The low-light image adaptive enhancement method based on physical prior guidance and spatiotemporal decoupling according to claim 1, characterized in that, Step 2 achieves feature decoupling and separation, specifically as follows: Step 2-1: Convert the original low-light RGB image With the aforementioned structural prior matrix By concatenating along the channel dimension, a fused input feature tensor is constructed. ; Step 2-2, Input mask decomposition network; through two consecutive layers Convolutional layers, along with the LeakyReLu activation function, extract shallow representations containing physical edge information. : Steps 2-3: At the end of the network, use two parallel... Convolutional prediction head and Sigmoid activation function Map it back to the corresponding physical space. Output 1 is the reflection component. : Output 2 is the initial illumination component. : in, This represents the convolution operation. and These represent the weights and biases of the corresponding convolutional layers.

4. The low-light image adaptive enhancement method based on physical prior guidance and spatiotemporal decoupling according to claim 1, characterized in that, Step 3 enhances the reflection component obtained in the previous step, specifically by: Step 3-1-1: Extract multi-head self-attention for the feature block of the reflection component. The calculation formula is as follows: in, These are the query matrix, key matrix, and value matrix, respectively; d is the channel dimension; and B is the relative position encoding bias matrix. Step 3-1-2: Replace the traditional LayerNorm normalization with BatchNorm normalization. The feature labeling formula during training is as follows; Step 3-1-3: Output Enhanced Reflection Map During the training phase, under normal lighting reference diagram Compared with the enhanced reflection map Introducing a color histogram matching loss function based on KL divergence : Where C represents the color channel of the image. For the number of intervals, This represents the normalized probability distribution of the intensity range of the i-th pixel in the color channel C of the normal lighting reference image. This represents the normalized color histogram value of the enhanced image output by the network in the i-th interval of channel C.

5. The low-light image adaptive enhancement method based on physical prior guidance and spatiotemporal decoupling according to claim 1, characterized in that, Step 3 enhances the illumination component obtained in the previous step, specifically by: Step 3-2-1: Obtain the initial illumination map The input is an adaptive Gamma U-Net (G-UNet), which outputs a gamma parameter matrix with the same spatial resolution as the image at the end of the decoder. Each element in the matrix represents the optimal gamma adjustment coefficient for the corresponding pixel. ; Step 3-2-2: Based on the prediction , for illumination diagram Perform pixel-by-pixel nonlinear exponential compensation to obtain the enhanced illumination map. The formula is: Step 3-2-3: Introduce a weighted squared deviation loss function with smoothing range constraints during training. Prevent overexposure: in, The desired smooth center value; This represents the total number of pixels. To predict the gamma parameters; This is the slope parameter; and These are the weighting coefficients; and These are the theoretical maximum and minimum values ​​of the parameters.

6. The low-light image adaptive enhancement method based on physical prior guidance and spatiotemporal decoupling according to claim 1, characterized in that, Step 4 reconstructs the enhanced reflection and illuminance components, specifically as follows: Step 4-1: Obtain the enhanced reflection map of branch 1 output. Adaptive illumination map of branch 2 output Performing element-wise multiplication operation pixel by pixel yields the final high-illumination image. The reconstruction formula is: At this time, it was obtained It has effectively eliminated noise in dark areas, restored true colors, and perfectly suppressed local overexposure areas such as streetlights and car lights; Step 4-2, output It is directly input into the subsequent machine vision sensing module.

Images