A gradient domain infrared structure constraint and dynamic gate fusion night dark light image enhancement network implementation method

Abstract

The present application belongs to the technical field of computer vision and low-light image processing, and particularly relates to a night low-light image enhancement network implementation method based on gradient domain infrared structure constraint and dynamic gating fusion, comprising the following steps: S1, inputting a pair of night low-light visible light image and infrared image, and constructing a Retinex enhancement framework introducing infrared gradient domain structure constraint; S2, performing illumination decoupling processing on the visible light image and the infrared image respectively through an illumination feature extraction module; S3, adopting a feature-guided multi-head self-attention mechanism; S4, performing dynamic weighted fusion on the aligned dual-mode illumination features; S5, constructing a U-shaped Transformer enhancement network guided by the fused features; the present application introduces infrared structure information as prior constraint of the reflection component into Retinex decomposition from the theoretical level by constructing a Retinex enhancement framework introducing infrared gradient domain structure constraint, and establishes a cross-modal structure consistency relationship in the gradient domain, thereby avoiding the intensity inversion and artifact problems caused by direct fusion in the pixel domain.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and low-light image processing technology, specifically relating to a method for implementing a nighttime low-light image enhancement network that combines gradient domain infrared structural constraints and dynamic gating. Background Technology

[0002] Low-light image enhancement at night is a fundamental research task in the field of computer vision, aiming to restore the brightness, contrast, and structural details of images under insufficient lighting conditions. It has significant application value in fields such as intelligent surveillance, autonomous driving, and nighttime security. Existing low-light enhancement techniques are mainly divided into three categories: methods based on traditional image processing, methods based on single-modal deep learning, and methods based on multimodal fusion.

[0003] Traditional methods, represented by histogram equalization and Retinex theory, enhance pixel distribution by explicitly adjusting the pixel distribution or decomposing the illumination and reflection components. However, in extremely low light conditions, they are prone to problems such as noise amplification, halo artifacts, and color distortion, making them difficult to cope with complex nighttime environments.

[0004] Single-modal methods based on deep learning learn degradation maps from data through convolutional networks or Transformer models, which significantly improves the enhancement effect. However, under extremely dark conditions at night, visible light images themselves suffer from severe structural degradation, texture loss, and reduced signal-to-noise ratio. The information from a single modality is insufficient to support accurate structural restoration, and problems such as local over-enhancement, texture blurring, and artifact diffusion are prone to occur.

[0005] To address these issues, current research has begun to incorporate infrared images as auxiliary information, improving low-light enhancement performance through cross-modal fusion. Infrared images rely on target thermal radiation imaging, are unaffected by ambient light, and provide stable structural contour information. However, existing multimodal fusion methods still have significant drawbacks: First, most methods employ direct feature stitching, static weighting, or unified attention mechanisms for fusion, assuming consistent bimodal structural representations and ignoring the response differences, intensity inversions, and structural misalignments between infrared and visible light in real-world scenarios, easily introducing fusion artifacts and error propagation. Second, fusion strategies lack regional adaptability, failing to dynamically adjust the fusion ratio based on the degradation level and modal reliability of different regions, resulting in insufficient structural recovery in dark areas and loss of detail in bright areas. Third, the enhancement stage often employs a uniform enhancement strategy, making it difficult to simultaneously address the differentiated recovery needs of large-scale low-response areas and localized high-brightness areas, leading to insufficient enhancement stability.

[0006] In summary, existing low-light enhancement technologies for nighttime still suffer from problems such as insufficient structural restoration capabilities, easy artifact generation during modal fusion, and poor adaptability of enhancement strategies in extremely dark scenes, and cannot meet the high-quality visual restoration requirements in complex nighttime environments. Summary of the Invention

[0007] The purpose of this invention is to provide a method for implementing a nighttime low-light image enhancement network that combines gradient domain infrared structural constraints with dynamic gating. This method aims to solve the technical problems of severe structural degradation of visible light images in extremely low-light nighttime scenes, structural mismatch and artifacts in cross-modal fusion, and poor adaptability of enhancement strategies, thereby achieving robust and high-quality image enhancement in extremely dark environments.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a method for implementing a nighttime low-light image enhancement network that fuses gradient domain infrared structural constraints and dynamic gating, comprising the following steps: S1. Input paired nighttime low-light visible light images and infrared images, construct a Retinex enhancement framework that introduces infrared gradient domain structure constraints, and model low-light enhancement as a constrained optimization problem with infrared structure priors. S2. The illumination feature extraction module performs illumination decoupling processing on the visible light image and the infrared image respectively, and outputs the pixel-by-pixel illumination map and high-dimensional illumination features; wherein the visible light branch multiplies the input low light image with the corresponding illumination map element by element to obtain the preliminary enhanced image; S3. A feature-guided multi-head self-attention mechanism is adopted to perform cross-modal spatial alignment of dual-modal illumination features, establish the structural correspondence between infrared and visible light features, and alleviate the problems of intermodal structural inconsistency and intensity reversal. S4. Through the adaptive illumination fusion module, spatially adaptive fusion weights are generated based on the gating mechanism. The aligned dual-modal illumination features are dynamically weighted and fused to obtain the fused illumination features. S5. Construct a U-shaped Transformer enhancement network guided by fusion features. Using the preliminary enhanced image as input and the fused illumination features as global guidance information, the network completes multi-scale feature modeling and gated jump connection fusion through an encoder-decoder structure, and finally outputs the enhanced visible light image. S6. Construct a composite loss function that includes reconstruction loss, illumination smoothing loss, and gradient domain structure consistency loss, and perform end-to-end supervised training on the entire augmentation network.

[0009] As a preferred embodiment of the nighttime low-light image enhancement network method of the present invention, which integrates gradient domain infrared structural constraints and dynamic gating, the infrared gradient domain structural constraint in S1 specifically involves establishing a structural consistency relationship between the visible light image reflection component and the infrared image in the gradient domain, expressed as: ; in, The gradient of the visible light reflectance component. The gradient of the infrared image. It is a spatially adaptive weight used to characterize the reliability of infrared structural information in different regions; The corresponding constrained optimization problem expression is: ; in, To observe low-light visible light images, For the reflection component, For illumination components, For element-wise multiplication, For illumination smoothing constraints, For infrared-guided structural consistency constraints. , These are the balancing weighting coefficients.

[0010] As a preferred embodiment of the nighttime low-light image enhancement network method of gradient domain infrared structure constraint and dynamic gating fusion of the present invention, the processing procedure of the illumination feature extraction module in S2 is as follows: S21. Predict the pixel-wise illumination adjustment curve parameters of the input image using a three-layer lightweight convolutional network. ; S22. Based on the illumination adjustment curve parameters, obtain the illumination map after illumination adjustment through iterative nonlinear illumination mapping. The iterative formula is: ; in, For each iteration round, the initial state is... Input image; S23. Adjusting the illumination curve parameters using 1×1 convolution. Channel dimension mapping is performed to obtain high-dimensional illumination features. This illumination feature only characterizes spatial illumination distribution information and does not contain texture or noise interference.

[0011] As a nighttime low-light image enhancement network implementation method of gradient domain infrared structure constraint and dynamic gating fusion according to the present invention, preferably, the feature-guided multi-head self-attention mechanism in S3 includes two working modes: Cross-modal alignment mode: Query, key, and value vectors are generated using illumination features of one modality, and the value vectors are modulated element-by-element using illumination features of the other modality as guiding information. Attention weights are calculated through cross-modal query-key similarity to achieve spatial alignment of bimodal features. Feature enhancement mode: The query, key, and value vectors are all derived from image features. The value vector is modulated by fusing illumination features as guiding information. Global contextual dependencies are modeled through self-attention to achieve selective feature enhancement.

[0012] As a preferred embodiment of the nighttime low-light image enhancement network method of gradient domain infrared structure constraint and dynamic gating fusion according to the present invention, the processing procedure of the adaptive illumination fusion module in S4 is as follows: The aligned visible light illumination features and infrared illumination features are concatenated along the channel dimension, and a spatially gated map is generated using a 1×1 convolution and a sigmoid activation function. ; The dual-modal features are weighted and fused based on the gated graph, and the fused feature expression is as follows: ; in, Infrared illumination characteristics, Gated graphs representing visible light illumination characteristics The value range is [0,1], which is used to dynamically adjust the modal contribution ratio at different spatial locations.

[0013] As a method for implementing a nighttime low-light image enhancement network that fuses gradient domain infrared structural constraints and dynamic gating according to the present invention, preferably, the encoder part of the S5U-type Transformer enhancement network is composed of stacked multi-level coding units, each coding unit sequentially including a feature enhancement calibration module and a feature-guided attention block, and connected to a downsampling operation at the end; The feature-guided attention block integrates a feature-guided multi-head self-attention mechanism, which uses the fusion of illumination features as a guide to adaptively reweight image features, highlighting the feature responses of areas with significant illumination degradation.

[0014] As a nighttime low-light image enhancement network implementation method of gradient domain infrared structure constraint and dynamic gating fusion according to the present invention, preferably, the feature enhancement calibration module includes a residual feature branch, an edge enhancement branch, and a multi-scale channel attention unit: The residual feature branch learns the residual information of the input features through two layers of convolution; The edge enhancement branch uses a convolutional layer initialized with a Laplacian kernel to extract second-order gradient information, thereby enhancing high-frequency structural features; The multi-scale channel attention unit performs multi-scale adaptive average pooling on the output features, generates channel weights through two 1×1 convolutions, and recalibrates the features channel by channel.

[0015] As a method for implementing a nighttime low-light image enhancement network based on gradient domain infrared structural constraints and dynamic gating fusion according to the present invention, preferably, the decoder part of the U-shaped Transformer enhancement network in S5 is symmetrically set with the encoder. Each decoding unit first performs upsampling and channel adjustment, and then fuses with the features of the corresponding encoder level through the gating fusion module. The gated fusion module uses the upsampled features as the query vector to generate spatial attention coefficients, performs weighted selection on the downsampled encoder features, suppresses background noise response, and focuses on the recovery of key structural regions.

[0016] As a preferred embodiment of the nighttime low-light image enhancement network method of gradient domain infrared structure constraint and dynamic gating fusion of the present invention, the expression of the composite loss function in S6 is: ; in, To calculate the reconstruction loss, the gradient domain L1 distance between the enhanced image and the ground truth image is used, combined with spatial weights. To mitigate the illumination smoothing loss, the relative relationship between the illumination component gradient and the input image gradient is constrained to ensure the spatial smoothness of the illumination estimation. As a gradient domain structure consistency loss, the L1 distance between the gradient of the enhanced image and the gradient of the infrared image is calculated to constrain structure alignment.

[0017] As a preferred implementation method of the nighttime low-light image enhancement network that integrates gradient domain infrared structural constraints and dynamic gating, this method is applied to low-light image enhancement in scenarios such as nighttime intelligent monitoring, autonomous driving, and robot navigation. The enhanced image can be directly input into downstream visual task networks for target detection and semantic segmentation.

[0018] Compared with the prior art, the beneficial effects of the present invention are: This invention constructs a Retinex enhancement framework with gradient domain infrared structural constraints. Theoretically, it introduces infrared structural information as a priori constraint for the reflection component into Retinex decomposition, establishes cross-modal structural consistency relationships in the gradient domain, avoids intensity inversion and artifact problems caused by direct pixel domain fusion, effectively compresses the solution space, and improves the stability and accuracy of structure recovery under extremely dark conditions.

[0019] This invention employs a two-stage fusion mechanism: "cross-modal feature alignment-adaptive gating fusion." First, feature-guided self-attention establishes a spatial correspondence between the two modalities, mitigating structural misalignment. Then, spatial gating weights enable dynamic weighting at the region level, automatically adjusting the modal contribution ratio based on structural reliability. This mechanism effectively suppresses structural mismatch, noise propagation, and fusion artifacts, achieving refined cross-modal information complementarity.

[0020] This invention constructs a U-shaped Transformer enhancement network guided by fusion features, injects the fused illumination prior as global guiding information into the enhancement process, and combines multi-scale channel attention and gating jump connection mechanism to achieve differentiated enhancement of different degradation regions. While improving global brightness, it effectively preserves local fine-grained structure, thus balancing enhancement effect and detail fidelity.

[0021] This invention achieves state-of-the-art (SOTA) performance enhancements while maintaining computational efficiency through a lightweight overall network structure with only 1.81M parameters and a single-graph inference speed of 52.26ms. It is suitable for both edge devices and real-time application scenarios. Furthermore, the enhanced images can directly support downstream tasks such as object detection and semantic segmentation, demonstrating high engineering application value. Attached Figure Description

[0022] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 The overall framework diagram of RetinexVRF is shown for the implementation method of nighttime low-light image enhancement network by fusion of gradient domain infrared structure constraints and dynamic gating.

[0023] Figure 2 The diagram shows the structure of the illumination decoupling and feature extraction module (LFEM) for the nighttime low-light image enhancement network implementation method that integrates gradient domain infrared structural constraints and dynamic gating.

[0024] Figure 3 This is a schematic diagram illustrating the principle of feature-guided multi-head self-attention (FG-MSA) for nighttime low-light image enhancement networks that integrate gradient domain infrared structural constraints and dynamic gating.

[0025] Figure 4 The diagram shows the structure of the Adaptive Gated Feature Fusion (ALF) module, which is used to implement a nighttime low-light image enhancement network based on gradient domain infrared structural constraints and dynamic gating fusion.

[0026] Figure 5 The diagram shows the structure of the Feature Enhancement and Calibration Module (FECM) for the implementation of a nighttime low-light image enhancement network that integrates gradient domain infrared structural constraints and dynamic gating.

[0027] Figure 6 The diagram shows the structure of the Light Guided Attention Block (FGAB) for the implementation of a nighttime low-light image enhancement network that integrates gradient domain infrared structural constraints and dynamic gating.

[0028] Figure 7 The diagram shows the structure of the gated fusion module (GFM) for the nighttime low-light image enhancement network implementation method of gradient domain infrared structural constraints and dynamic gating fusion. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Please see Figure 1-7 The present invention provides the following technical solution: a method for implementing a nighttime low-light image enhancement network that fuses gradient domain infrared structure constraints and dynamic gating, the specific process of which is as follows: The proposed RetinexVRF augmentation network consists of two main stages: a bimodal illumination feature extraction and fusion stage, and a Transformer image augmentation stage guided by fused features. The complete execution flow is as follows: Input paired low-light visible light and infrared images → respectively enter the illumination feature extraction module to decouple illumination information → visible light branch generates preliminary augmented images, bimodal output illumination features → bimodal illumination features are aligned through cross-modal self-attention → adaptive gating fusion generates fused illumination features → preliminary augmented images are input into a U-shaped Transformer augmentation network, with fused illumination features used as global guidance → encoder performs multi-scale feature extraction → decoder performs gated skip-connection fusion reconstruction → outputs the final augmented image. The network employs a composite loss constrained by gradient domain structure for end-to-end training to ensure improved quality and cross-modal consistency.

[0031] The theoretical basis of Retinex with gradient domain constraints Classical Retinex theory decomposes a visible light image into an element-wise product of the reflection and illumination components: ; in To observe visible light images, To characterize the reflection components of the inherent structure of the scene, To characterize the illumination component of the light distribution, a noise term needs to be added for extremely low-light scenes. .

[0032] In extremely dark scenes, visible light structural information is severely lacking, causing Retinex decomposition to become an ill-posed problem with an excessively large solution space prone to artifacts. This invention introduces a structural prior in infrared images, based on the principle that edges and structures are determined by gradient changes, and even with intensity reversal in the bimodal domain, the spatial location of the gradient remains highly consistent. Therefore, constraints are established in the gradient domain, using the infrared gradient as a soft constraint on the reflection component gradient. ; in The weights are spatially adaptive, and the network automatically learns the reliability of infrared information in different regions. Regions with consistent structures have high weights, while regions with modal conflicts have low weights.

[0033] Therefore, low-light enhancement is transformed into a constrained optimization problem: ; The first item is the data fidelity item, which ensures the consistency of Retinex reconstruction; The second item is the illumination smoothing constraint, which ensures the spatial smoothness of the illumination components and avoids drastic changes in illumination estimation. The third term is infrared structural constraint, which uses the L1 norm to constrain the consistency between the reflection component and the gradient of the infrared image.

[0034] Illumination feature extraction and fusion stage 1. Illumination Feature Extraction Module (LFEM) This module is used to decouple illumination information from the input image and output pixel-level illumination map and feature-level illumination representation. The specific steps are as follows: Illumination curve prediction: Pixel-wise illumination adjustment curve parameters are predicted using 3 layers of lightweight convolution (with ReLU activation). The output range is then constrained by the Tanh function to ensure numerical stability.

[0035] Iterative illumination enhancement: Based on the illumination curve parameters, an iterative nonlinear mapping is performed on the input image. The iterative formula is as follows: ; After multiple iterations, the lighting map was obtained. This reflects the overall illumination distribution trend of the image. This method achieves complex nonlinear illumination mapping with a small number of parameters while maintaining the differentiability of the network.

[0036] Illumination feature generation: Illumination curve parameters are generated through 1×1 convolution. Channel upscaling is performed to obtain high-dimensional illumination features. This feature is generated solely from lighting parameters, avoiding interference from non-lighting factors such as texture and noise. It can accurately characterize the trend of spatial lighting changes, providing a pure prior for subsequent guidance.

[0037] For the visible light branch, the original low-light image is multiplied element-wise with the illumination map to obtain a preliminary enhanced image, which is used as the input for the subsequent enhancement network; the infrared branch only outputs illumination features for subsequent fusion and does not perform pixel-level brightening.

[0038] 2. Feature-guided multi-head self-attention (FG-MSA) The core of this module is to embed guiding features into attention calculations, modulate the value vector element-by-element, and achieve conditional feature aggregation. The calculation process is as follows: The input features are reshaped into a token sequence, divided into k attention heads along the channel dimension, and then mapped through an unbiased fully connected layer to generate query Q, key K, and value V vectors.

[0039] The guiding feature Y is also divided into k heads, and the value vector is modulated element-wise. The attention calculation formula is as follows: ; in These are learnable scaling parameters used to adaptively adjust the attention weight distribution.

[0040] The multi-head outputs are spliced ​​together, and after passing through a fully connected layer and learnable position encoding, they are reshaped into output features with the original spatial dimensions.

[0041] This module is reused twice in the network, corresponding to two independent working modes: Cross-modal alignment mode: Used before fusion, it uses bimodal illumination features as inputs and guides to establish spatial correspondence through cross-modal QK similarity, which alleviates structural misalignment and intensity reversal problems. Its core function is feature alignment.

[0042] Feature enhancement mode: Enhances the network internally. Q / K / V are all derived from image features. Illumination features are fused as a guide. Region-selective enhancement is achieved by modulating the value vector. The core function is conditional feature enhancement.

[0043] 3. Adaptive Light Blending Module (ALF) This module uses a gating mechanism to achieve dynamic fusion of dual-modal features, replacing traditional static weighting or direct splicing. The specific process is as follows: The aligned visible light illumination features and infrared illumination features are stitched together in the channel dimension.

[0044] Spatial gating maps are generated using 1×1 convolution and the Sigmoid activation function. Each spatial location corresponds to a weight between 0 and 1.

[0045] Weighted fusion based on gating weights: ; The channel dimensions are then compressed by a 1×1 convolution to obtain the final fused lighting features.

[0046] This mechanism enables region-adaptive fusion: in dark areas and areas lacking texture, the gating weights are biased towards infrared features, utilizing their stable structural information; in areas rich in detail and with sufficient illumination, the weights are biased towards visible light features, preserving the original texture details.

[0047] Table 2: Performance comparison of various methods on PSNR and SSIM on the LLVIP and MSRS datasets. Retinex indicates whether the method is based on Retinex theory. Bold text indicates the best and second-best results.

[0048] Feature-guided Transformer image reconstruction stage This stage employs a U-shaped encoder-decoder structure, using the initially enhanced image as input and incorporating illumination features as global guidance to complete multi-scale adaptive enhancement.

[0049] Feature Enhancement and Calibration Module (FECM) This module, located at the encoder input, is used to correct for local over-enhancement, noise amplification, and texture distortion caused by the initial enhancement. It contains three core units: Residual feature branch: Learn the residual information of the input features through two layers of convolution (LeakyReLU activation), transforming the learning objective from direct mapping to residual fitting, reducing the optimization difficulty and alleviating gradient vanishing.

[0050] Edge enhancement branch: Convolutional layers initialized with Laplacian kernels extract second-order gradient information to enhance high-frequency edge structures. This is the branch weight coefficient, used to control the edge enhancement intensity.

[0051] Multi-scale channel attention: The output features are subjected to adaptive average pooling at three scales, and after concatenation, they are generated by two 1×1 convolutions to generate channel weights. The features are recalibrated channel by channel to adapt to the multi-scale illumination distribution in low-light scenes.

[0052] The module's final output is the sum of the input features, residual branches, and edge branches. After channel attention calibration, the output can stabilize the feature distribution and provide reliable input for subsequent attention modeling.

[0053] Light-guided attention blocks (FGAB) The core unit of each layer of the encoder adopts the classic Transformer structure of "layer normalization → FG-MSA (feature enhancement mode) → layer normalization → feedforward network", with two residual connections to ensure gradient propagation. This module is guided by the fusion of illumination features, models global context dependencies through self-attention, and adaptively weights features of different regions according to illumination guidance: the feature responses of dark areas and structural areas are enhanced, while the responses of noise and background areas are suppressed, achieving selective enhancement of illumination perception.

[0054] Each stage of the encoder contains an FGAB and a downsampling layer, progressively reducing spatial resolution and increasing the number of channels to extract multi-scale semantic and structural features.

[0055] Gated Fusion Module (GFM) Each layer of the decoder fuses upsampled features with the corresponding downsampled features from the encoder using GFM, replacing the traditional direct concatenation or addition. The specific process is as follows: The upsampled and downsampled feature channels are concatenated, and spatial attention coefficients are generated by element-wise addition, ReLU activation, 1×1 convolution, and the Sigmoid function.

[0056] The attention coefficients are resampled to the corresponding size, and the downsampled encoder features are weighted.

[0057] This module uses the global semantic information carried by the upsampled features as a query to guide the model to select key structural information from the encoder features, suppress the spread of background noise, and focus on the structural recovery of the target region.

[0058] The decoder maps the features to a three-channel RGB image through a 3×3 convolution. The image is then added to the input pre-enhanced image via residual connection to obtain the final enhancement result, thus avoiding overall tone and structure shift.

[0059] Loss function and training / inference process Composite loss function The network is trained using a three-factor joint supervision method, which corresponds one-to-one with the theoretical model: Reconstruction losses Calculate the L1 gradient distance between the enhanced image and the ground truth image, and combine it with spatial weights. The focus is on constraining the reconstruction accuracy of structural areas to ensure visual detail.

[0060] Light smoothing loss The formula for constraining the relative ratio of the illumination component gradient to the input image gradient is: ; in It is set to a minimum value to prevent division by zero errors.

[0061] Structural consistency loss Calculate the L1 distance between the gradient of the enhanced image and the gradient of the infrared image to ensure cross-modal structure alignment and implement gradient domain constraints.

[0062] Table 2. Target detection results on enhanced images generated by different methods, where “” indicates that the corresponding target was not detected by the network. The best results are shown in bold red, and the second best results are shown in bold blue.

[0063]

[0064] The total loss is: ; , These are hyperparameters; the weights are adjusted during training based on the characteristics of the dataset.

[0065] Training and reasoning process The network is implemented using the PyTorch framework and trained end-to-end on the LLVIP and MSRS nighttime datasets. The Adam optimizer is used, with an initial learning rate of [missing information]. The momentum parameters are (0.9, 0.999), and a cosine annealing learning rate strategy with restart is adopted. During training, data augmentation methods such as reducing exposure and adding noise are used to simulate extremely dark conditions. A mix-up strategy is introduced to alleviate overfitting. The model is validated every 1000 iterations, and converges after about 100k iterations.

[0066] Table 2: Ablation studies of the Retinex VRF module on the LLVIP and MSRS datasets

[0067] Table 3: Ablation experiments for removing infrared information branches

[0068] During inference, the network takes a pair of low-light visible light and infrared images as input, and can output an enhanced visible light image in a single forward propagation without additional post-processing, allowing it to be directly connected to downstream vision tasks.

[0069] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for implementing a nighttime low-light image enhancement network that fuses gradient domain infrared structural constraints and dynamic gating, characterized in that, Includes the following steps: S1. Input paired nighttime low-light visible light images and infrared images, construct a Retinex enhancement framework that introduces infrared gradient domain structure constraints, and model low-light enhancement as a constrained optimization problem with infrared structure priors. S2. The visible light image and infrared image are decoupled by the illumination feature extraction module to output pixel-by-pixel illumination map and high-dimensional illumination features. The visible light branch multiplies the input low-light image element-wise with the corresponding illumination map to obtain a preliminary enhanced image; S3. A feature-guided multi-head self-attention mechanism is adopted to perform cross-modal spatial alignment of dual-modal illumination features, establish the structural correspondence between infrared and visible light features, and alleviate the problems of intermodal structural inconsistency and intensity reversal. S4. Through the adaptive illumination fusion module, spatially adaptive fusion weights are generated based on the gating mechanism, and the aligned dual-modal illumination features are dynamically weighted and fused to obtain the fused illumination features. S5. Construct a U-shaped Transformer enhancement network guided by fusion features. Using the preliminary enhanced image as input and the fused illumination features as global guidance information, the network completes multi-scale feature modeling and gated jump connection fusion through an encoder-decoder structure, and finally outputs the enhanced visible light image. S6. Construct a composite loss function that includes reconstruction loss, illumination smoothing loss, and gradient domain structure consistency loss, and perform end-to-end supervised training on the entire enhancement network.

2. The method for implementing a nighttime low-light image enhancement network based on gradient domain infrared structural constraints and dynamic gating fusion as described in claim 1, characterized in that: The infrared gradient domain structural constraint in S1 specifically involves establishing a structural consistency relationship between the visible light image reflection component and the infrared image in the gradient domain, expressed as follows: ； in, The gradient of the visible light reflectance component. The gradient of the infrared image. It is a spatially adaptive weight used to characterize the reliability of infrared structural information in different regions; The corresponding constrained optimization problem expression is: ； in, To observe low-light visible light images, For the reflection component, For illumination components, For element-wise multiplication, For illumination smoothing constraints, For infrared-guided structural consistency constraints. , These are the balancing weighting coefficients.

3. The method for implementing a nighttime low-light image enhancement network based on gradient domain infrared structural constraints and dynamic gating fusion as described in claim 1, characterized in that: The specific processing procedure of the illumination feature extraction module in S2 is as follows: S21. Predict the pixel-wise illumination adjustment curve parameters of the input image using a three-layer lightweight convolutional network. ; S22. Based on the illumination adjustment curve parameters, obtain the illumination map after illumination adjustment through iterative nonlinear illumination mapping. The iterative formula is: ; in, For each iteration round, the initial state is... Input image; S23. Adjusting the illumination curve parameters using 1×1 convolution. Channel dimension mapping is performed to obtain high-dimensional illumination features. This illumination feature only characterizes spatial illumination distribution information and does not contain texture or noise interference.

4. The method for implementing a nighttime low-light image enhancement network based on gradient domain infrared structural constraints and dynamic gating fusion as described in claim 1, characterized in that: The feature-guided multi-head self-attention mechanism in S3 includes two operating modes: Cross-modal alignment mode: Query, key, and value vectors are generated using illumination features of one modality, and the value vectors are modulated element-by-element using illumination features of the other modality as guiding information. Attention weights are calculated through cross-modal query-key similarity to achieve spatial alignment of bimodal features. Feature enhancement mode: The query, key, and value vectors are all derived from image features. Illumination features are fused as guiding information to modulate the value vector. Global contextual dependencies are modeled through self-attention. To achieve selective enhancement of features.

5. The method for implementing a nighttime low-light image enhancement network based on gradient domain infrared structural constraints and dynamic gating fusion according to claim 1, characterized in that: The specific processing procedure of the adaptive illumination fusion module in S4 is as follows: The aligned visible light illumination features and infrared illumination features are concatenated along the channel dimension, and a spatially gated map is generated using a 1×1 convolution and a sigmoid activation function. ; Based on the gated graph, the dual-modal features are weighted and fused. The fused feature expression is as follows: ; in, Infrared illumination characteristics, Gated graphs representing visible light illumination characteristics The value range is [0,1], which is used to dynamically adjust the modal contribution ratio at different spatial locations.

6. The method for implementing a nighttime low-light image enhancement network based on gradient domain infrared structural constraints and dynamic gating fusion according to claim 1, characterized in that: The encoder part of the S5U Transformer enhancement network is composed of multiple levels of coding units stacked together. Each level of coding unit includes a feature enhancement calibration module and a feature-guided attention block in sequence, and a downsampling operation is connected at the end. The feature-guided attention block integrates a feature-guided multi-head self-attention mechanism, which uses the fusion of illumination features as a guide to adaptively reweight image features, highlighting the feature responses of areas with significant illumination degradation.

7. The method for implementing a nighttime low-light image enhancement network based on gradient domain infrared structural constraints and dynamic gating fusion according to claim 1, characterized in that: The feature enhancement and calibration module includes a residual feature branch, an edge enhancement branch, and a multi-scale channel attention unit. The residual feature branch learns the residual information of the input features through two layers of convolution; The edge enhancement branch uses a convolutional layer initialized with a Laplacian kernel to extract second-order gradient information, thereby enhancing high-frequency structural features; The multi-scale channel attention unit performs multi-scale adaptive average pooling on the output features, generates channel weights through two 1×1 convolutions, and recalibrates the features channel by channel.

8. The method for implementing a nighttime low-light image enhancement network based on gradient domain infrared structural constraints and dynamic gating fusion according to claim 1, characterized in that: In the S5, the decoder part of the U-shaped Transformer enhancement network is symmetrically set with the encoder. Each decoding unit first performs upsampling and channel adjustment, and then fuses with the features of the corresponding encoder level through the gated fusion module. The gated fusion module uses the upsampled features as the query vector to generate spatial attention coefficients, performs weighted selection on the downsampled encoder features, suppresses background noise response, and focuses on the recovery of key structural regions.

9. The method for implementing a nighttime low-light image enhancement network based on gradient domain infrared structural constraints and dynamic gating fusion according to claim 1, characterized in that: The expression for the composite loss function in S6 is as follows: ； in, To calculate the reconstruction loss, the gradient domain L1 distance between the enhanced image and the ground truth image is used, combined with spatial weights. To mitigate the illumination smoothing loss, the relative relationship between the illumination component gradient and the input image gradient is constrained to ensure the spatial smoothness of the illumination estimation. As a gradient domain structure consistency loss, the L1 distance between the enhanced image gradient and the infrared image gradient is calculated to constrain structure alignment.

10. The method for implementing a nighttime low-light image enhancement network based on gradient domain infrared structural constraints and dynamic gating fusion according to claim 1, characterized in that: This method is applied to low-light image enhancement in scenarios such as nighttime intelligent monitoring, autonomous driving, and robot navigation. The enhanced images can be directly input into downstream visual task networks for target detection and semantic segmentation.

Images