An implicit retinex low-light image enhancement method based on illumination texture collaborative interaction

Abstract

The application discloses a kind of fusion implicit Retinex constraint and light texture collaborative interaction light low light enhancement method. The method first constructs double-flow architecture, utilizes the global flow of Transformer to capture long-range light distribution prior, and is extracted through multi-scale structured texture recovery module in parallel Multi-band local feature. Subsequently, design light guide feature adaptive modulation module, map global prior to spatially adaptive scaling and offset factor, perform pixel-level gain adjustment on local feature, realize the accurate guidance of light information to texture recovery. Finally, in the training stage, introduce implicit Retinex loss function containing structure perception smoothing and color consistency constraint, internalize physical law to network weight, generate enhanced image through simple linear transformation. The application effectively suppresses color deviation and artifact without increasing inference overhead, significantly improves the robustness of lightweight network under complex lighting, and is suitable for mobile real-time visual tasks.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, specifically to a lightweight low-light image enhancement method based on deep learning.

[0002] In particular, the present invention relates to an enhancement technique that integrates implicit Retinex physical prior constraints and a two-stream feature interaction architecture, aiming to solve the problems of noise suppression and texture restoration of non-uniformly illuminated images under computationally limited environments. Background Technology

[0003] With the rapid development of the Internet of Things, smart cities, and autonomous driving technologies, the reliability of image acquisition equipment in all-weather environments has become crucial. However, under non-ideal lighting conditions such as nighttime, backlighting, and cloudy / rainy days, images acquired are often characterized by extremely low brightness, poor contrast, severe noise interference, and color distortion due to limitations in sensor physical performance. This "safety blind spot" directly leads to a sharp drop in the recall rate of computer vision systems (such as autonomous driving object detection and semantic segmentation), posing a significant safety hazard.

[0004] In the field of digital image processing, existing low-light image enhancement techniques can be mainly divided into three categories: methods based on traditional Retinex theory, data-driven methods based on convolutional neural networks (CNN), and cutting-edge exploration methods based on Transformer in recent years.

[0005] While the traditional Retinex algorithm is based on physical modeling, it is prone to producing halo artifacts when processing high-contrast edges and often involves complex variational optimization processes, resulting in significant computational overhead. Although CNN-based deep learning methods achieve end-to-end augmentation, they often neglect the physical consistency of image illumination decomposition, leading to a lack of model interpretability.

[0006] In recent years, lightweight networks, such as the Illumination Adaptive Transformer (IAT), have significantly improved inference speed. However, they are essentially based on data-driven parameter estimation and lack explicit modeling of the physical model of illumination. In practical applications, such methods have significant technical limitations: First, their global illumination adjustment and local texture restoration processes are often independent, leading to local overexposure or under-enhancement in scenes with extremely uneven illumination distribution. Second, in regions with extremely low signal-to-noise ratios, the network struggles to balance high-frequency noise suppression with the preservation of subtle texture details, easily resulting in the loss of smooth details or the fragmentation of texture structure.

[0007] Therefore, how to introduce physical priors to improve the detail recovery capability and physical consistency in scenes with uneven illumination distribution while ensuring lightweight real-time inference has become a key technical challenge that urgently needs to be solved in the field of low-light enhancement. Summary of the Invention

[0008] This invention provides a lightweight low-light image enhancement method that integrates implicit Retinex constraints and lighting-texture collaborative interaction. It aims to solve the technical problems of lack of physical interpretability and disconnect between lighting and texture processing in existing lightweight networks under complex lighting environments by using physical prior constraints and feature-level interaction.

[0009] The image enhancement method provided by this invention includes the following steps:

[0010] S1: Two-stream feature extraction: The input low-light RGB image is fed into a two-stream network architecture; the global stream branch is used to capture the global illumination distribution prior based on the long-range modeling capability of Transformer; at the same time, the local stream branch is used to capture subtle texture details based on the Multi-Scale Structured Texture Recovery Module (MSTB).

[0011] S2: Illumination-guided feature modulation: Design an illumination-guided feature adaptive modulation module (IGFA) to map the global illumination prior to a scaling factor and an offset factor through an upsampling alignment operation, and perform pixel-level affine transformation recalibration on the features extracted from the local flow branch.

[0012] S3: Implicit Physical Parameter Prediction: Predicting the Multiplication Map using the results of dual-stream feature fusion. (Simulated inverse illumination transformation) and additive fine-tuning diagram (Simulate environmental noise or lighting shift);

[0013] S4: Linear Transformation Reconstruction: Executes pixel-level linear transformation formulas during the inference phase. Generate enhanced images with zero inference increment in computational cost;

[0014] S5: Physically Constrained Training: An implicit Retinex loss function is introduced during the training phase. The structure-aware smoothing loss forces the multiplicative map to have spatial smoothness, and the color consistency loss eliminates color distortion, thus internalizing physical laws into the network weights.

[0015] The present invention has the following beneficial effects:

[0016] (1) Strong physical interpretability: The optical physical constraints are internalized into the training stage through the implicit Retinex paradigm, which not only preserves the color and structure consistency of the physical model, but also avoids the complex explicit sub-network decomposition of traditional models, thus achieving a unity of lightweight and physicality.

[0017] (2) Precise interaction mechanism: The IGFA module is used to realize feature guidance from global to local, which effectively solves the problem of local overexposure or missing dark texture caused by uneven illumination distribution in traditional methods.

[0018] (3) Excellent deployment performance: The architecture only involves linear matrix operations during the inference phase, with extremely low computational overhead, making it particularly suitable for real-time enhancement tasks in resource-constrained scenarios such as mobile devices and security monitoring. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall architecture of the lightweight low-light image enhancement network (RGI-Net) that integrates implicit Retinex constraints and lighting texture collaborative interaction as described in this invention.

[0020] Figure 2 This is a schematic diagram of the illumination-guided feature adaptive modulation module (IGFA) described in this invention, illustrating the mapping process from global prior to local feature modulation.

[0021] Figure 3 This is a schematic diagram of the structure of the Multi-Scale Structured Texture Recovery Module (MSTB) described in this invention, which illustrates the fusion logic of parallel convolutional branches and channel attention mechanisms.

[0022] Figure 4 This is a schematic diagram of the training logic based on implicit physical constraints described in this invention, illustrating the multiplication mapping graph. With addition fine-tuning diagram The generation process under loss function constraints.

[0023] Figure 5 This is a flowchart of an implicit Retinex low-light image enhancement method based on lighting and texture collaborative interaction. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0025] like Figure 1 As shown, the image enhancement method (RGI-Net) based on implicit Retinex constraints and coordinated interaction of illumination and texture provided in this embodiment of the invention mainly includes the following processing stages:

[0026] S1: Dual-stream architecture feature extraction: The input low-light RGB image is fed into the Global Branch and the Local Branch. The Global Branch uses the long-range modeling capability of the Transformer architecture to extract global illumination distribution priors (Query Embeddings); the Local Branch uses the Multi-Scale Structured Texture Recovery (MSTB) module to extract the micro-texture and local detail features of the image.

[0027] S2: Illumination-guided feature modulation (IGFA): such as Figure 2 As shown, the global illumination prior output by the global flow branch is bilinearly upsampled to align its spatial dimensions with the intermediate feature maps of the local flow branch. The scaling factor is then predicted in parallel using two independent convolutional heads. ) and offset factor (i.e. And, by constraining the numerical range using the Sigmoid activation function, pixel-by-pixel affine transformation recalibration of local flow branch features is achieved. The modulation formula is as follows: .

[0028] Through this mechanism, the network achieves content-aware modulation guided by global illumination.

[0029] S3: Parameter Prediction and Linear Transformation Reconstruction: Based on the recalibrated feature map, predict the multiplication map. With addition fine-tuning diagram During the inference phase, pixel-level linear transformations are performed directly. This process abandons explicit physical model iteration and achieves lightweight reconstruction with "zero inference increment".

[0030] S4: Physically Constrained Training: An implicit Retinex loss function is introduced during the training phase. By constructing a Structure-Aware Smoothness Loss and a Color Constancy Loss, the multiplicative mapping graph is processed during training. The mandatory constraint is a lighting prior with edge-preserving properties, and the additive fine-tuning map is applied. The constraint is a bias term with noise sparsity, thereby internalizing the physical laws of Retinex into the network weights, ensuring the visual realism and physical consistency of the augmented results.

[0031] like Figure 1 As shown, the RGI-Net described in this embodiment of the invention includes a global branch and a local branch. In the global branch, a Transformer module based on a self-attention mechanism is used to capture the prior of the illumination distribution of the image across the entire image by calculating the Query-Key similarity matrix; in the local branch, a multi-scale structured texture restoration module (MSTB) is constructed.

[0032] like Figure 3 As shown, the MSTB module is implemented as follows: The input local feature map is simultaneously fed into three parallel dilated convolution branches, with dilation rates of 1, 2, and 3, respectively, to cover microscopic details, mesoscale texture, and macroscopic structural information. The features extracted from each branch are concatenated, then compressed in channel dimension using a 1×1 convolution, and connected to a channel attention mechanism (SE-Block). The SE-Block obtains global channel descriptors through a global average pooling layer, generates channel weight vectors through two fully connected layers, and performs channel-wise multiplication with the input features to adaptively suppress high-noise regions and highlight structural texture features.

[0033] like Figure 2 As shown, the specific processing flow of the IGFA module is as follows: The global illumination prior feature map output by the global flow branch is processed... Perform bilinear upsampling to make it spatially similar to the intermediate feature map of the local flow branch. Alignment. Then, two independent 3×3 convolutional heads are used in parallel to process the aligned features, generating scaling factor maps respectively. With offset factor plot Furthermore, the Sigmoid function maps the values ​​to the (0,1) interval. Finally, by executing... The pixel-level affine transformation completes the dynamic recalibration of local features, realizing the physical guidance of lighting for texture restoration.

[0034] like Figure 4 As shown, the implicit Retinex loss function used in the training phase of this embodiment of the invention Structure-aware smoothing loss and color consistency loss The components, working together, adjust the model's parameters. The calculation formula is as follows: .

[0035] in, Represents the gradient operator, Hyperparameters preserved to balance the edge structure. This term, guided by the gradient of the original image, forces the multiplicative mapping graph to... Maintain smoothness in flat regions (small gradients) and prevent structural degradation in edge regions (large gradients). Color consistency loss. By constraining the difference between the enhanced image and the original image in the RGB three channels, it ensures that the lighting enhancement process does not destroy the inherent color properties of the object.

[0036] Through the organic combination of the above modules, this invention not only achieves lightweight edge deployment, but also internalizes the physical laws of Retinex into network parameter constraints, effectively improving the robustness and realism of image enhancement under complex lighting conditions.

Claims

1. A lightweight low-light image enhancement method that integrates implicit Retinex constraints and lighting-texture collaborative interaction, characterized in that, The steps include: Step 1: The input low-light image is fed into the global flow branch and the local flow branch in the two-stream architecture for feature extraction; Step 2: In the global flow branch, the Transformer architecture is used to capture long-range dependencies to extract the global illumination distribution prior. Step 3: In the local flow branch, multi-band local texture features are extracted in parallel using a multi-scale structured texture recovery module (MSTB). Step 4: An illumination-guided feature adaptive modulation module (IGFA) is designed to map the global illumination prior output from the global flow branch to spatial adaptive modulation parameters, and the output features of the local flow branch are recalibrated pixel-by-pixel. Step 5: Based on the recalibrated feature prediction multiplication map... With addition fine-tuning diagram According to the linear transformation formula Generate enhanced image; Step 6: Introduce implicit Retinex loss function during the training phase for the multiplication map. Addition fine-tuning diagram Imposing physical constraints internalizes physical knowledge into network weights.

2. The method according to claim 1, characterized in that: The MSTB module in step 3 contains multiple parallel convolutional branches, each with a different dilation rate to capture microscopic details, mesoscale textures, and macroscopic structural information, respectively.

3. The method according to claim 2, characterized in that: The MSTB module incorporates a channel attention mechanism (SE-Block) after multi-scale feature fusion. It learns channel weights through global average pooling and fully connected layers, adaptively suppressing high-noise channels and enhancing structural feature channels.

4. The method according to claim 1, characterized in that: The implementation process of the IGFA module in step 4 is as follows: Step 1: Upsample the global illumination prior to align its spatial dimension with the local texture features; Step 2: Use a dual-head convolution structure to generate a scaling factor for controlling feature intensity and an offset factor for adjusting the reference value; Step 3: Use the scaling factor and the offset factor to perform pixel-wise affine transformation modulation on the local feature map.

5. The method according to claim 1, characterized in that: The implicit Retinex loss function in step 6 includes at least: Structure-aware smoothing loss: constructing a weight graph using the exponential term of the original graph's gradient, and forcing a multiplicative mapping in flat regions. Smoothing preserves structural information in edge regions; Color consistency loss: Based on the gray-world assumption, the global average intensity of the RGB three channels of the image is made more consistent by constraining it, thus eliminating color shift.

6. The method according to claim 5, characterized in that: The implicit Retinex loss function also includes spatial consistency loss, which constrains the difference vectors of the enhanced image and the original image to maintain monotonic consistency in the local neighborhood, thus preventing image geometric distortion.

7. The method according to claim 1, characterized in that: The linear transformation generation process in step 5 is a simplified matrix operation, which achieves physical internalization with zero inference increment during the inference stage.

Images