A low-illumination image enhancement method based on Gaussian light field sputtering guided self-attention

Abstract

The present application belongs to the technical field of image processing and computer vision, and particularly relates to a low-illumination image enhancement method based on Gaussian light field sputtering guided self-attention. Firstly, multi-scale feature extraction is performed on the low-illumination image, and Gaussian geometric parameters and semantic Token are generated by using multi-scale Gaussian markers; then in the self-attention calculation process, Gaussian affinity is constructed according to the Gaussian geometric parameters, and is introduced into the attention weight calculation as a physical bias to obtain enhanced features fused with physical light field prior; a continuous space gain field is reconstructed based on the enhanced features, and the input image is corrected in brightness by using the space gain field to obtain an enhanced image; and the network is optimized by using an unsupervised training target containing a color vector angle loss and a brightness edge loss. The present application can improve the light recovery capability, color fidelity capability and detail preservation capability in a complex low-light scene, and can be applied to the fields of intelligent monitoring, automatic driving, mobile terminal imaging and the like.

Description

Technical Field

[0001] This invention belongs to the field of digital image processing technology, specifically relating to a low-light image enhancement method based on Gaussian light field sputtering-guided self-attention. Background Technology

[0002] Images captured in low-light environments often suffer from low brightness, poor contrast, color distortion, and significant noise, severely impacting the performance of subsequent advanced computer vision tasks. Therefore, Low-Light Image Enhancement (LLIE) technology has emerged and become a research hotspot in the field of image processing.

[0003] Existing LLIE methods are mainly divided into two categories. One category is traditional methods based on physical models, such as Retinex theory and its variants, which enhance images by estimating and adjusting illumination components. The other category is deep learning-based methods, where supervised learning methods have made significant progress by training networks with paired data. However, paired data is difficult to obtain in real-world scenes, limiting its generalization ability. Therefore, unsupervised / zero-shot learning paradigms that do not require paired data are receiving increasing attention, such as EnlightenGAN based on generative adversarial networks, the Zero-DCE series based on depth curve estimation, and methods based on implicit neural representations.

[0004] While the aforementioned unsupervised methods have achieved some success in improving overall visibility, they still commonly suffer from localized exposure imbalances and color distortions when dealing with non-uniform and complex spatially varying lighting conditions. In recent years, the Visual Transformer (ViT) has been introduced into the field of image restoration due to its superior long-range dependency modeling capabilities. However, these models often operate as purely data-driven black boxes, and their self-attention mechanisms typically rely solely on learnable positional biases for prior information, severely lacking the ability to model physical lighting degradation processes such as anisotropic scattering and attenuation of light. This makes it difficult for these models to infer smooth enhancement gains that conform to the characteristics of natural light fields under extremely non-uniform lighting conditions.

[0005] Meanwhile, continuous explicit scene representation techniques, represented by 3D / 2D Gaussian sputtering, have shown great potential in 3D reconstruction and 2D image compression due to their ability to fit complex geometry and appearance using anisotropic Gaussian functions. Recently, some pioneering works have begun to explore the introduction of 2D Gaussian sputtering into low-light enhancement tasks.

[0006] However, all the aforementioned existing technologies share a common fundamental limitation: they treat Gaussian sputtering merely as a standalone, front-end image representation tool or a back-end gain generator, failing to seamlessly integrate its continuous physical light field priors into the core network architecture, particularly the internal computation of the Transformer's self-attention mechanism. In other words, the learning of Gaussian geometry and semantic features is separate; the physical priors fail to guide the network's nonlocal information interaction, thus remaining fundamentally insufficient to solve the problem of accurate reconstruction under complex, non-uniform illumination.

[0007] Therefore, how to break through the architectural barriers of existing large-scale visual models under an unsupervised framework, and deeply embed the continuous physical light field prior of 2D Gaussian sputtering as an explicit anisotropic bias into the self-attention mechanism of Transformer, so that the model can perceive and follow the physical lighting structure to enhance the image, thereby effectively overcoming local exposure imbalance and color distortion, is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0008] This invention aims to address the shortcomings of existing low-light image enhancement methods in handling complex, non-uniform lighting scenes. These methods lack effective modeling of prior physical illumination degradation, leading to problems such as local exposure imbalance, insufficient recovery of details in dark areas, color distortion, and degradation of structural information during the enhancement process. The invention provides a low-light image enhancement method based on Gaussian light field sputtering-guided self-attention.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: A low-light image enhancement method based on Gaussian light field sputtering-guided self-attention includes the following steps: A low-light image to be enhanced is acquired, and the low-light image is input into an encoder for feature extraction to obtain a multi-scale feature map. A multi-scale Gaussian tokenizer is constructed based on the multi-scale feature map to predict the Gaussian geometric parameters and semantic tokens corresponding to each scale. The Gaussian geometric parameters include at least the Gaussian center position, the inverse covariance matrix, and the transparency parameter. The Gaussian geometric parameters are associated with the semantic token and then input into the Gaussian light field sputtering guided self-attention module; During the self-attention calculation process, the Gaussian affinity between tokens is calculated based on the Gaussian geometric parameters, and the Gaussian affinity is used as a physical bias to be introduced into the attention weight calculation to obtain enhanced features that fuse physical light field priors. Reconstruct the continuous spatial gain field based on the enhanced features; The spatial gain field is used to perform brightness correction on the input low-light image to obtain an enhanced image; The network parameters are optimized based on unsupervised training objectives, which include at least color vector angle loss and brightness edge loss.

[0010] Furthermore, the multi-scale Gaussian tokenizer includes a coarse-grained branch and a fine-grained branch, which are used to extract global illumination structure information and local geometric detail information, respectively. The semantic tokens output by the two branches are fused to obtain the final Gaussian semantic token feature, and the transparency output by the two branches is fused to obtain the final transparency.

[0011] Furthermore, the Gaussian geometric parameters are generated as follows: the center position is obtained by superimposing the relative offset of the network prediction on the normalized grid; the original value of the network prediction is nonlinearly transformed and truncated to obtain the diagonal and off-diagonal elements of the covariance inverse matrix; the original value of the network prediction is added with a learnable bias and then the transparency is obtained by the Sigmoid function.

[0012] Furthermore, the Gaussian affinity is obtained by calculating the Mahalanobis distance based on the relative coordinate difference between any two Tokens and the inverse covariance matrix corresponding to the query Token, and generating the corresponding Gaussian affinity based on the Mahalanobis distance; even further, the Gaussian affinity is obtained by performing a negative exponential mapping on the Mahalanobis distance.

[0013] Furthermore, the Gaussian light field sputtering guided self-attention module includes multiple attention heads, each corresponding to an independent learnable temperature parameter. The attention weights are obtained by fusing semantic attention terms and physical bias terms through learnable gating. The module also includes a geometry refinement branch, which is used to generate Gaussian parameter residuals based on the current semantic token and iteratively update the Gaussian geometric parameters.

[0014] Furthermore, the continuous spatial gain field is generated by a Gaussian light field reconstruction module, which includes an intrinsic illumination prediction head, a noise estimation head, and a gain field refinement head; the gain field refinement head generates the spatial gain field based on the intrinsic illumination field, noise field, and transparency parameters.

[0015] Furthermore, the unsupervised training objective also includes at least one of exposure consistency loss, spatial consistency loss, and physical field smoothing loss; wherein, the color vector angle loss is used to constrain the directional consistency of the corresponding pixel RGB vectors before and after enhancement; and the brightness edge loss is used to constrain the edge structure consistency in the brightness channel of the enhanced image and the input image.

[0016] The present invention also provides a low-light image enhancement system based on Gaussian light field sputtering guided self-attention, including an image acquisition module, a feature extraction module, a Gaussian parameter generation module, a Gaussian light field sputtering guided self-attention module, a light field reconstruction module, and an image enhancement module, wherein each module works together to execute the above-mentioned low-light image enhancement method.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention constructs a multi-scale Gaussian marker to represent scene lighting information as a set of Gaussian parameters with geometric constraints, thereby enabling explicit modeling of complex non-uniform lighting structures and improving lighting characterization capabilities.

[0018] This invention introduces a physical bias based on Gaussian affinity during the self-attention calculation process, enabling the Transformer to perceive the spatial distribution characteristics of the light field while modeling long-distance dependencies, thereby improving the accuracy of illumination recovery in complex low-light scenes.

[0019] This invention utilizes a Gaussian light field reconstruction module to generate a continuous spatial gain field, enabling adaptive adjustment of brightness in local areas and effectively mitigating the problems of local overexposure and underexposure that are prone to occur in traditional enhancement methods. This invention improves the accuracy and stability of illumination field reconstruction by setting up a geometric refinement mechanism to iteratively update Gaussian geometric parameters, thereby enabling the Gaussian parameters and semantic features to be co-optimized. This invention introduces color vector angle loss and brightness edge loss during the model training phase, which enhances brightness while maintaining hue consistency and edge structure integrity, reduces color distortion and detail blurring, and improves the visual quality of the enhanced image. Attached Figure Description

[0020] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will now be described in detail with reference to the accompanying drawings, wherein... Figure 1 A schematic diagram of the overall architecture of the GLFS model used in the low-light image enhancement method based on Gaussian light field sputtering guided self-attention provided by the present invention; Among them, (A) is the overall framework, which adopts an encoder-decoder structure, with the core being the GS-ViT bottleneck layer that embeds multiple physical priors; (B) is the Multi-Scale Gaussian Tokenizer (MGT), which is used to infer the geometric parameters of the Gaussian function from multi-scale features; (C) is the Gaussian Splash Self-Attention Module (GS-ViT), which introduces a physical bias based on anisotropic Gaussians into the self-attention mechanism; and (D) is the Gaussian Illumination Field (GIF) module, which reconstructs the continuous light field and generates the gain field through Gaussian splashing. Figure 2 The visualization results of the physical prior decomposition and enhancement in the method of the present invention are arranged from left to right as the input low light image, the final enhanced image, the intrinsic illumination field L, the spatial gain field Γ, and the noise map N; Figure 3 Visualization of Gaussian parameters inferred from a multi-scale Gaussian marker; Among them, (A) represents the center offset, transparency, and covariance components of the coarse-grained branch output; (B) represents the four-stage comparison of transparency evolution from coarse-grained, fine-grained, fused to refined; and (C) represents the comparison before and after transparency smoothing. Figure 4 The hierarchical evolution diagram of GS-ViT anisotropic Gaussian splashing self-attention; Among them, (A) is the attention distribution of the central token in the four GS-ViT blocks, which is diffuse in the shallow layer and gradually presents an anisotropic structure in the deep layer; (B) is the collaborative perception of different attention heads at different scales and directions through learnable temperature. Figure 5 This is a visual comparison of the method of this invention and existing methods on the LOL dataset; Figure 6 This is a visual comparison of the method of this invention and the existing technology on the LSRW-HUAWEI dataset; Figure 7 This is a visual comparison of the method of this invention and existing methods on the LSRW-NIKON dataset; Figure 8 This is a comparison of the detailed features of the method of this invention and the existing technology on the LSRW-HUAWEI dataset, where each method is shown as a magnified view of the details within the red box. Detailed Implementation

[0021] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0022] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures, and should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0023] Example 1 This embodiment provides a low-light image enhancement method based on Gaussian light field sputtering-guided self-attention, which can be applied to hardware devices that require real-time or offline improvement of low-light image quality, such as vehicle-mounted night vision systems, intelligent security monitoring analysis platforms, and smartphone night scene shooting modules. All or part of the steps of this method can be stored in a computer-readable storage medium as software modules, and can be called and executed by a processor (e.g., GPU, NPU, DSP, or dedicated image signal processor ISP).

[0024] This embodiment provides a GLFS model with optimal network parameters obtained through unsupervised training and describes its execution steps in practical low-light image enhancement.

[0025] Step 1: Obtain the low-light image to be enhanced.

[0026] Obtain a raw low-light image using image acquisition devices such as vehicle-mounted cameras, surveillance cameras, or the rear-facing camera of a mobile phone. It measures H×W and has three color channels: RGB. Image data is stored in on-chip cache or external dynamic memory that can be directly accessed by the processor.

[0027] Step 2: Encoding stage.

[0028] The processor will process the low-light image The encoder input to the pre-trained encoder-decoder type GLFS network The encoder first passes through a 7×7 convolutional layer and three downsampling convolutional blocks with a stride of 2, generating hierarchical features at four different scales. ;in, , , , The resolution decreases layer by layer, while the number of channels increases layer by layer, encoding textures, contours, and global context at different scales.

[0029] Step 3: Multiscale Gaussian tokenization (MGT).

[0030] like Figure 1 As shown in (B), the processor will use the deepest two levels of features. Input to multi-scale Gaussian marker In, that is .

[0031] MGT contains a coarse-grained branch and a fine-grained branch, which act on... This study aims to collaboratively characterize two different scales of physical signals: global light source distribution and local high-frequency compensation. In each branch, a shared 3×3 convolutional backbone is first used to extract the spatial context, followed by two 1×1 convolutions to predict a 6-dimensional geometric description tensor and a d-dimensional semantic feature tensor, respectively. The 6-dimensional geometric channels correspond to the center offset (2-dimensional), the independent elements of the covariance inverse matrix (3-dimensional), and the transparency (1-dimensional), respectively.

[0032] Specifically: (1) Central position Parameterization To make the center position Normalized domain strictly falling on pixel coordinates Internally, it uses a fixed normalized network. Based on this, the network can predict subtle relative shifts. The offset It is hyperbolic tangent contracted and coupled to the image resolution:

[0033] in, and This represents the spatial resolution of coarse-grained features.

[0034] (2) Covariance inverse matrix Parameterization Anisotropic covariance inverse matrix The legitimacy of a document must be strictly guaranteed to ensure its positive definiteness.

[0035] Diagonal elements a and c are adopted The form of adding a bias of 1 and truncation ensures both non-degeneracy and boundedness: Where a(x) is the first diagonal element (horizontal precision) of the covariance inverse matrix at pixel position x. These are the raw values ​​predicted by the network; This is the upper limit cutoff value for a(x).

[0036] The second diagonal element (Vertical accuracy) adopts the same as They have the exact same parameterization form; the only difference is that the original values ​​predicted by the network are... , .

[0037] off-diagonal elements Strictly limited to the radius of the discriminant that guarantees positive definiteness: ;in, These are the raw values ​​predicted by the network.

[0038] (3) Transparency Parameterization transparency By a learnable bias term of produce: Among them, trainable scalar parameters Initialize to positive to encourage dense activation in the early stages of training and accelerate the geometric convergence of the MGT.

[0039] To simultaneously depict global illumination and local details, MGT obtains data on two separate branches. , and , Semantic tokens for fine-grained branches by the processor Perform convolutional downsampling with a stride of 2, and combine it with the semantic token of the coarse-grained branch. Alignment, concatenation along the channel dimension, followed by a 1×1 fusion convolution. The final Gaussian semantic token features are obtained: .

[0040] Meanwhile, multi-scale fusion of transparency is achieved through a 2→1 convolution with a sigmoid function, which adaptively weights global illumination intensity with local detail transparency. ;in, This indicates channel splicing.

[0041] Figure 3 The above multi-scale collaborative process is intuitively demonstrated, with the global light source center being... Dominant, while local details such as reflections and shadow boundaries are determined by Compensation, ultimately in the fusion This is comprehensively reflected in the Chinese context.

[0042] Step 4: Transformer feature refinement based on Gaussian splashing self-attention.

[0043] like Figure 1 As shown in (C), the processor will use the initial Gaussian geometric parameters obtained in step 3. The semantic token T is input into N concatenated GS-ViT (Gaussian Splatting Vision Transformer) blocks. In this embodiment, N=4, and each block contains h=4 attention heads.

[0044] Step 401: Conditional position encoding.

[0045] Applying conditional positional encoding in the form of depthwise separable convolution to the token sequence T enables the tokens to carry spatial structure awareness, avoiding the limitation of hard-coded positional embeddings on arbitrary resolutions: .

[0046] Step 402: Multi-head linear projection.

[0047] The encoded token is subjected to standard multi-head linear projection to obtain the query Q, key K, and value V tensors: Where h is the number of heads, It is a single-head dimension.

[0048] Step 403: Calculate the anisotropic Gaussian physical bias.

[0049] For any token pair (i,j), first obtain their normalized coordinate difference. And using the inverse covariance matrix carried by the query Token i as a metric, its anisotropy is defined as a Markov property. Square distance: This distance strictly inherits the anisotropic properties of the Gaussian function: when When a certain eigenvalue has a large eigenvalue in a certain direction, the distance along that direction is significantly magnified, and vice versa.

[0050] Will Mapped to a bounded Gaussian affinity: .

[0051] To enable different attention heads to flexibly control the bias scale, each head is assigned an independent learnable temperature. ,pass To ensure the value is positive, and to prevent numerical overflow through truncation: . Upper limit cutoff value .

[0052] Step 404: Integrate attention scores based on physical bias.

[0053] For each attention head h, its final attention The semantic dot product and the physical Gaussian bias term are learned through a gating mechanism. It is composed of weighted combinations. Then, the Logit is Softmax normalized along the j-th dimension, multiplied by the value matrix V of the h-th head, and concatenated along the head dimension before linear projection. Get the output.

[0054]

[0055]

[0056] in, Let i be the query vector of the i-th token in the h-th attention head; Let be the key vector of the j-th token in the h-th attention head.

[0057] The hierarchical evolution pattern of attention graphs is as follows: Figure 4 As shown, the attention distribution of shallow modules is closer to pure semantic clustering, while the attention distribution of deep modules gradually exhibits a relationship with transparency. Anisotropic structures with uniform distribution that diffuse around the center of the light source.

[0058] Step 405: Geometric parameter refinement.

[0059] To enable the co-evolution of Gaussian geometric and semantic features, each GS-ViT block includes a lightweight geometric refinement branch after the feedforward layer. This branch takes the current token features as input and regresses a 6-dimensional residual. And through a learnable gating Make minor adjustments to the original Gaussian parameters.

[0060] The fine-tuning of the center position is achieved using the formula... The same normalized truncation form: .

[0061] The fine-tuning of the covariance inverse matrix is ​​performed by applying a positive definite projection again after the residual update. : .

[0062] The fine-tuning of transparency is truncated to the legal probability range. .

[0063] Each update increment is controlled in increments of 0.005–0.02, combined with learnable gating. small The output enables Gaussian geometry to continuously absorb semantic evidence in layer-by-layer nonlocal interactions, but the update amplitude is extremely small each time, thus ensuring the stability of the overall light field structure.

[0064] Refined transparency With initial transparency The contrast in Figure 3 This is reflected in (C): After GS-ViT bottleneck refinement The response is sharper at the boundaries and smoother within the uniform light source coverage area.

[0065] After N GS-ViT blocks, the refined parameters are output. ) and semantic tokens .

[0066] Step 5: Gaussian light field reconstruction (GIF).

[0067] The refined Token T (It has been reshaped into a two-dimensional spatial form) ) and transparency α Input Gaussian light field reconstruction module ,Right now This module contains three parallel prediction heads: (1) Internal illumination prediction head : Using the refined Token tensor T As input, through a 3×3+1×1 convolution and the Sigmoid function, the output is an image with a resolution of [resolution value missing]. Internal illumination estimation map , The normalized luminous flux at pixel x represents the normalized luminous flux.

[0068] Noise estimation head Using the same Token tensor as input, and through convolution and sigmoid, output a normalized noise response map. .

[0069] Gain Field Refinement Head : Illumination field Noise field With refined transparency By stitching along the channels, applying two layers of convolution followed by Softplus, and then adding a bias of 1, a low-resolution coarse gain is obtained. .

[0070] To avoid the "blocky" artifacts caused by local abrupt changes in Gaussian geometry, Full resolution was restored using bilinear upsampling, with a fixed 5×5 isotropic Gaussian kernel. ( (Non-learnable) spatial smoothing is performed, and finally truncation is performed to [ , The final gain field is obtained by considering the interval [1,4]. .in, Ensure the gain field does not attenuate the input brightness, i.e., only enhance and not darken it. Extensive experiments have shown that this is a reasonable threshold for balancing visibility enhancement and noise amplification.

[0071] The final enhanced image is synthesized from two complementary paths: One is the decoded feature D, which is fused with the encoded features through multi-scale upsampling and channel attention gating, and the downsampled gain field Down(Γ) is convolved with gain injection. Alignment and fusion yield enhanced decoding features. .

[0072] The other is a direct global residual path, through learnable gating. By controlling the blending ratio of the global residual path, the final enhanced image is generated. .in, A reflection-filled output header with a 7×7 convolution is used, and ω is initialized to a small value. This is to ensure structural stability during the initial training phase.

[0073] The physics visualization effect of the GIF module is as follows: Figure 2 As shown, the input low-light image is naturally decomposed into three physically interpretable visual components: intrinsic illumination, spatial gain, and noise, and finally synthesized into a uniform and color-fidelity enhanced result.

[0074] Step 6: Output the enhanced image.

[0075] The calculated enhanced image The image is stored in a memory or directly fed into subsequent visual task processing units, such as object detection networks or image display drivers, to complete the entire process of low-light image enhancement.

[0076] Example 2 This embodiment describes how to obtain the GLFS network parameters used in Embodiment 1 above through unsupervised training. The training process is also executed by the processor and requires the preparation of two separate sets of unpaired images: a low-light image set and a normal-light image set.

[0077] The processor uses the total loss function As the optimization objective, the total loss consists of eight main losses, the meanings and calculation formulas of which are as follows: (1) Combating losses The least squares approach is used to merge the true and false constraints of the multi-scale PatchGAN from two directions into a single adversarial objective. This refers to a darkening generator that transitions from normal light to dim light. This refers to an enhancement generator that transforms weak light into normal light. , These are the corresponding multi-scale discriminators.

[0078] (2) Cross-domain consistency loss The loss incorporates both cycle consistency and identity identity constraints, which are combined into a single loss. Here, η represents the relative weight of the identity sub-item.

[0079] (3) Perceived loss Based on VGG-19 multilayer sensing loss, the 5-layer slicing design is adopted. Activation calculation for each layer Distance accumulates in both directions: (4) Structural similarity loss Structural similarity loss is applied to the cyclic reconstruction.

[0080] (5) Loss of exposure and spatial consistency The goal is to achieve unified exposure control and spatial consistency. Among these, for Average pooling (p=16), four-directional difference operator Target value of local brightness in the enhancement domain Local brightness target value of the dark region The weight of the dark area brightness term in the exposure loss =0.3, the weight of the spatial consistency term in exposure loss. =1.0, dark area image compression .

[0081] (6) Color vector angle loss The RGB triplet of each pixel is treated as a three-dimensional vector, and only its direction (hue) is constrained, not its magnitude (brightness).

[0082] (7) Brightness edge loss ,in, Pick and First, RGB is passed through... Converted to luminance channel, then... Operator extracts gradient magnitude Then in both directions Distance matching enhances the edge of the cyclic domain.

[0083] (8) Physical smoothing loss This combines enhanced TV smoothing, intrinsic lighting field TV smoothing, and Gaussian transparency TV smoothing into a single objective. Among these, The mean of the differences in the horizontal and vertical directions. , This represents the weight of the sub-item.

[0084] The final total loss function is: .

[0085] The weights for each loss are set according to the ablation experiment as follows: .

[0086] Brief explanation of the training process: The GLFS model parameters used in steps 1-5 above, including all learnable weights of the encoder, MGT, GS-ViT, and GIF, were obtained through unsupervised training. The training process used unpaired low-light image sets and normal-light image sets, with the total loss function... End-to-end optimization is performed. Specifically, the generator and two multi-scale discriminators are updated alternately, with the optimizer being Adam, an initial learning rate of 2e-4, and training for 100 epochs. After training, only the generator part is retained for the inference steps described above.

[0087] Example 3 To verify the effectiveness of the proposed GLFS method in unsupervised low-light image enhancement tasks, this embodiment performs qualitative and quantitative evaluations on several publicly available benchmark datasets. All experiments use the exact same network architecture and training parameters as in Embodiments 1 and 2.

[0088] I. Dataset and Evaluation Metrics The experiments employed two widely recognized benchmark datasets: the LOL dataset and the LSRW dataset. LOL is the first publicly available paired dataset meticulously constructed for low-light enhancement tasks, encompassing synthetic and real-world low-light-normal-light corresponding images. LSRW is the first large-scale paired dataset for real-world scenes, containing two independent subsets captured by a Huawei P40 Pro smartphone and a Nikon D7500 DSLR camera, respectively. During training, the paired relationships between images were deliberately broken, placing normal-light and low-light images in two separate subdirectories and randomly combining them into the network without pairing. During testing, the original paired structure was restored, and objective quantitative evaluation was performed using full-reference metrics.

[0089] Evaluation metrics include full-reference metrics: Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Learned Perceptual Patch Similarity (LPIPS); and non-reference metrics: Natural Image Quality Evaluator (NIQE), Luminance Order Error (LOE), Discrete Entropy (DE), and Enhancement Metric Evaluation (EME).

[0090] II. Visual Effects Comparison Figure 5The enhancement effects of various methods on the LOL dataset are demonstrated. As shown in the figure, the enhancement results of methods such as EFI-NET, SCI, RUAS, and ZeroDCE are generally darker and lack sufficient detail recovery; NoiSER has excessive brightness and is accompanied by oversaturation; LIME introduces significant noise and has poor visual quality; EnlightenGAN and PairLIE exhibit fogging and blurring. In contrast, the GLFS method of this invention is closer to the real reference image (Ground Truth) in terms of brightness enhancement, color restoration, and detail preservation, resulting in a natural and clear overall image and the best overall visual effect.

[0091] Figure 6 The second set of comparison images from the LSRW-HUAWEI dataset is shown. The input image is extremely dark overall, with almost no scene details visible. EFI-NET, SCI, RUAS, ZeroDCE, and ZeroDCE++ enhancements are insufficient, and the image remains too dark; EnlightenGAN and NoiSER exhibit significant overexposure, fogging, and color distortion; LIME introduces strong noise; FMR-NET, FRR-NET, and PairLIE improve brightness but still appear grayish. GLFS enhancement results in natural brightness, colors close to the real reference image, and clearer details in vehicles, ground, and background, achieving the best overall effect.

[0092] Figure 7 This section presents the third set of comparative images on the LSRW-NIKON dataset. The input image is low in brightness, and details of the vehicle and ground are unclear. EFI-NET and SCI offer limited brightness improvements and still suffer from being too dark; EnlightenGAN and ChebyLighter exhibit noticeable haze; NoiSER and ZeroIG show overexposure and color oversaturation; LIME is accompanied by noise and color cast; FMR-NET, FRR-NET, and UTV-NET perform relatively stably but with slightly weaker details. GLFS offers moderate brightness, natural colors, clearer vehicle outlines and ground textures, and is closer to the real reference image overall.

[0093] Figure 8 Further comparison of the detail recovery capabilities of various algorithms on the LSRW-HUAWEI dataset was conducted by zooming in on specific areas. ChebyLighter, FMR-NET, and FRR-NET showed good brightness enhancement, but the edges were slightly blurred; RUAS, SCI, ZeroDCE, etc., still had issues with being too dark or having color casts; LIME showed significant noise, and NoiSER was over-enhanced. GLFS was clearer in terms of car body edges, leaf textures, and background details, with less noise, and was the closest to the real reference image overall.

[0094] In summary, the GLFS method of this invention is at a leading level compared with existing methods in subjective visual evaluations such as brightness, color, detail recovery and sharpness, while also having superior performance in suppressing noise and avoiding overexposure.

[0095] III. Comparison of Objective Indicators Table 1 presents the quantitative comparison results on the LOL dataset. GLFS achieved best or near-best results in all seven metrics: PSNR, SSIM, LPIPS, NIQE, LOE, DE, and EME. Specifically, PSNR reached 23.09 dB, SSIM reached 0.88, LPIPS dropped to 0.13, NIQE decreased to 3.71, LOE decreased to 5.44, DE improved to 2.71, and EME improved to 27.15. Compared to existing state-of-the-art methods, PSNR improved by approximately 0.83 dB (compared to 22.26 dB for FRR-NET), SSIM improved by approximately 0.04 (compared to 0.84 for UTV-NET), and LPIPS decreased by 0.01 (compared to 0.14 for UREtinex-NET).

[0096] Table 2 presents the quantitative comparison results on the LSRW-HUAWEI dataset. GLFS achieves a PSNR of 19.76 dB, an SSIM of 0.83, a low LPIPS of 0.31, and a LOE of 7.76. In real-world mobile phone shooting scenarios where low-light enhancement is more challenging, GLFS maintains its leading position.

[0097] Table 3 presents the quantitative comparison results on the LSRW-NIKON dataset. GLFS achieves a PSNR of 19.99 dB, an SSIM of 0.80, a low LPIPS of 0.14, a reduced LOE of 14.19, an improved DE of 1.74, and an improved EME of 15.37. Despite its large number of parameters, GLFS ranks first in both PSNR and SSIM, demonstrating the generalization ability and robustness of the method presented in this invention.

[0098] Note: The bolded values ​​in the table represent the best and second-best results under the current indicators.

[0099] IV. Ablation Experiments and Parameter Sensitivity Analysis Table 4 verifies the effectiveness of the core architecture. Due to the limited receptive field, Base-CNN (pure convolutional network) has an SSIM of only 0.77 and a PSNR of only 20.32; Base-ViT (standard self-attention, no physical prior) improves SSIM to 0.82, but still has noise in dark areas; the GLFS of this invention significantly improves PSNR to 23.09 dB, SSIM to 0.88, and reduces NIQE to 3.71, demonstrating the strong nonlocal perception capability of Transformer and the high complementarity of 2DGS continuous physical light field representation.

[0100] Table 5 verifies the necessity of physical prior modeling. Completely removing the physical prior (without the physical prior) causes the NIQE to rise to 4.11; using an isotropic Gaussian improves the PSNR to 22.43 and the SSIM to 0.86; a fully anisotropic Gaussian that allows elliptical spots in any direction achieves the best performance: PSNR 23.09 and SSIM 0.88, proving the scientific validity of the direction-adaptive Gaussian function superposition.

[0101] Table 6 verifies the necessity of each loss function in the unsupervised training objective. Removing the color vector angle loss causes PSNR to drop to 21.96 and SSIM to drop to 0.79; removing the luminance edge loss causes SSIM to drop to 0.86; removing the exposure and spatial consistency loss or the physical smoothing loss both lead to a decrease in various indicators. The complete objective function achieves the optimal result.

[0102] Table 7 validates the necessity of multi-scale Gaussian labeling. Only 1 / 16 of the single coarse scale yields an SSIM of 0.83 and a PSNR of 22.76; only 1 / 4 of the single fine scale yields an SSIM of 0.86 and a PSNR of 22.82; the multi-scale fused GLFS achieves the best results with an SSIM of 0.88, a PSNR of 23.09, and an NIQE of 3.71, demonstrating the synergistic effect of coarse-scale global illumination smoothing and fine-scale geometric detail correction.

[0103] Table 8 illustrates the trade-off between model complexity and performance. When the number of GS-ViT blocks N increases from 2 to 4, PSNR improves from 18.32 to 23.09 and SSIM from 0.72 to 0.88. Further increasing to 6, while PSNR and SSIM still show slight improvements (23.29 dB and 0.89 respectively), the computational resource overhead increases dramatically. Considering all factors, this invention selects N=4 and H=4 as the default configuration, combining high performance enhancement with deployment potential in resource-constrained environments.

[0104] Table 1

[0105] Table 2

[0106] Table 3

[0107] Table 4

[0108] Table 5

[0109] Table 6

[0110] Table 7

[0111] Table 8

[0112] V. Conclusion Qualitative and quantitative experiments, ablation studies, and parameter sensitivity analyses on the three benchmark datasets mentioned above fully demonstrate the advancement and effectiveness of the GLFS method proposed in this invention in unsupervised low-light image enhancement tasks, achieving state-of-the-art performance in key metrics such as PSNR, SSIM, and LPIPS.

[0113] In other embodiments, in order to adapt to scenarios with different computing resources or different degrees of light degradation, the above embodiments can be modified as follows, and these modifications all fall within the protection scope of the present invention.

[0114] Variation Example 1 The number of scales in the multi-scale Gaussian labeler is adjustable. In Example 1, the MGT uses two scales: a coarse-grained scale of 1 / 16 and a fine-grained scale of 1 / 4. In other implementations, three or more scales can be used depending on the resolution of the input image or the complexity of the scene, with each branch predicting Gaussian parameters for its corresponding scale, and then integrated through multi-scale fusion convolution. This extension can further improve the fitting accuracy between extremely bright highlight regions and extremely deep shadow boundaries.

[0115] Variation Example 2 Adaptive selection of the number of GS-ViT blocks and attention heads. Example 1 uses N=4 blocks and h=4 heads. According to the sensitivity analysis in Table 8, in scenarios with sufficient computing resources, a configuration of N=6, h=4 or N=4, h=8 can be used to obtain a small gain; in mobile or embedded devices, a lightweight configuration of N=2, h=2 can be used. The core of all variants is that the physical bias fusion mechanism based on anisotropic Gaussian Mahalanobis distance is used within the GS-ViT blocks, and this feature remains unchanged.

[0116] Variation Example 3 The weights of the loss function are adaptively adjusted. In Example 1, the weights of each loss term are fixed hyperparameters. In other implementations, uncertainty weighting or a cosine decay / heating strategy based on the number of training iterations can be introduced, for example, by learning trainable task-related noise parameters to dynamically adjust the relative contribution of each loss term, thereby avoiding manual parameter tuning.

[0117] Variation Example 4 Hardware-specific optimizations. In automotive or mobile scenarios, the convolutional operations in the MGT can be fused with the downsampling layer into a single depthwise separable convolutional unit, and the exponential operation in Gaussian affinity calculation can be approximated using a lookup table, thereby reducing computational latency. These optimizations do not change the mathematical essence of the algorithm, but only involve engineering modifications for specific deployments.

[0118] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to specific embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A low-light image enhancement method based on Gaussian light field sputtering-guided self-attention, characterized in that, Includes the following steps: A low-light image to be enhanced is acquired, and the low-light image is input into an encoder for feature extraction to obtain a multi-scale feature map. A multi-scale Gaussian tokenizer is constructed based on the multi-scale feature map to predict the Gaussian geometric parameters and semantic tokens corresponding to each scale. The Gaussian geometric parameters include at least the Gaussian center position, the inverse covariance matrix, and the transparency parameter. The Gaussian geometric parameters are associated with the semantic token and then input into the Gaussian light field sputtering guided self-attention module; During the self-attention calculation process, the Gaussian affinity between tokens is calculated based on the Gaussian geometric parameters, and the Gaussian affinity is used as a physical bias to be introduced into the attention weight calculation to obtain enhanced features that fuse physical light field priors. Reconstruct the continuous spatial gain field based on the enhanced features; The spatial gain field is used to perform brightness correction on the input low-light image to obtain an enhanced image; The network parameters are optimized based on unsupervised training objectives, which include at least color vector angle loss and brightness edge loss.

2. The method according to claim 1, characterized in that, The multi-scale Gaussian labeler includes a coarse-grained branch and a fine-grained branch; the coarse-grained branch is used to extract global illumination structure information from low-resolution feature maps, and the fine-grained branch is used to extract local geometric detail information from high-resolution feature maps. The semantic tokens output from the two branches are fused to obtain the final Gaussian semantic token feature, and the transparency output from the two branches is fused to obtain the final transparency.

3. The method according to claim 2, characterized in that, The Gaussian geometric parameters are generated as follows: the center position is obtained by superimposing the relative offset of the network prediction on the normalized grid; the original value of the network prediction is nonlinearly transformed and truncated to obtain the diagonal and off-diagonal elements of the covariance inverse matrix; the original value of the network prediction is added with a learnable bias and then the transparency is obtained by the Sigmoid function.

4. The method according to claim 1, characterized in that, The Gaussian affinity is obtained by: calculating the Mahalanobis distance based on the relative coordinate difference between any two tokens and the inverse covariance matrix corresponding to the query token; and generating the corresponding Gaussian affinity based on the Mahalanobis distance.

5. The method according to claim 4, characterized in that, The Gaussian affinity is obtained by performing a negative exponential mapping on the Mahalanobis distance.

6. The method according to claim 1, characterized in that, The Gaussian light field sputtering guided self-attention module includes multiple attention heads, each corresponding to an independent learnable temperature parameter; the attention weight is obtained by learning-gated fusion of semantic attention terms and physical bias terms.

7. The method according to claim 1, characterized in that, The Gaussian light field sputtering guided self-attention module also includes a geometry refinement branch, which generates Gaussian parameter residuals based on the current semantic token and uses the Gaussian parameter residuals to iteratively update the Gaussian geometric parameters.

8. The method according to claim 1, characterized in that, The continuous spatial gain field is generated by a Gaussian light field reconstruction module; the Gaussian light field reconstruction module includes an intrinsic illumination prediction head, a noise estimation head, and a gain field refinement head; the gain field refinement head generates the spatial gain field based on the intrinsic illumination field, noise field, and transparency parameters.

9. The method according to claim 1, characterized in that, The unsupervised training objective also includes at least one of exposure consistency loss, spatial consistency loss, and physical field smoothing loss. The color vector angle loss is used to constrain the directional consistency of the corresponding pixel RGB vectors before and after enhancement. The brightness edge loss is used to constrain the edge structure consistency in the brightness channel of the enhanced image and the input image.

10. A low-light image enhancement system based on Gaussian light field sputtering-guided self-attention, characterized in that, include: The image acquisition module is used to acquire the low-light image to be enhanced; The feature extraction module is used to encode the low-light image and generate a multi-scale feature map; The Gaussian parameter generation module is used to generate Gaussian geometric parameters and semantic tokens based on the multi-scale feature map; A Gaussian light field sputtering guided self-attention module is used to generate Gaussian affinity based on the Gaussian geometric parameters and to use the Gaussian affinity as a physical bias in the attention weight calculation. The optical field reconstruction module is used to reconstruct the continuous spatial gain field based on the enhancement features; The image enhancement module is used to perform brightness correction on the low-light image using the spatial gain field and output the enhanced image; Each module collaboratively executes the low-light image enhancement method based on Gaussian light field sputtering guided self-attention as described in any one of claims 1-10.

Images