Low-light image enhancement method and system based on signal-to-noise ratio guided two-branch dynamic fusion

Abstract

The application discloses a low-illumination image enhancement method and system based on signal-to-noise ratio guided double-branch dynamic fusion, and relates to the technical field of image processing. The specific steps comprise generating a signal-to-noise ratio feature map based on an input low-illumination image, and in a double-branch U-Net network, extracting local features and global features in parallel through a convolutional neural network and Mamba at each layer of the U-Net except the deepest layer, and performing dynamic fusion under the guidance of the signal-to-noise ratio feature map. In the deepest layer of the U-Net, only one double-branch cross-attention feature interaction between the I branch and the HV branch is performed to obtain an enhanced feature map. Through the signal-to-noise ratio feature map generation, the dynamic fusion of the local features and the global features based on the signal-to-noise ratio, and the single double-branch cross-attention feature interaction in the deepest layer, the application reduces the parameter quantity and the computational complexity, and improves the detail recovery, noise suppression and overall visual consistency of the low-illumination image.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically to a low-light image enhancement method and system based on signal-to-noise ratio-guided dual-branch dynamic fusion. Background Technology

[0002] In low-light environments, due to limited or uneven illumination, acquired images often suffer from insufficient brightness, decreased contrast, increased noise, and degraded detail, resulting in a significant decline in visual quality and making it difficult to meet the application requirements of subsequent visual perception and image analysis tasks. To improve the imaging quality of low-light images, existing technologies typically enhance them through brightness boosting, noise suppression, and color correction, aiming to restore images with better visual naturalness and recognizability.

[0003] Currently, most low-light image enhancement methods are based on end-to-end enhancement models constructed using deep neural networks. Regarding color space processing, some existing methods directly learn the mapping relationship from low-light images to normal-light images in the RGB (Red, Green, Blue) color space. However, the color channels in the RGB color space are strongly coupled. During enhancement, adjusting one channel often has a ripple effect on other channels, easily leading to color distortion, local oversaturation, and interference between texture details and color information. To mitigate the impact of RGB channel coupling, some methods convert the image to HSV (Hue Saturation Value) color spaces and enhance the brightness-related components separately. Although these methods achieve a certain degree of separation between brightness and color information, the hue component in the HSV color space exhibits circular discontinuities, and the conversion process between different color spaces is usually accompanied by nonlinear errors. Therefore, under extremely low light or complex lighting conditions, problems such as color shift, black artifacts, and unnatural visual effects are still prone to occur. It is evident that existing technologies still struggle to simultaneously achieve brightness enhancement, color naturalness, and preservation of structural information at the color space processing level.

[0004] Besides color space processing issues, existing low-light image enhancement methods also face a difficult trade-off between feature modeling capabilities and computational complexity. One class of methods primarily relies on Convolutional Neural Networks (CNNs) for feature extraction and enhancement. While these methods possess strong local receptive field modeling capabilities and offer advantages in texture detail restoration, their ability to model long-range dependencies, global illumination distribution, and cross-regional contextual information is relatively limited. Therefore, when processing low-light images with complex brightness distributions or high noise levels, they are prone to problems such as uneven enhancement in local areas, noise amplification, and insufficient overall brightness consistency. Another class of methods improves feature representation capabilities by introducing global modeling mechanisms such as Transformers. Although this can improve the modeling effect of global dependencies to some extent, it typically suffers from a large number of parameters, high computational complexity, and high deployment costs, making it difficult to balance enhancement performance with the efficiency requirements of practical applications.

[0005] Furthermore, existing low-light image enhancement methods typically lack explicit characterization of imaging reliability and noise levels at different spatial locations within the image. They often employ a uniform feature extraction method or enhancement strategy to process the entire image. However, signal quality varies significantly across different regions of a low-light image: for regions with good imaging quality and relatively reliable detail, excessive smoothing or over-reliance on global modeling can easily lead to texture detail loss; conversely, for regions with strong noise and weak signal, focusing solely on local detail enhancement can easily result in noise amplification or the generation of false details. Therefore, establishing an effective adaptive control mechanism to address the differences in imaging quality across different spatial regions, thereby coordinating detail restoration, noise suppression, and overall illumination consistency, remains a crucial problem that current technologies urgently need to solve.

[0006] Furthermore, in existing schemes involving the synergistic enhancement of luminance and color structure information, there is room for further optimization in the information interaction methods between different branches. If there is a lack of effective information complementarity mechanisms between branches, it is difficult to simultaneously achieve luminance restoration and structure preservation. On the other hand, frequent complex interactions in shallow, intermediate, or multi-layered network structures can easily introduce additional parameter redundancy and computational burden, hindering model simplification and engineering deployment. Therefore, how to ensure effective complementarity between luminance and color structure information while controlling network complexity is also an aspect that requires further improvement in existing technologies.

[0007] Based on this, existing technologies have at least the following shortcomings: First, existing color space enhancement methods cannot simultaneously take into account color naturalness, structural stability, and visual consistency during the brightness enhancement process; second, existing enhancement methods generally lack explicit perception and adaptive adjustment mechanisms for noise levels and imaging reliability in different spatial regions, thus making it difficult to effectively balance detail restoration and noise suppression; third, in existing network structure designs, it is difficult to balance local feature modeling and global feature modeling, and it is also difficult to achieve a good balance between the information interaction effect and computational complexity between the brightness branch and the color structure branch, thereby limiting the enhancement effect and deployment feasibility of low-light image enhancement methods in complex real-world scenarios. Summary of the Invention

[0008] 1. The technical problem to be solved by the present invention

[0009] To address the problems mentioned in existing technologies, this invention proposes a low-light image enhancement method and system based on signal-to-noise ratio (SNR)-guided dual-branch dynamic fusion. The core technologies include SNR feature map generation, dynamic fusion of local and global features guided by SNR in all layers of U-Net except the deepest layer, and dual-branch cross-attention feature interaction between the HV and I branches performed only once in the deepest layer of U-Net. These technologies aim to solve the problem that existing solutions struggle to simultaneously achieve detail restoration, noise suppression, overall illumination consistency, and computational complexity control. Specifically, HVI color space conversion and HVI inverse transformation with trainable parameter density-k serve as input preprocessing and output restoration steps, respectively.

[0010] 2. Technical Solution To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention proposes a low-light image enhancement method with signal-to-noise ratio guided dual-branch dynamic fusion, comprising the following steps: Obtain the original RGB image, perform HVI color space conversion on the original RGB image to obtain the intensity map and horizontal and vertical color maps; A signal-to-noise ratio (SNR) feature map is generated based on the original RGB image, wherein the SNR feature map is calculated from the grayscale image corresponding to the input low-light image, the noise-free image obtained by mean filtering, and the noise map formed by the difference between the two; Convolutional feature extraction is performed on the intensity map and the HVI map composed of the intensity map and horizontal and vertical color maps to obtain intensity map features and HVI map features of the same dimension. The intensity map features and HVI map features are then input into a pre-constructed dual-branch U-Net network. In each layer of the U-Net except the deepest layer, local features are extracted by convolutional neural network and global features are extracted by Mamba. The local features and global features are then dynamically fused under the guidance of the signal-to-noise ratio feature map. In the deepest layer of U-Net, only one bi-branch cross-attention feature interaction is performed between the I branch and the HV branch, so that the HV branch provides the I branch with the noise-suppressed structural information, and the I branch provides the HV branch with the prior information of the illumination distribution, thus obtaining the enhanced feature map. The enhanced feature map is processed by the HVI inverse transform with trainable parameter density-k to obtain the enhanced image.

[0011] As a further improvement of the present invention, the process of generating the signal-to-noise ratio feature map includes: The input low-light image is converted to grayscale and then denoised using mean filtering to obtain a noise-free image. The noise image is obtained by subtracting the absolute value of the grayscale image corresponding to the input low-light image from the noise-free image. Divide the noise-free image by the noise image to obtain the signal-to-noise ratio feature map; The signal-to-noise ratio feature map is used to characterize the noise intensity and imaging reliability in different regions.

[0012] As a further improvement of the present invention, feature extraction is performed on the intensity map and the HVI map composed of the intensity map and the horizontal and vertical color maps to obtain intensity map features and HVI map features of the same dimension, including: Feature extraction is performed on the intensity map and HVI map based on convolutional layers. The convolutional kernel size in the convolutional layer is 3×3 and the stride is 1. Zero padding is applied to the input through the convolutional layer to obtain intensity map features and HVI map features with consistent dimensions.

[0013] As a further improvement of the present invention, both the I branch and the HV branch of the dual-branch U-Net network include a dynamic fusion module of global features and local features. The intensity map feature enhancement process of the I branch is as follows: The I branch performs at least one downsampling on the intensity map features to obtain multi-scale brightness features; Local brightness features are obtained by extracting local features from multi-scale brightness features using a convolutional neural network. Global feature modeling of multi-scale brightness features is performed based on Mamba to obtain global brightness features; Based on the signal-to-noise ratio feature map, the local and global brightness features are dynamically fused to obtain the brightness fusion feature map. The local feature extraction, global feature modeling, and dynamic fusion processes are repeated in all layers of U-Net except the deepest layer; The fused brightness features are upsampled at least once to obtain the I-branch feature map; The HV branch performs HVI graph feature enhancement processing as follows: The HV branch performs at least one downsampling on the HVI map features to obtain multi-scale color structure features; Local feature extraction of multi-scale color structure features is performed based on convolutional neural networks to obtain local color structure features; Global feature modeling of multi-scale color structure features is performed based on Mamba to obtain global color structure features; Based on the signal-to-noise ratio feature map, the local and global features of the color structure are dynamically fused to obtain the color structure fusion feature map. The local feature extraction, global feature modeling, and dynamic fusion processes are repeated in all layers of U-Net except the deepest layer; The fused color structure features are upsampled at least once to obtain the HV branch feature map.

[0014] As a further improvement of the present invention, the process of dynamically fusing local features and global features based on the signal-to-noise ratio feature map includes: The signal-to-noise ratio feature map is normalized to obtain a spatial weight map, and the local features extracted by the convolutional neural network and the global features extracted by Mamba are weighted and summed using the spatial weight map to obtain a fused feature map. Increasing the weight of local features in regions with high signal-to-noise ratio and increasing the weight of global features in regions with low signal-to-noise ratio achieves synergistic optimization of detail restoration, noise suppression, and overall illumination consistency.

[0015] As a further improvement of the present invention, the bi-branch cross-attention feature interaction process performed in the deepest layer of U-Net includes: The deep luminance fusion feature of the I branch is used as the first query vector, and the deep color structure fusion feature of the HV branch is used as the first key vector and the first value vector, so that the I branch can obtain the structural information after noise suppression. The deep color structure fusion features of the HV branch are used as the second query vector, and the deep brightness fusion features of the I branch are used as the second key vector and the second value vector, so that the HV branch can obtain prior information on illumination distribution.

[0016] As a further improvement of the present invention, the dual-branch cross-attention feature interaction is only set in the deepest layer of U-Net, so as to complete the complementarity of structural information and illumination prior information at the position where the context information is most sufficient, and avoid parameter redundancy and computational overhead caused by repeated interaction in shallow and intermediate layers. The I-branch feature map and the HV-branch feature map after the completion of the bi-branch cross-attention feature interaction are concatenated to obtain an enhanced feature map. The enhanced feature map is processed by the HVI inverse transform with trainable parameter density-k to obtain the enhanced image.

[0017] Secondly, this invention proposes a low-light image enhancement system with signal-to-noise ratio guided dual-branch dynamic fusion, comprising: The preprocessing module is used to acquire the original RGB image and perform HVI color space conversion on the original RGB image to obtain the intensity map and horizontal and vertical color maps; The signal-to-noise ratio feature map generation module is used to generate a signal-to-noise ratio feature map based on the original RGB image; The feature extraction and dynamic fusion module is used to extract features from the intensity map and the HVI map composed of the intensity map and horizontal and vertical color maps, to obtain intensity map features and HVI map features of the same dimension. The intensity map features and HVI map features are then input into a pre-constructed dual-branch U-Net network. In each layer of the U-Net except the deepest layer, local and global features are extracted in parallel through convolutional neural networks and Mamba, and then dynamically fused under the guidance of the signal-to-noise ratio feature map. The deepest layer single-time bi-branch cross-attention interaction module is used to perform bi-branch cross-attention feature interaction only once on the I branch and HV branch in the deepest layer of U-Net, so as to achieve complementarity between structural information and illumination prior information and obtain enhanced feature maps. The output module is used to perform HVI inverse transform processing on the enhanced feature map with trainable parameter density-k to obtain the enhanced image.

[0018] Thirdly, the present invention proposes a low-light image enhancement device with signal-to-noise ratio feature map-guided local-to-global fusion and deepest single-branch interaction, comprising a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the low-light image enhancement method with signal-to-noise ratio feature map-guided local-to-global fusion and deepest single-branch interaction as described above.

[0019] Fourthly, the present invention proposes a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the low-light image enhancement method described above, which uses signal-to-noise ratio feature map-guided local-to-global fusion and deepest single-branch interaction.

[0020] Compared with existing technologies, the low-light image enhancement system and method with signal-to-noise ratio guided dual-branch dynamic fusion provided by the present invention has the following beneficial effects: Compared with existing technologies, the low-light image enhancement method and system with signal-to-noise ratio guided dual-branch dynamic fusion provided by this invention has the following beneficial effects: Firstly, this invention employs HVI color space conversion to decouple the original RGB image into an intensity map (I) and horizontal and vertical color maps (HV). The enhanced result is then reconstructed through an inverse HVI transform with a trainable parameter `density-k`. These technical features reduce color crosstalk caused by strong coupling between RGB color space channels and avoid color shifts and black artifacts that easily occur in HSV color space processing. Therefore, compared to existing RGB or HSV enhancement schemes, this invention can better maintain color naturalness and structural stability while improving brightness.

[0021] Secondly, this invention constructs a signal-to-noise ratio (SNR) feature map using grayscale images, mean-filtered noise-free images, and differential noise maps. This explicitly models the imaging reliability and noise intensity at different spatial locations, and uses this SNR feature map as the control basis for subsequent feature fusion. Through these technical features, differentiated enhancement strategies can be adopted for high and low SNR regions. Therefore, compared to existing solutions that use a uniform enhancement method, this invention can more effectively suppress noise amplification while preserving effective detail information.

[0022] Thirdly, in the dual-branch U-Net network, this invention extracts local features through convolutional neural networks and global features through Mamba in all layers except the deepest layer, and dynamically fuses them based on the signal-to-noise ratio (SNR) feature map. Through these technical features, high SNR regions prioritize detail restoration, while low SNR regions prioritize noise suppression and overall illumination consistency. Therefore, compared to existing solutions that rely solely on CNNs or Transformer-like mechanisms, this invention can better balance local detail and global enhancement effects while maintaining relatively controllable computational complexity.

[0023] Fourth, this invention only sets up a single bi-branch cross-attention feature interaction between the I branch and the HV branch in the deepest layer of the U-Net, enabling structural information and prior lighting information to complement each other at the point where the contextual information is most abundant. Through these technical features, the information coordination capability between branches is improved, while avoiding parameter redundancy and computational burden caused by multi-layered repeated interactions. Therefore, compared to multi-layered interaction schemes or simple cascaded schemes, this invention achieves lower model complexity while ensuring information complementarity.

[0024] Fifth, this invention achieves a better balance between detail restoration, noise suppression, color naturalness, and overall visual consistency by decoupling the HVI color space, guiding the signal-to-noise ratio feature map, dynamically fusing local and global features, and coordinating the interaction of the deepest single-branch cross-attention features. Therefore, it is more suitable for image enhancement applications in complex and realistic low-light scenes. Attached Figure Description

[0025] Figure 1 A flowchart illustrating a low-light image enhancement method based on signal-to-noise ratio guided dual-branch dynamic fusion according to an embodiment of this disclosure is shown. Figure 2 The diagram shows the network structure of a low-light image enhancement method with signal-to-noise ratio guided dual-branch dynamic fusion according to an embodiment of this disclosure. Figure 3 This diagram illustrates the structure of the SNR-guided global and local feature dynamic fusion module in an embodiment of this disclosure. Figure 4 A structural diagram of the cross-attention interaction module in an embodiment of this disclosure is shown; Figure 5 The following is a visual comparison of the proposed method of this invention with existing models on a paired dataset in an embodiment of this disclosure. Referring to Table 1, a is a low-light image, b is model 1, c is model 2, d is model 3, e is the proposed method of this invention, and f is a reference image. Figure 6 The following is a visualization of the effects of the SNR module, Mamba module, and CAI module on the low-light enhancement network in the embodiments of this disclosure. Referring to Table 2, a is a low-light image, b is model A, c is model B, d is model C, e is the method proposed in this invention, and f is a reference image. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0027] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0028] See Figure 1 and Figure 2 This invention proposes a low-light image enhancement method with signal-to-noise ratio guided dual-branch dynamic fusion, comprising the following steps: The original RGB image is obtained, and the original RGB image is converted to HVI color space to obtain intensity map I and horizontal and vertical color maps HV; where, in this scheme, intensity map I is represented by I, and horizontal and vertical color maps HV are represented by HV. A signal-to-noise ratio feature map is generated based on the original RGB image; Feature extraction is performed on the intensity map and the HVI map composed of the intensity map and horizontal and vertical color maps to obtain intensity map features and HVI map features of the same dimension. Local features and global features are extracted in each layer of U-Net except the deepest layer through convolutional neural network and Mamba respectively, and dynamic fusion is performed under the guidance of signal-to-noise ratio feature map. In the deepest layer of U-Net, only one bi-branch cross-attention feature interaction is performed on the I branch and the HV branch to obtain the enhanced feature map; The enhanced feature map is processed by the HVI inverse transform with trainable parameter density-k to obtain the enhanced image.

[0029] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0030] Step 1, in this embodiment, involves converting the original RGB image to the HVI color space as follows: First, the original RGB image is acquired. Then, the original RGB image is converted to HVI color space to obtain an intensity map containing scene illumination information and horizontal and vertical color maps containing scene color and structural information.

[0031] In this embodiment, the above conversion is only used as network input preprocessing, so that the intensity information and color structure information are respectively entered into the subsequent enhancement process.

[0032] Step 2, the process of generating the signal-to-noise ratio feature map in the embodiment is as follows: First, the input low-light image is converted to grayscale, and then mean filtering is used to remove noise, resulting in a noise-free image. The calculation formula is as follows:

[0033] In the formula: This represents the input low-light image; This represents the noise-free image after denoising. (·) indicates grayscale processing; (·) indicates mean filtering.

[0034] Secondly, the absolute value of the grayscale image corresponding to the input low-light image is subtracted from that of the noise-free image to obtain the noise image, and the calculation formula is as follows:

[0035] In the formula: N This represents a noise graph; |·| represents absolute value operations.

[0036] Finally, the noise-free image is divided by the noise image to obtain the signal-to-noise ratio feature map, which is calculated as follows:

[0037] In the formula: SNR The signal-to-noise ratio (SNR) feature map is used to characterize the noise intensity and imaging reliability in different regions, and serves as an explicit control basis for the subsequent dynamic fusion of local and global features.

[0038] Step 3: In the disclosed embodiment of the present invention, a convolutional layer with a kernel size of 3×3, a stride of 1, and a zero-padding mode is used to perform initial feature extraction on the intensity map and the HVI map composed of the intensity map and the horizontal and vertical color maps, so as to obtain intensity map features and HVI map features with the same dimensions.

[0039] Step 4, the process of extracting and fusing global and local features in this embodiment is as follows: For the I branch, the intensity map features are first downsampled at least once to obtain multi-scale brightness features; then, in each layer, a convolutional neural network is used to extract local brightness features to recover fine-grained brightness details.

[0040] Meanwhile, Mamba is used to perform long-range dependency modeling on the multi-scale brightness features to obtain global brightness features, which characterize the overall brightness trend and the relationship between light and dark transitions.

[0041] Subsequently, the local and global brightness features are dynamically fused based on the signal-to-noise ratio feature map. In regions with high signal-to-noise ratio, the weight of the local brightness features is increased to enhance texture and local contrast recovery. In regions with low signal-to-noise ratio, the weight of the global brightness features is increased to suppress noise amplification and maintain the consistency of illumination distribution.

[0042] The fused brightness features are upsampled at least once to obtain the I-branch feature map.

[0043] For the HV branch, the color structure input features are first downsampled at least once to obtain multi-scale color structure features; then, in each layer, convolutional neural networks are used to extract local color structure features to restore local color details and edge structures.

[0044] Simultaneously, Mamba is used to perform long-range dependency modeling on the multi-scale color structure features to obtain global color structure features, thereby acquiring cross-regional color consistency and structural context information.

[0045] Subsequently, the local and global color structure features are dynamically fused based on the signal-to-noise ratio feature map. In high signal-to-noise ratio regions, local color details extracted by the convolutional neural network are preferentially retained. In low signal-to-noise ratio regions, global structural information extracted by Mamba is preferentially retained to reduce the influence of noise and false colors in dark areas.

[0046] The core computational relationship for dynamic fusion based on the signal-to-noise ratio feature map can be expressed as follows:

[0047] In the formula: Indicates fusion features, This represents the spatial weight map obtained by normalizing the signal-to-noise ratio feature map. This represents the local features extracted by the convolutional neural network. This represents the global features extracted by Mamba. Through the above calculations, more local texture and edge details are preserved in high signal-to-noise ratio regions, while more global illumination consistency and structural context are preserved in low signal-to-noise ratio regions, thus distinguishing it from enhancement methods such as fixed-weight fusion or simple feature cascading.

[0048] Step 5: The interaction process between HV branch features and I branch features in this embodiment is as follows: In the deepest layer of U-Net, a cross-attention interaction module is used to perform a two-branch cross-attention feature interaction on the I-branch features and HV-branch features. Before entering the deepest layer, the features of each layer have already achieved reliability after noise suppression and rationality after illumination enhancement through dynamic fusion based on the signal-to-noise ratio. This enables the HV branch to provide denoised structural information to the I branch, and enables the I branch to provide prior information on illumination distribution to the HV branch. This reduces the number of network parameters while ensuring that the information of the two branches is complementary.

[0049] The single bi-branch cross-attention feature interaction performed only in the deepest layer of U-Net can be represented as:

[0050]

[0051] In the formula: This represents the deep features of the I branch after interaction. The deep features of the HV branch after interaction are represented, and CAI(·) represents the bi-branch cross-attention interaction operator. By performing the above interaction only once in the deepest layer, structural information and illumination prior can be complemented at the position with the most sufficient contextual information, avoiding parameter redundancy and computational overhead caused by repeated interactions in multiple layers.

[0052] After the dual-branch cross-attention feature interaction, the HV branch features are upsampled to obtain the HV branch feature map, and the I branch feature map is concatenated with the HV branch feature map to obtain the enhanced feature map.

[0053] Step 6: Perform HVI inverse transformation based on trainable parameter density-k on the enhanced feature map, convert the corrected HVI map into RGB color space, and obtain the enhanced image.

[0054] The inverse transformation described above serves as the output restoration process, where density-k is used to adjust the color density of the dark areas and the brightness restoration amplitude so that the enhanced result retains natural colors and stable brightness when returned to the RGB space.

[0055] As shown in Table 1 and Figure 5 As shown, the proposed method is compared with existing models on three datasets: LOLv1 (Low Light Dataset Version 2 - Real Acquisition Subset), LOLv2-Real (Low Light Dataset Version 2 - Real Acquisition Subset), and LOLv2-syn (Low Light Dataset Version 2 - Synthetic Subset). Model 1 is SNR-Net (Signal-to-Noise Ratio Aware Low Light Image Enhancement Network), Model 2 is RetinexFormer (Retinex-based Single-Stage Transformer Low Light Image Enhancement Network), and Model 3 is CIDNet (Color and Intensity Decoupling Network). The performance of the models is evaluated by peak signal-to-noise ratio and structural similarity.

[0056] Table 1. Performance comparison of the proposed method with existing models on paired datasets.

[0057] As shown in Table 1, on the LOLv2-Real dataset, the method proposed in this invention achieves the best results in both peak signal-to-noise ratio (PSNR) and structural similarity. On the LOLv1 dataset, the method proposed in this invention is optimal in terms of structural similarity, but its PSNR is slightly weaker than that of Model 1 and Model 2. On the LOLv2-syn dataset, the method proposed in this invention achieves the best results in both PSNR and structural similarity. Although the method proposed in this invention has a larger number of parameters compared to current mainstream models, its computational complexity is significantly lower than other methods, except for Model 3.

[0058] pass Figure 5 The comparison shows that the algorithm proposed in this invention exhibits superior performance in image texture detail restoration and noise suppression, and also possesses strong generalization ability. (See Table 1 and...) Figure 5Overall, the method proposed in this invention outperforms the mainstream methods compared in terms of various indicators and visualization effects, demonstrating a better enhancement effect.

[0059] like Figure 6 As shown, Model A is trained using only the U-Net structure and the deepest layer with a cross-attention interaction module, i.e., it does not include the SNR module and the Mamba module, and only promotes information interaction between the HV branch and the I branch in the deepest layer of U-Net; Model B only includes the SNR module and the Mamba module, i.e., it includes a complete global and local feature dynamic fusion module, which aims to better extract global contextual information and local information of the image; Model C adds a CAI module and a global and local feature fusion module without signal-to-noise ratio guidance, which aims to extract global contextual information and local information of the image while promoting information interaction between the HV branch and the I branch.

[0060] from Figure 6 The experimental results show that Model A is trained using only the U-Net structure and the deepest layer with a cross-attention interaction module, without including the SNR and Mamba modules. The results demonstrate that Model A achieves improvements in both peak signal-to-noise ratio (PSNR) and structural similarity on both the LOLv1 and LOLv2-syn datasets. This is because, in RGB images after HVI transformation, the HV branch features and I branch features represent hue saturation and brightness intensity information, respectively, and these two features are highly complementary. Introducing the CAI module into the deepest layer of U-Net for a cross-attention interaction promotes information exchange and supplementation between the two branches, thereby improving image detail, texture, and brightness to some extent. However, because Model A lacks the SNR guidance provided by the SNR module and does not incorporate the Mamba module to model global contextual information, it still has limitations in detail recovery and noise suppression.

[0061] Table 2. Impact of the SNR module, Mamba module, and CAI module of this invention on low-light enhancement networks.

[0062] As shown in Table 2, Model B only includes the SNR module and the Mamba module, meaning it incorporates a complete global and local feature dynamic fusion module, aiming to better extract global and local contextual information from images. Experimental results show that Model B further improves both peak signal-to-noise ratio (PSNR) and structural similarity on the LOLv1 and LOLv2-syn datasets. This is because the SNR module generates a signal-to-noise ratio feature map, guiding the feature fusion process in different regions. This allows the network to adaptively adjust the fusion weights of global and local information in the HV and I branches based on the reliability of the image regions. Simultaneously, the Mamba module effectively extracts global features and dynamically fuses them with the local features extracted by the CNN under the guidance of SNR. Therefore, Model B not only enhances image detail recovery capabilities but also effectively suppresses noise interference in low-light scenes, thereby improving the overall image enhancement effect.

[0063] Model C incorporates a CAI module and a global and local feature fusion module without SNR guidance, aiming to extract global and local contextual information from images while promoting information interaction between the HV and I branches. Experimental results show that Model C achieves good enhancement results on the LOLv1 and LOLv2-syn datasets. This is because the CAI module can promote cross-attention interaction between HV and I branch features, enhancing the complementary fusion of information from different branches; simultaneously, the global and local feature fusion module also helps extract richer contextual information, improving the network's ability to represent complex scenes. However, due to the lack of SNR guidance in Model C, the feature fusion process cannot adaptively adjust according to the signal quality of different regions, thus its enhancement effect in noisy or complex texture regions is still somewhat limited.

[0064] The complete method proposed in this invention simultaneously possesses three mechanisms: SNR-guided dynamic fusion, CAI deepest-layer interaction, and Mamba global modeling. On the LOLv1 and LOLv2-syn datasets, it achieves optimal results in both peak signal-to-noise ratio (PSNR) and structural similarity, and its overall performance surpasses that of models A, B, and C. This is because the CAI module effectively promotes full interaction between the HV and I branches in the deepest layer of U-Net, enhancing the complementary expression of features from different branches; the SNR module adaptively guides the fusion process using SNR feature maps, making the network focus more on information transmission in high-reliability regions and suppressing noise interference in low-reliability regions; and the Mamba module further models the global contextual information of the image and forms dynamic collaboration with local features. The synergy of these three mechanisms results in superior performance in detail restoration, structure preservation, and noise suppression.

[0065] In summary, the experiments demonstrated that the SNR, Mamba, and CAI modules designed in this invention exhibit excellent synergistic gain. By introducing a signal-to-noise ratio-guided dynamic fusion mechanism, combined with the deepest-layer dual-branch cross-attention interaction and global context modeling capabilities, this invention significantly improves the performance of low-light image enhancement networks. Although there is an increase in the number of parameters and computational complexity, the experimental results show that this multi-module collaborative design plays a crucial role in improving image enhancement effects.

[0066] Based on the same inventive concept, this invention also provides a low-light image enhancement system with signal-to-noise ratio-guided dual-branch dynamic fusion. Since the principle of this low-light image enhancement system with signal-to-noise ratio feature map-guided local-global fusion and deepest single dual-branch interaction is similar to the aforementioned low-light image enhancement method with signal-to-noise ratio feature map-guided local-global fusion and deepest single dual-branch interaction, the implementation of this low-light image enhancement system with signal-to-noise ratio feature map-guided local-global fusion and deepest single dual-branch interaction can refer to the implementation of the low-light image enhancement method with signal-to-noise ratio feature map-guided local-global fusion and deepest single dual-branch interaction, and the repeated parts will not be described again.

[0067] In specific implementation, the low-light image enhancement system with signal-to-noise ratio feature map-guided local-to-global fusion and deepest-layer single-pass bi-branch interaction provided in this embodiment of the invention specifically includes: The preprocessing module is used to acquire the original RGB image and perform HVI color space conversion on the original RGB image to obtain the intensity map and horizontal and vertical color maps; The signal-to-noise ratio feature map generation module is used to generate a signal-to-noise ratio feature map based on the original RGB image; The feature extraction and dynamic fusion module is used to extract features from the intensity map and the HVI map composed of the intensity map and horizontal and vertical color maps, to obtain intensity map features and HVI map features of the same dimension. The intensity map features and HVI map features are then input into a pre-constructed dual-branch U-Net network. Local features and global features are extracted in each layer of the U-Net except for the deepest layer through convolutional neural networks and Mamba, respectively, and then dynamically fused under the guidance of the signal-to-noise ratio feature map. The deepest layer single-time bi-branch cross-attention interaction module is used to perform bi-branch cross-attention feature interaction only once on the I branch and HV branch in the deepest layer of U-Net, so as to achieve complementarity between structural information and illumination prior information and obtain enhanced feature maps. The output module is used to perform HVI inverse transform processing on the enhanced feature map with trainable parameter density-k to obtain the enhanced image.

[0068] Accordingly, embodiments of the present invention also provide a low-light image enhancement device with signal-to-noise ratio feature map-guided local-to-global fusion and deepest single-branch interaction, including a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the low-light image enhancement method with signal-to-noise ratio feature map-guided local-to-global fusion and deepest single-branch interaction as provided in the embodiments of the present invention.

[0069] For more detailed information on the above methods, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0070] Accordingly, embodiments of the present invention also provide a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the low-light image enhancement method provided in the embodiments of the present invention, which uses signal-to-noise ratio feature map-guided local-to-global fusion and deepest single-branch interaction.

[0071] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems, devices, and storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0072] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0073] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0074] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0075] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the scope of the invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0076] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be appropriately combined to form other embodiments that can be understood by those skilled in the art. The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

Claims

1. A low-light image enhancement method based on signal-to-noise ratio guided dual-branch dynamic fusion, characterized in that, Includes the following steps: S1, acquire the original RGB image, perform HVI color space conversion on the original RGB image to obtain the intensity map I and the horizontal and vertical color maps HV; S2, Generate a signal-to-noise ratio (SNR) feature map based on the original RGB image. The SNR feature map is calculated based on the grayscale image of the original low-light image, the clean image after mean filtering and noise reduction, and the noisy image constructed from the difference between the two. The SNR feature map is used to characterize the noise intensity and imaging reliability of different regions. S3, perform convolutional feature extraction on the intensity map and the HVI map composed of the intensity map and horizontal and vertical color maps to obtain intensity map features and HVI map features of the same dimension, and input the intensity map features and HVI map features into a pre-constructed dual-branch U-Net network. In each layer of the U-Net except the deepest layer, local features are extracted by convolutional neural network and global features are extracted by Mamba, and the local features and global features are dynamically fused under the guidance of the signal-to-noise ratio feature map. S4. In the deepest layer of U-Net, only one bi-branch cross-attention feature interaction is performed on the intensity map I branch features and the horizontal and vertical color map HV branch features. This allows the horizontal and vertical color map HV branch to provide noise-suppressed structural information to the intensity map I branch, and the intensity map I branch to provide prior information on illumination distribution to the horizontal and vertical color map HV branch, thus obtaining the enhanced feature map. S5. Perform HVI inverse transformation based on trainable parameter density-k on the enhanced feature map to obtain the enhanced image.

2. The low-light image enhancement method with signal-to-noise ratio guided dual-branch dynamic fusion according to claim 1, characterized in that, The generation process of the signal-to-noise ratio feature map in S2 includes: S2.1: Convert the input low-light image to a grayscale image and use mean filtering to remove noise, resulting in a noise-free image; S2.2: Subtract the absolute value of the grayscale image corresponding to the input low-light image from the noise-free image to obtain the noise image; S2.3: Divide the noise-free image by the noise image to obtain the signal-to-noise ratio feature map.

3. The low-light image enhancement method based on signal-to-noise ratio guided dual-branch dynamic fusion according to claim 1, characterized in that, Feature extraction is performed on the intensity map I and the HVI map composed of the intensity map I and the horizontal and vertical color maps HV, resulting in intensity map features and HVI map features of the same dimension, including: Feature extraction is performed on the intensity map and HVI map based on convolutional layers. The convolutional kernel size in the convolutional layer is 3×3 and the stride is 1. Zero padding is applied to the input through the convolutional layer to obtain intensity map features and HVI map features with consistent dimensions.

4. The low-light image enhancement method with signal-to-noise ratio guided dual-branch dynamic fusion according to claim 1, characterized in that, The intensity map I branch and the horizontal and vertical color map HV branch of the dual-branch U-Net network in S3 both include a dynamic fusion module of global and local features; The intensity map feature enhancement process for the intensity map I branch is as follows: The intensity map I branch performs at least one downsampling on the intensity map features to obtain multi-scale brightness features; Local features of brightness are extracted from multi-scale brightness features using a convolutional neural network to obtain local brightness features; global features of brightness are modeled using Mamba to obtain global brightness features; local and global brightness features are dynamically fused based on the signal-to-noise ratio feature map to obtain a fused brightness feature map; the local feature extraction, global feature modeling, and dynamic fusion process are repeated in all layers of U-Net except the deepest layer; the fused brightness features are upsampled at least once to obtain the intensity map I branch feature map; The process of enhancing HVI map features through the horizontal and vertical color map HV branches is as follows: The horizontal and vertical color map HV branches downsample the HVI map features at least once to obtain multi-scale color structure features; local feature extraction is performed on the multi-scale color structure features based on a convolutional neural network to obtain local color structure features; global feature modeling is performed on the multi-scale color structure features based on Mamba to obtain global color structure features; local and global color structure features are dynamically fused according to the signal-to-noise ratio feature map to obtain a fused color structure feature map; the local feature extraction, global feature modeling, and dynamic fusion process are repeated in each layer of U-Net except for the deepest layer; the fused color structure features are upsampled at least once to obtain the horizontal and vertical color map HV branch feature map.

5. The low-light image enhancement method with signal-to-noise ratio guided dual-branch dynamic fusion according to claim 4, characterized in that, The process of dynamically fusing local and global features based on the signal-to-noise ratio feature map includes: The signal-to-noise ratio feature map is normalized to obtain a spatial weight map. The spatial weight map is then used to perform a weighted summation of the local features extracted by the convolutional neural network and the global features extracted by Mamba to obtain a fused feature map. In regions with high signal-to-noise ratio, the weight of local features is increased to enhance detail recovery, while in regions with low signal-to-noise ratio, the weight of global features is increased to suppress noise amplification and maintain consistent illumination distribution.

6. The low-light image enhancement method with signal-to-noise ratio guided dual-branch dynamic fusion according to claim 1, characterized in that, The bi-branch cross-attention feature interaction process performed in the deepest layer of U-Net includes: The deep brightness fusion feature of the intensity map I branch is used as the first query vector, and the deep color structure fusion feature of the horizontal and vertical color maps HV branches is used as the first key vector and the first value vector for cross-attention calculation, so that the intensity map I branch can obtain the structural information after noise suppression. The deep color structure fusion features of the horizontal and vertical color maps HV branch are used as the second query vector, and the deep brightness fusion features of the intensity map I branch are used as the second key vector and the second value vector for cross-attention calculation, so that the HV branch can obtain prior information on illumination distribution.

7. The low-light image enhancement method with signal-to-noise ratio guided dual-branch dynamic fusion according to claim 1, characterized in that, The dual-branch cross-attention feature interaction is only set in the deepest layer of U-Net; The dual-branch cross-attention feature interaction set only in the deepest layer of U-Net is used to complete the information complementarity between the horizontal and vertical color map HV branch and the intensity map I branch at the position where the context information is most sufficient, and avoids parameter redundancy and computational overhead caused by repeated interaction in shallow and intermediate layers. The intensity map I branch feature map after the completion of the bi-branch cross-attention feature interaction is concatenated with the horizontal and vertical color map HV branch feature map to obtain the enhanced feature map.

8. A low-light image enhancement system based on signal-to-noise ratio-guided dual-branch dynamic fusion, applicable to the low-light image enhancement method based on signal-to-noise ratio-guided dual-branch dynamic fusion as described in any one of claims 1-7, characterized in that, include: Preprocessing module: used to acquire the original RGB image and perform HVI color space conversion on the original RGB image to obtain intensity map I and horizontal and vertical color maps HV; Signal-to-noise ratio feature map generation module: used to generate signal-to-noise ratio feature maps based on the original RGB image; Feature extraction and dynamic fusion module: This module is used to perform convolutional feature extraction on intensity map I and HVI map composed of intensity map I and horizontal and vertical color maps HV, to obtain intensity map features and HVI map features of the same dimension. The intensity map features and HVI map features are then input into a dual-branch U-Net network. In each layer of U-Net except the deepest layer, local features are extracted by convolutional neural network and global features are extracted by Mamba. Dynamic fusion is then performed under the guidance of signal-to-noise ratio feature map. The deepest layer single-time bi-branch cross-attention interaction module is used to perform a bi-branch cross-attention feature interaction only on the intensity map I branch features and the horizontal and vertical color map HV branch features in the deepest layer of U-Net. This allows the horizontal and vertical color map HV branch to provide noise-suppressed structural information to the intensity map I branch, and the intensity map I branch to provide prior information on illumination distribution to the horizontal and vertical color map HV branch, resulting in an enhanced feature map. Output module: Used to perform HVI inverse transform processing on the enhanced feature map with trainable parameter density-k to obtain the enhanced image.

9. A low-light image enhancement device based on signal-to-noise ratio feature map-guided local-to-global fusion and deepest-layer single-pass bi-branch interaction, characterized in that, The system includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the low-light image enhancement method as described in any one of claims 1 to 7, which involves signal-to-noise ratio feature map-guided local-to-global fusion and deepest single-branch interaction.

10. A computer-readable storage medium, characterized in that, This is used to store a computer program, wherein when executed by a processor, the computer program implements a low-light image enhancement method based on signal-to-noise ratio-guided dual-branch dynamic fusion, comprising: acquiring an original RGB image and performing HVI color space conversion to obtain an intensity map and horizontal and vertical color maps; generating a signal-to-noise ratio feature map based on the original RGB image; performing convolutional feature extraction on the intensity map features and HVI map features and inputting them into a dual-branch U-Net network; extracting local and global features respectively through convolutional neural networks and Mamba in each layer of the U-Net except for the deepest layer, and performing dynamic fusion under the guidance of the signal-to-noise ratio feature map; performing dual-branch cross-attention feature interaction between the intensity map I branch and the horizontal and vertical color map HV branch only once in the deepest layer of the U-Net; and obtaining the enhanced image through an HVI inverse transform based on a trainable parameter density-k.

Images