Low-light image enhancement method based on multi-branch feature fusion

Abstract

The invention discloses a low-light image enhancement method based on multi-branch feature fusion. The method comprises the steps of obtaining low-light and normal-light images, and dividing training and test sets; processing the normal light image to obtain a single-channel color-enhanced, brightness-enhanced and texture-enhanced component graph; obtaining a multi-branch feature fusion low-light image enhancement neural network model; and inputting a to-be-measured low-light image into the model to enhance the backbone network, and outputting a restored image under a normal illumination condition. According to the invention, a color dynamic calibration module is provided, and a plug-and-play color correction normal form is provided for the model; an external source wound modulation module is provided, and key information can be focused while interference information such as noise is restrained. A brand new shared convolution is provided, so that the model can adaptively adjust the characteristic response intensity of each channel. An exogenous creative shared residual block structure is constructed, so that the enhanced backbone network can utilize the internal feature abstraction ability and the external auxiliary information guide advantage at the same time.

Description

Technical Field

[0001] This invention belongs to the field of image processing and computer vision technology, and specifically relates to a low-light image enhancement method based on multi-branch feature fusion. Background Technology

[0002] In the field of computer vision and image processing, image quality degradation in low-light environments has long constrained the intelligent development of scenarios such as nighttime surveillance, autonomous driving, and medical imaging. With the continuous evolution of visual perception technology, intelligent devices are placing higher demands on the robustness of image enhancement algorithms under low-light conditions. However, low-light images generally suffer from physical limitations such as contrast attenuation, Gaussian noise contamination, and dynamic range compression. Traditional enhancement methods based on Retinex theory, due to their over-reliance on manually designed reflectance estimation models, inevitably introduce halo artifacts and color distortion problems.

[0003] While breakthroughs in deep learning technology have brought about a paradigm shift in the field, existing single-branch network architectures still face technical bottlenecks such as multi-scale feature fusion mismatch and insufficient utilization of contextual information under complex lighting conditions. In particular, it is difficult to balance detail and brightness recovery, as well as the optimization goal of color fidelity, in extremely low-light scenes. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to provide a low-light image enhancement method based on multi-branch feature fusion.

[0005] To achieve the above objectives, the low-light image enhancement method based on multi-branch feature fusion provided by the present invention includes the following steps performed in sequence:

[0006] 1) Select a publicly available standard dataset consisting of low-light images and corresponding normal light images. Then, perform preprocessing on all low-light and normal light images, including invalid data removal, normalization, and fusion of physical model priors, to enhance the generalization ability of the images and obtain preprocessed low-light and normal light images. After that, divide all preprocessed low-light and normal light images into training and test sets according to the proportion.

[0007] 2) Process the preprocessed normal light image obtained in step 1) to obtain single-channel color enhancement component map, brightness enhancement component map and texture enhancement component map respectively;

[0008] 3) Construct a low-light image enhancement neural network model; the low-light image enhancement neural network model consists of three parallel color restoration sub-networks, brightness restoration sub-networks, and texture restoration sub-networks cascaded with an enhancement backbone network. The enhancement backbone network includes a multi-scale feature extraction branch and a texture auxiliary branch, and a target loss function is designed. Then, the single-channel color enhancement component map, brightness enhancement component map, and texture enhancement component map obtained in step 2) are input into the color restoration sub-network, brightness restoration sub-network, and texture restoration sub-network, respectively. At the same time, the preprocessed low-light images in the training set are input into the enhancement backbone network, thereby training the low-light image enhancement neural network model and evaluating it with a test set, finally obtaining a multi-branch feature fusion low-light image enhancement neural network model.

[0009] 4) After preprocessing the low-light image to be tested according to the method in step 1), input it into the enhanced backbone network of the low-light image enhancement neural network model obtained in step 3) to output the restored image under normal lighting conditions.

[0010] In step 1), the standard dataset is the LOLV1 standard dataset; the ratio of the training set to the test set is 8:2.

[0011] In step 2), the method for processing the preprocessed normal light image obtained in step 1) to obtain the single-channel color enhancement component map, brightness enhancement component map, and texture enhancement component map is as follows:

[0012] The method for obtaining a single-channel color enhancement component map is as follows:

[0013] First, the preprocessed normal light image is converted from the RGB color space to the YCbCr color space, where R, G, and B represent the three channels of the normal light image, Y represents color brightness, and Cb and Cr represent the blue and red intensity shifts, respectively. Then, the color brightness Y, the blue intensity shift Cb, and the red intensity shift Cr are linearly superimposed using weighting coefficients of 0.3, 0.3, and 0.4 to generate a single-channel color component enhancement image C, as shown in the following formula:

[0014] Y = 0.299R + 0.587G + 0.114B

[0015] Cb=128-0.16873R-0.331264G+0.5B

[0016] Cr=128+0.5R-0.418688G-0.081312B

[0017] C = 0.3Y + 0.3Cb + 0.4Cr

[0018] The method for obtaining the luminance enhancement component map is as follows:

[0019] First, the maximum value of the input tensor of the preprocessed normal light image in the channel dimension is obtained, which is the luminance component V in the HSV color space, where H represents chroma, S represents saturation, and V represents luminance. The luminance component V is then constrained to the interval [0.0, 1.0] so that the essential features of the luminance component V can be realized by obtaining the maximum values ​​of the three RGB channels, where R, G, B∈[0,1] represent the normalized red, green, and blue channel intensities, respectively. Finally, the luminance component enhancement image L is obtained, as shown in the following formula:

[0020] L = V = max(R, G, B)

[0021] The method for obtaining the texture enhancement component map is as follows:

[0022] First, the preprocessed normal light image is converted from the RGB color space to the YUV color space and the Y channel is accurately extracted, where Y represents luminance and U and V both represent chrominance. Then, the Scharr operator is used for fine processing to obtain the texture enhancement component map T, also known as the gradient map.

[0023] 4. The low-light image enhancement method based on multi-branch feature fusion according to claim 1, characterized in that: in step 3), the color restoration sub-network adopts an encoder-decoder architecture, wherein the encoder uses a multi-level convolutional structure for deep feature extraction, each level is composed of a convolutional layer, an instance normalization layer and a ReLU activation function, and a color dynamic calibration module is embedded between the second and third level convolutional structures; the color dynamic calibration module includes a global average pooling layer and two 1x1 convolutional layers; the decoder gradually restores the spatial resolution and restores the number of channels through two bilinear upsampling operations, and after each upsampling, it is fused with the feature map of the corresponding level of the encoder through skip connections;

[0024] The brightness restoration subnetwork includes an encoder and a decoder. Both the encoder and decoder employ a multi-level convolutional structure. The encoder also embeds two downsampling modules, a feature enhancement module, and a bottleneck layer between layers. The decoder includes two upsampling modules and a convolutional output layer. The feature enhancement module consists of a 1x1 convolutional layer, an instance normalization layer, and a ReLU activation function. The bottleneck layer consists of a 1x1 convolutional layer, an instance normalization layer, and a ReLU activation function. In the decoder, the enhanced features of the corresponding layer of the encoder are fused using a concatenation method, and the fused features are processed through additional convolutional layers.

[0025] The texture restoration subnetwork comprises two main structures: a texture restoration preprocessing module and an encoder-decoder. The texture restoration preprocessing module includes three two-layer adaptive compression attention modules, which are used to fuse the texture-aided restoration information from the enhanced backbone network with the texture information initially extracted from the texture enhancement component map T through a 3×3 convolutional layer and the ReLU activation function in an element-wise addition manner. The two-layer adaptive compression attention module includes an adaptive squeezing compression module SE.

[0026] Both the encoder and decoder employ a multi-level convolutional structure; the encoder includes an initial feature extraction layer, a multi-scale feature extractor, two downsampling modules, inter-layer embedded residual dense blocks, a channel attention mechanism module, and an output layer; the decoder uses bilinear upsampling combined with skip connections to recover spatial resolution and fuse low-level details and high-level semantic information from the encoder.

[0027] The enhanced backbone network includes a texture-assisted sub-network and a multi-scale feature extraction sub-network; the texture-assisted sub-network uses a multi-level convolutional structure to extract texture-assisted information; the multi-scale feature extraction sub-network uses an encoder-decoder architecture to receive recovery information from three recovery sub-networks to enhance the low-light image;

[0028] The encoder in the multi-scale feature extraction branch includes an initial 3×3 convolutional layer, a dual-path exogenous modulation residual block constructed based on the exogenous modulation module, a single-path exogenous modulation residual block constructed based on the exogenous modulation module, and a DownSample module; the encoder in the texture-assisted branch includes an initial 3×3 convolutional layer, a single-path exogenous modulation residual block constructed based on the exogenous modulation module, and a DownSample module; seven sets of exogenous shared residual blocks are embedded between the encoder and the decoder. This module, based on the above-mentioned exogenous modulation module, integrates improved shared separable convolutions to further enhance the multi-scale context.

[0029] The low-light image enhancement method based on multi-branch feature fusion provided by this invention has the following beneficial effects:

[0030] 1) A dynamic color calibration module is proposed in the color restoration subnetwork, which significantly improves color restoration capability through an end-to-end feature-level adaptive mechanism. This module first extracts channel-level statistical features using global average pooling, then generates affine transformation parameters through a lightweight bottleneck structure, and finally constrains the output range using a sigmoid function, achieving feature space color correction through channel-adaptive dynamic affine transformation. This module provides a plug-and-play color correction paradigm for image enhancement networks and offers an effective solution to the color distortion problem in low-light image enhancement.

[0031] 2) An exogenous modulation module is proposed, which employs an innovative feature modulation structure to achieve dynamic adaptive enhancement of input features through multi-mechanism collaboration. Its core uses a channel adapter to uniformly map single-channel or multi-channel modulation features to the target number of channels, ensuring compatibility; simultaneously, input features, after being modulated together with adapted features, generate parameters to achieve feature fusion. Based on this, a novel exogenous modulation residual block is proposed in the backbone network, enabling better integration of auxiliary enhancement information obtained from the two parallel sub-networks of color and brightness recovery with the enhanced backbone network. This allows the final result to focus on key information while suppressing noise and other interference.

[0032] 3) A novel shared convolution is proposed, which achieves significant performance improvements while maintaining an extremely low number of parameters compared to traditional shared convolution. By introducing a learnable channel-wise scaling factor (adding only C parameters), the model can adaptively adjust the feature response intensity of each channel, greatly enhancing feature diversity while effectively improving training stability and nonlinear modeling capabilities. In the enhanced backbone network, this innovative shared convolution is combined with an exogenous generative modulation module to construct another novel exogenous generative shared residual block structure. The combined effect of the two residual blocks enables the enhanced backbone network to simultaneously utilize the internal feature abstraction capabilities and the advantages of external auxiliary information guidance, achieving the optimal balance between noise suppression and key detail preservation in low-light image enhancement tasks, ultimately outputting a restored image with significantly improved structural fidelity and visual naturalness. Attached Figure Description

[0033] Figure 1 This is a structural diagram of the low-light image enhancement neural network model with multi-branch feature fusion in this invention.

[0034] Figure 2 This is a diagram of the color restoration subnetwork structure in this invention.

[0035] Figure 3 This is a structural diagram of the color dynamic calibration module in this invention.

[0036] Figure 4 This is a diagram of the brightness recovery subnetwork structure in this invention.

[0037] Figure 5 This is a diagram of the texture restoration subnetwork structure in this invention.

[0038] Figure 6 This is a diagram of the enhanced backbone network structure in this invention.

[0039] Figure 7 This is a structural diagram of the exogenous generation modulation module in this invention.

[0040] Figure 8This is a structural diagram of the dual-path externally generated modulation residual block in this invention.

[0041] Figure 9 This is a structural diagram of the single-channel externally generated modulation residual block in this invention.

[0042] Figure 10 This is a schematic diagram of the shared residual block generated by the external source in this invention.

[0043] Figure 11 This is a schematic diagram of the low-light image to be tested in this invention.

[0044] Figure 12 This is a schematic diagram of the restored image in this invention. Detailed Implementation

[0045] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0046] The low-light image enhancement method based on multi-branch feature fusion provided by this invention includes the following steps performed in sequence:

[0047] 1) Select a publicly available standard dataset consisting of low-light images and corresponding normal light images. Then, perform preprocessing on all low-light and normal light images, including invalid data removal, normalization, and fusion of physical model priors, to enhance the generalization ability of the images and obtain preprocessed low-light and normal light images. After that, divide all preprocessed low-light and normal light images into training and test sets according to the proportion.

[0048] In this invention, the standard dataset used is the LOLV1 standard dataset; the ratio of the training set to the test set is 8:2.

[0049] 2) Process the preprocessed normal light image obtained in step 1) to obtain single-channel color enhancement component map, brightness enhancement component map and texture enhancement component map respectively;

[0050] The method for obtaining a single-channel color enhancement component map is as follows:

[0051] First, the preprocessed normal light image is converted from the RGB color space to the YCbCr color space, where R, G, and B represent the three channels of the normal light image, Y represents color luminance, and Cb and Cr represent the blue and red intensity shifts, respectively. Then, the color luminance Y, blue intensity shift Cb, and red intensity shift Cr are linearly superimposed using weighting coefficients of 0.3, 0.3, and 0.4 to generate a single-channel color component enhancement map C, aiming to integrate luminance and chromaticity information to meet the needs of extracting color restoration information; the formula is as follows:

[0052] Y = 0.299R + 0.587G + 0.114B

[0053] Cb=128-0.16873R-0.331264G+0.5B

[0054] Cr=128+0.5R-0.418688G-0.081312B

[0055] C = 0.3Y + 0.3Cb + 0.4Cr

[0056] The method for obtaining the luminance enhancement component map is as follows:

[0057] First, the maximum value of the input tensor of the preprocessed normal light image in the channel dimension is obtained, which is the luminance component V in the HSV color space, where H represents chroma, S represents saturation, and V represents luminance. The luminance component V is then constrained to the interval [0.0, 1.0] so that the essential features of the luminance component V can be realized by obtaining the maximum values ​​of the three RGB channels, where R, G, B∈[0,1] represent the normalized red, green, and blue channel intensities, respectively. Finally, the luminance component enhancement image L is obtained, as shown in the following formula:

[0058] L = V = max(R, G, B)

[0059] The method for obtaining the texture enhancement component map is as follows:

[0060] First, the preprocessed normal light image is converted from the RGB color space to the YUV color space, and the Y channel is accurately extracted, where Y represents luminance, and U and V both represent chrominance. This is because the Y channel in the YUV color space has the characteristics of focusing luminance information and significantly reducing the interference of color noise on gradient calculation. Meanwhile, to enhance the directional sensitivity of the texture enhancement component map, this invention uses the Scharr operator (an optimized Sobel operator, which is an optimized operator for image edge detection) for detailed processing. This is because the Scharr operator has a more prominent response in the 45° direction compared to the Sobel operator, and its template coefficients show a more significant difference. This characteristic makes the Scharr operator more sensitive to weak edges in low-light images, thereby effectively improving the ability to capture image details, resulting in the texture enhancement component map T, also known as the gradient map.

[0061] 3) Constructing such Figure 1The low-light image enhancement neural network model shown is composed of three parallel sub-networks—color restoration, brightness restoration, and texture restoration—cascaded with an enhancement backbone network. The enhancement backbone network includes a multi-scale feature extraction branch and a texture auxiliary branch, and a target loss function is designed. Then, the single-channel color enhancement component map, brightness enhancement component map, and texture enhancement component map obtained in step 2) are input into the color restoration sub-network, brightness restoration sub-network, and texture restoration sub-network, respectively. At the same time, the preprocessed low-light images in the training set are input into the enhancement backbone network, thereby training the low-light image enhancement neural network model and evaluating it with a test set, finally obtaining a multi-branch feature fusion low-light image enhancement neural network model.

[0062] like Figure 2 As shown, the color restoration sub-network adopts an encoder-decoder architecture, which obtains color restoration information in the single-channel color enhancement component map C through a multi-level feature extraction method. The encoder uses a multi-level convolutional structure for deep feature extraction, with each level consisting of a convolutional layer, an instance normalization layer, and a ReLU activation function. Two convolutional layers with a stride of 2 progressively downsample the input single-channel color enhancement component map C to 1 / 4 of its original size, while simultaneously increasing the number of channels. Furthermore, a method is embedded between the second and third level convolutional structures, such as... Figure 3 The color dynamic calibration module shown employs an innovative feature modulation structure, including a global average pooling layer and two 1x1 convolutional layers. It uses the global average pooling layer to obtain channel-level statistical information, which is then processed by the two 1x1 convolutional layers to generate scaling and offset parameters. The core of the color dynamic calibration module uses a channel adapter to uniformly map single-channel or multi-channel modulation features to the target number of channels, ensuring compatibility. Simultaneously, the input single-channel color enhancement component map C, after being modulated together with the adapted features, generates parameters to achieve feature fusion. The decoder gradually restores the spatial resolution and the number of channels through two bilinear upsampling operations. After each upsampling, it is fused with the feature map of the corresponding layer of the encoder through skip connections, effectively combining high-level semantic information and low-level detailed features. Finally, it outputs color restoration information and feeds it into the multi-scale feature extraction branch in the enhanced backbone network.

[0063] like Figure 4As shown, the luminance restoration sub-network includes an encoder-decoder for extracting luminance restoration information from the luminance enhancement component map L. Both the encoder and decoder employ a multi-level convolutional structure. The encoder also embeds two downsampling modules, a feature enhancement module, and a bottleneck layer between layers. The decoder includes two upsampling modules and a convolutional output layer. First, the two downsampling modules use 3x3 convolutions with a stride of 2 to progressively extract and compress features. After each downsampling module, a feature enhancement module consisting of a 1x1 convolutional layer, an instance normalization layer, and a ReLU activation function is added to improve non-linear expressive power. A bottleneck layer consisting of a 1x1 convolutional layer, an instance normalization layer, and a ReLU activation function is embedded after the feature enhancement module for feature processing, which can reduce computational parameters. While reducing quantity and model complexity, the model retains and extracts effective information as much as possible. Resolution is then gradually restored using two upsampling modules with 1x1 convolutions combined with bilinear interpolation. In the decoder, concatenation (rather than addition) is used to fuse the enhanced features from the corresponding encoder layer. Additional convolutional layers process the fused features, thus better preserving and integrating multi-level information. The entire brightness restoration sub-network extensively uses instance normalization layers and ReLU activation functions to stabilize training and introduce non-linearity, forming an efficient feature extraction and reconstruction model. Finally, a convolutional output layer outputs brightness restoration information, which is then fed into the multi-scale feature extraction path and texture auxiliary branch in the enhanced backbone network.

[0064] like Figure 5 As shown, the texture restoration subnetwork includes two main structures: a texture restoration preprocessing module and an encoder-decoder. The texture restoration preprocessing module includes three two-layer adaptive compressed attention modules, which are used to fuse the texture-aided restoration information from the enhanced backbone network with the texture information obtained by the texture enhancement component map T through a 3×3 convolutional layer and the ReLU activation function in an element-wise addition manner.

[0065] The dual-layer adaptive compression attention module includes an adaptive squeezing and compression module (SE), which dynamically enhances key texture features through a channel attention mechanism (based on global averaging / max pooling channel weight learning) and a spatial attention mechanism (establishing inter-pixel dependencies). In particular, after acquiring the channel attention mechanism, the SE further optimizes it. This is because the channel attention mechanism has a more fundamental regulatory role in the discriminativeness and robustness of feature representation—in visual tasks, different channels often correspond to visual representations at different semantic levels. When the target region and the background have color similarity, spatial attention may fail due to local pixel-level similarity, while channel attention can dynamically adjust channel weights through global statistics, effectively enhancing discriminative feature channels and suppressing redundant or interfering channels.

[0066] The fused texture information obtained after processing by the texture restoration preprocessing module is fed into the encoder-decoder. Both the encoder and decoder employ multi-level convolutional structures to perform in-depth mining of texture restoration information from the input texture enhancement component map T. The encoder includes an initial feature extraction layer, a multi-scale feature extractor, two downsampling modules, inter-layer embedded residual dense blocks, a channel attention mechanism module, and an output layer. First, the encoder captures the fused texture information through the initial feature extraction layer. Then, it uses a multi-scale feature extractor with parallel 1x1, 3x3, 5x5, and 7x7 convolutional kernels to capture texture patterns of different sizes. Finally, it uses residual dense blocks to promote feature reuse through a progressive feature growth strategy and dense connections to enhance texture representation capabilities. Then, two downsampling modules are used to gradually reduce the feature map size to expand the receptive field. At the same time, a channel attention mechanism (generating channel weights through global pooling and fully connected networks) is introduced at a deeper level to optimize feature representation. The decoder uses bilinear upsampling combined with skip connections to restore spatial resolution and fuse low-level details and high-level semantic information from the encoder. Then, residual dense blocks and channel attention modules are used to further enhance texture reconstruction capabilities. Finally, the output layer generates texture enhancement features through convolution and adds the input texture enhancement component map T to the processed feature map through global residual connections to ensure that texture enhancement does not affect the overall image structure. Finally, texture restoration information is obtained and fed into the multi-scale feature extraction branch in the enhancement backbone network.

[0067] The enhanced backbone network includes a texture-assisted branch and a multi-scale feature extraction branch, such as... Figure 6 As shown, the upper part is the texture-assisted sub-path, which uses a multi-level convolutional structure to extract texture-assisted restoration information; the lower part is the multi-scale feature extraction sub-path, which uses an encoder-decoder architecture to receive restoration information from three restoration sub-networks to enhance the low-light image.

[0068] The encoder in the multi-scale feature extraction branch includes an initial 3×3 convolutional layer, a dual-channel exogenous modulation residual block constructed based on the exogenous modulation module, a single-channel exogenous modulation residual block constructed based on the exogenous modulation module, and a DownSample module; the encoder in the texture-assisted branch includes an initial 3×3 convolutional layer, a single-channel exogenous modulation residual block constructed based on the exogenous modulation module, and a DownSample module; the encoder fuses color and brightness recovery information through the dual-channel exogenous modulation residual block constructed based on the exogenous modulation module; the texture-assisted branch only incorporates brightness recovery information into the single-channel exogenous modulation residual block, thus focusing on gradient feature extraction; the encoder and... The multi-level convolutional structure of the texture-assisted sub-path first extracts basic features through an initial 3×3 convolutional layer and a ReLU activation function. Then, the multi-scale feature extraction sub-path fuses luminance and color restoration information at the original resolution using two dual-channel exogenous modulation residual blocks, while the texture-assisted sub-path fuses luminance restoration information using only two single-channel exogenous modulation residual blocks. Next, it downsamples to a 1 / 2 scale using a DownSample module. The multi-scale feature extraction sub-path then uses two extended-channel single-channel exogenous modulation residual blocks to perform deep feature fusion on the feature information obtained from the upper dual-channel exogenous modulation residual blocks; the texture-assisted sub-path uses two extended-channel single-channel exogenous modulation residual blocks to continue fusing only luminance restoration information. Both sub-paths then downsample to a 1 / 4 scale again using a DownSample module. Finally, the texture-assisted sub-path incorporates the obtained texture-assisted restoration information into the texture restoration sub-network, while the multi-scale feature extraction sub-path incorporates the fused feature information obtained from the encoder into seven sets of shared exogenous modulation residual blocks for further cross-membrane fusion.

[0069] The exogenously generated modulation residual block is based on, for example, Figure 7 The exogenous generative modulation module shown realizes cross-modal feature interaction. Its principle is to generate dynamic parameters based on the recovery information and combine them with the spatial-channel attention mechanism to realize conditional feature transformation and feature fusion.

[0070] The exogenous modulation module obtains its exogenous modulation features through external input features, and its mathematical formalization is expressed by a phased formula as follows:

[0071]

[0072] in, For external input features, C 1×1 Channel-adaptive convolution.

[0073] The exogenous modulation module uses a channel adapter (1×1 convolution) to uniformly map single-channel or multi-channel modulation features to the target number of channels to ensure compatibility. External input features, after being standardized by group normalization (GroupNorm), participate in modulation together with the adapted features: the modulation features undergo nonlinear transformation through a double convolutional layer, and their global information is adaptively pooled and convolved with a 1×1 layer to generate three sets of dynamic modulation parameters (γ, β, α), as shown in the following formula:

[0074] γ,β,α=split(P·GAP·F P (F adapt ))

[0075] F P =conv 3×3 ·Relu·conv 3×3

[0076] P = conv 1×1 GAP

[0077] Where γ is the scaling modulation parameter, β is the translation modulation parameter, α is the gating modulation parameter, and GAP is the global average pooling.

[0078] Channel attention and spatial attention are applied to the modulation features in parallel, modeling inter-channel dependencies and spatial importance distributions respectively, as shown in the following formula:

[0079] Channel attention:

[0080] Spatial attention: A s (F)=σ(S2·Relu·S1(F))

[0081] Where σ is the sigmoid activation function, S1 is the first spatial convolution operation that compresses the channels, and S2 is the second spatial convolution operation that compresses the channels to 1.

[0082] The final modulation process integrates standardized features, dynamic parameters, and dual attention weights: First, it combines affine transformation (scaling modulation parameter γ controls scaling, and translation modulation parameter β controls translation) with residual design (scaling modulation parameter γ is Sigmoided and then incremented by 1 to retain the original information). Then, it performs overall weighting with gated modulation parameter α (Tanh constraint range). Finally, it multiplies element-wise with the channel / spatial attention map to achieve refined feature selection, as shown in the following formula:

[0083]

[0084] Among them, X main The main input feature is represented by ⊙, which indicates element-wise multiplication. GN represents element-wise multiplication of space and channel, and group normalization.

[0085] Subsequently, based on the exogenous creation modulation module, as follows was constructed: Figure 8 The dual-channel externally generated modulation residual block shown and Figure 9 The single-channel external-source generated modulation residual block shown is innovative in the dual-channel external-source generated modulation residual block, which is based on the dual-feature fusion process of its dual-channel external-source modulation architecture and multi-stage feature integration mechanism. The difference between the single-channel and dual-channel external-source generated modulation residual blocks is that the former only fuses a single feature.

[0086] The dual-path externally generated modulation residual block, given the main input features With two external features F1 and F2, the computation flow strictly follows:

[0087] First, feature preprocessing is performed:

[0088] H0 = ReLU(C 3×3 (X))

[0089] Among them, C 3×3 For convolutional layers with the same padding size, extract basic feature representations.

[0090] Next, cascaded dual modulation is employed:

[0091] H1 = EGMM(H0, F1)

[0092] H2 = EGMM(H1, F2)

[0093] Each dual-path externally generated modulation residual block executes:

[0094] Exogenous feature channel adaptation: F adapt =A channel (F)

[0095] Dynamic parameter creation: (γ,β,α)=P·GAP·F P (F adapt )

[0096] Feature transformation: X mod =tanh(α)☉(GN(X)☉(σ(γ)+1)+β)

[0097] Dual attention weighting:

[0098] Channel Refinement:

[0099] Channel attention (C) with a compression ratio r = 8 (down) :C→C / 8,C (up) :C / 8→C) Recalibrate the characteristic response.

[0100] Dual-path fusion:

[0101]

[0102] Where [·, ·] represents concatenation of channel dimensions. Achieve cross-path feature integration.

[0103] Use the ReLU activation function to preserve non-linear expressive power:

[0104] Y = ReLU(H4)

[0105] Subsequently, seven sets of exogenous generative shared residual blocks are embedded between the encoder and decoder. This module, based on the aforementioned exogenous generative modulation module, incorporates improved shared separable convolutions to further enhance the multi-scale context. The structure of each set of exogenous generative shared residual blocks is as follows: Figure 10 As shown.

[0106] In the design of shared separable convolutions, a parameter-sharing convolutional kernel architecture is first constructed, and a dynamic per-channel scaling factor mechanism is introduced to enhance the feature representation capability of the convolutional layers while maintaining lightweight characteristics. This exogenously generated shared residual block employs a Kaiming initialization strategy to optimize weight distribution, and combined with an optional batch normalization layer to improve training stability, achieving an effective balance between computational efficiency and model performance.

[0107] This can be formally described through mathematical modeling as follows:

[0108] Define the fundamental convolution kernel tensor W∈R 1×1×K×K A channel-by-channel weight matrix is ​​generated through channel expansion operations. in Indicates broadcast extension, 1 C (where C is a unit vector), combined with a learnable channel scaling factor Γ∈R C×1×1×1 Achieve dynamic feature enhancement This design, employing a channel-wise Hadamard product, significantly reduces the parameter complexity of traditional depthwise separable convolution from an order of magnitude dependent on the square of the number of channels and the square of the kernel size to an order of magnitude linearly dependent only on the square of the kernel size and the number of channels. Simultaneously, a composite feature transform is constructed through the synergistic effect of a nonlinear activation function and a batch normalization layer. This ensures that the model can achieve a dynamic balance between capacity and computational efficiency.

[0109] The decoder employs a multi-stage progressive structure, achieving image reconstruction through feature fusion and dynamic modulation. First, the deep feature map output from the encoder undergoes 1×1 convolutional dimensionality reduction, followed by bilinear interpolation upsampling to restore spatial resolution. In the intermediate feature processing stage, texture restoration information from the texture restoration sub-network is introduced for cross-layer fusion. Subsequently, single-channel and dual-channel externally generated modulation residual blocks are used together to perform multi-level dynamic modulation, achieving feature adaptive transformation for illumination perception, chromaticity adaptation, and texture detail, ultimately outputting an enhanced image I. out .

[0110] The target loss function is designed to optimize the low-light image enhancement neural network model; the target loss function includes the L1 loss function, the SSIM loss function, and the PSNR loss function.

[0111] The use of the L1 loss function (mean absolute error) is primarily based on several key advantages: First, compared to the L2 loss function (mean squared error), the L1 loss function is less sensitive to outliers (such as extreme brightness values ​​or noisy pixels). Low-light images often contain significant sensor noise, especially in dark areas. The L2 loss function squares the error of these noisy pixels, causing the model to overemphasize noise restoration rather than overall structure, potentially resulting in oversmoothing or artifacts. The L1 loss function, however, assigns linear weights to all pixel errors, avoiding excessive penalty for noise and contributing to visually more natural and textured results. Second, the L1 loss function tends to produce sparser errors, which helps to better preserve image edges and details while suppressing noise, crucial for recovering subtle structures lost in low light. Third, the gradient of the L1 loss function is constant at ±1 during optimization, making it less susceptible to large gradient values. Training is typically more stable, contributing to more robust model convergence. Therefore, the L1 loss function can more evenly handle different regions of the image, especially the information-rich dark areas, and is more aligned with the goal of low-light enhancement in terms of noise suppression, detail preservation, and visual quality enhancement.

[0112]

[0113] Among them, I out I represents the enhanced image, and I represents the normal light image.

[0114] The SSIM loss function better preserves the structural information of an image by introducing a structural similarity metric. It can improve image brightness while better preserving the texture, edges, and overall structure of the original image, avoiding artifacts and detail blurring problems common in traditional methods.

[0115] L ssim =1-SSIM(I out -I)

[0116] Among them, I out I represents the enhanced image, and I represents the normal light image.

[0117] The PSNR loss function, which uses peak signal-to-noise ratio as its optimization objective, can effectively improve the pixel-level similarity between the enhanced image and the reference image. Combining it with a perceptual loss function (based on a VGG network) can compensate for the PSNR loss function's shortcomings in perceptual quality assessment.

[0118] L p =[1-PSNR(I out -I)]+L perceptual

[0119] Among them, I out I represents the enhanced image, and I represents the normal light image.

[0120] The target loss function is:

[0121]

[0122] During the training process of the low-light image enhancement neural network model using the training set, the low-light image enhancement neural network model is converged by minimizing the target loss function, thus obtaining the trained low-light image enhancement neural network model. Then, low-light images from the test set are input into the enhanced backbone network of the trained low-light image enhancement neural network model for model evaluation, ultimately obtaining a multi-branch feature fusion low-light image enhancement neural network model.

[0123] 4) As Figure 11 The low-light image to be tested, as shown, is preprocessed according to the method in step 1) and then input into the enhanced backbone network of the multi-branch feature fusion low-light image enhancement neural network model obtained in step 3). The output is as follows: Figure 12 The restored image is shown under normal lighting conditions.

Claims

1. A low-light image enhancement method based on multi-branch feature fusion, characterized in that: The low-light image enhancement method based on multi-branch feature fusion comprises the following steps performed in sequence: 1) Select a public standard data set composed of low-light images and normal-light images corresponding to the low-light images one by one, then perform preprocessing on all low-light images and normal-light images, including invalid data removal, normalization and physical model prior fusion, to enhance the generalization ability of the images, obtain preprocessed low-light images and normal-light images, and then divide all preprocessed low-light images and normal-light images into a training set and a test set according to a proportion; 2) Process the preprocessed normal-light images obtained in step 1) respectively to obtain a single-channel color enhancement component map, a brightness enhancement component map and a texture enhancement component map; 3) Construct a low-light image enhancement neural network model; the low-light image enhancement neural network model is composed of three parallelly arranged color recovery subnetworks, brightness recovery subnetworks and texture recovery subnetworks and a enhancement main network, wherein the enhancement main network comprises a multi-scale feature extraction branch and a texture auxiliary branch, a target loss function is designed, then the single-channel color enhancement component map, the brightness enhancement component map and the texture enhancement component map obtained in step 2) are input into the color recovery subnetworks, the brightness recovery subnetworks and the texture recovery subnetworks respectively, and the preprocessed low-light images in the training set are input into the enhancement main network, thereby training the low-light image enhancement neural network model and evaluating it with the test set, and finally obtaining a multi-branch feature fusion low-light image enhancement neural network model; 4) After preprocessing the to-be-tested low-light image according to the method in step 1), input it into the enhancement main network of the multi-branch feature fusion low-light image enhancement neural network model obtained in step 3), and output a restored image under normal lighting conditions.

2. The low-light image enhancement method based on multi-branch feature fusion according to claim 1, characterized in that: In step 1), the standard data set adopts the LOLV1 standard data set; the proportion of the training set and the test set is 8:

2.

3. The low-light image enhancement method based on multi-branch feature fusion according to claim 1, characterized in that: In step 2), the method of processing the preprocessed normal-light images obtained in step 1) respectively to obtain a single-channel color enhancement component map, a brightness enhancement component map and a texture enhancement component map is as follows: The method of obtaining a single-channel color enhancement component map is as follows: First, convert the preprocessed normal-light image from an RGB color space to a YCbCr color space, wherein R, G and B represent three channels of the normal-light image, Y represents color brightness, Cb and Cr represent blue and red concentration offset respectively, then perform linear superposition on the color brightness Y, the blue concentration offset Cb and the red concentration offset Cr through weighting coefficients 0.3, 0.3 and 0.4 to generate a single-channel color component enhancement map C, and the formula is as follows: Y = 0.299R + 0.587G + 0.114B Cb = 128 - 0.16873R - 0.331264G + 0.5B Cr = 128 + 0.5R - 0.418688G - 0.081312B C = 0.3Y + 0.3Cb + 0.4Cr The method of obtaining a brightness enhancement component map is as follows: First, the maximum value of the input tensor of the pre-processed normal light image in the channel dimension, i.e. the brightness component V in the HSV color space, is obtained, wherein H represents chroma, S represents saturation, and V represents brightness, and the brightness component V is constrained to the interval [0.0, 1.0] so that the essential characteristics of the brightness component V can be realized by obtaining the maximum value of the RGB three channels, wherein R, G, B∈[0, 1] represent the normalized red, green and blue channel intensities respectively, and finally the brightness component enhancement map L is obtained, and the formula is as follows: L = V = max(R, G, B) The method for obtaining the texture enhancement component map is as follows: First, the pre-processed normal light image is converted from the RGB color space to the YUV color space and the Y channel is accurately extracted, wherein Y represents brightness, and U and V both represent chroma; then a Scharr operator is used for detailed processing to obtain a texture enhancement component map T, also known as a gradient map.

4. The low-light image enhancement method based on multi-branch feature fusion according to claim 1, characterized in that: In step 3), the color restoration subnetwork adopts an encoder-decoder architecture, wherein the encoder adopts a multi-level convolution structure for deep feature extraction, each level is composed of a convolution layer, an instance normalization layer and a ReLU activation function, and a color dynamic calibration module is embedded between the second and third convolution structures; The color dynamic calibration module includes a global average pooling layer and two 1x1 convolution layers; The decoder gradually restores the spatial resolution and restores the number of channels through two times of bilinear upsampling operations, and after each upsampling, the features of the corresponding level of the encoder are fused through a skip connection; The brightness restoration subnetwork includes an encoder-decoder; both the encoder and the decoder adopt a multi-level convolution structure, wherein the encoder additionally embeds two down-sampling modules, a feature enhancement module and a bottleneck layer between the levels, and the decoder contains two up-sampling modules and a convolution output layer; the feature enhancement module is composed of a 1x1 convolution layer, an instance normalization layer and a ReLU activation function; the bottleneck layer is composed of a 1x1 convolution layer, an instance normalization layer and a ReLU activation function; in the decoder, the enhanced features of the corresponding level of the encoder are fused in a splicing manner, and the fused features are processed through an additional convolution layer; The texture restoration subnetwork includes a texture restoration preprocessing module and an encoder-decoder as two main structures; The texture restoration preprocessing module includes three double-layer adaptive compression attention modules, which are used to fuse the texture auxiliary restoration information from the enhanced backbone network with the texture information obtained by preliminarily extracting the texture enhancement component map T through a 3x3 convolution layer and a ReLU activation function in an element-by-element addition manner; the double-layer adaptive compression attention module is provided with an adaptive squeeze compression module SE; Both the encoder and the decoder adopt a multi-level convolution structure; The encoder includes an initial feature extraction layer, a multi-scale feature extractor, two down-sampling modules, a residual dense block embedded between levels, a channel attention mechanism module and an output layer; the decoder uses bilinear upsampling combined with a skip connection to restore the spatial resolution and fuse the low-level details and high-level semantic information of the encoder; The enhanced backbone network includes a texture auxiliary branch and a multi-scale feature extraction branch; The texture auxiliary branch adopts a multi-level convolution structure to extract texture auxiliary information; the multi-scale feature extraction branch adopts an encoder-decoder architecture to receive recovery information from three recovery subnetworks and enhance the low-light image; The encoder in the multi-scale feature extraction branch includes an initial 3*3 convolution layer, a two-way exogenous creation modulation residual block constructed based on an exogenous creation modulation module, a single-way exogenous creation modulation residual block constructed based on the exogenous creation modulation module, and a DownSample module; the encoder in the texture auxiliary branch includes an initial 3*3 convolution layer, a single-way exogenous creation modulation residual block constructed based on the exogenous creation modulation module, and a DownSample module; seven groups of exogenous creation shared residual blocks are embedded between the encoder and the decoder, the module is based on the exogenous creation modulation module, and improved shared separable convolution is used to further strengthen the multi-scale context; The color recovery subnetwork obtains color recovery information in a single-channel color enhancement component map C through a multi-level feature extraction method, and feeds the color recovery information into the multi-scale feature extraction branch in the enhancement backbone network; The brightness recovery subnetwork is used to extract brightness recovery information in a brightness enhancement component map L, and feed the brightness recovery information into the multi-scale feature extraction branch and the texture auxiliary branch in the enhancement backbone network; The texture recovery subnetwork is used to add the texture auxiliary recovery information from the enhancement backbone network and the texture information obtained by preliminary extraction of a texture enhancement component map T in an element-by-element form, finally obtain texture recovery information, and feed the texture recovery information into the multi-scale feature extraction branch in the enhancement backbone network; the texture auxiliary branch in the enhancement backbone network adopts a multi-level convolution structure to extract texture auxiliary recovery information; The texture auxiliary branch adopts a multi-level convolution structure to extract texture auxiliary information; the multi-scale feature extraction branch adopts an encoder-decoder architecture to receive recovery information from three recovery subnetworks and enhance the low-light image; The encoder in the multi-scale feature extraction branch includes an initial 3*3 convolution layer, a two-way exogenous creation modulation residual block constructed based on an exogenous creation modulation module, a single-way exogenous creation modulation residual block constructed based on the exogenous creation modulation module, and a DownSample module; the encoder in the texture auxiliary branch includes an initial 3*3 convolution layer, a single-way exogenous creation modulation residual block constructed based on the exogenous creation modulation module, and a DownSample module; seven groups of exogenous creation shared residual blocks are embedded between the encoder and the decoder, the module is based on the exogenous creation modulation module, and improved shared separable convolution is used to further strengthen the multi-scale context; The color recovery subnetwork obtains color recovery information in a single-channel color enhancement component map C through a multi-level feature extraction method, and feeds the color recovery information into the multi-scale feature extraction branch in the enhancement backbone network; The brightness recovery subnetwork is used to extract brightness recovery information in a brightness enhancement component map L, and feed the brightness recovery information into the multi-scale feature extraction branch and the texture auxiliary branch in the enhancement backbone network; The texture recovery subnetwork is used to add the texture auxiliary recovery information from the enhancement backbone network and the texture information obtained by preliminary extraction of a texture enhancement component map T in an element-by-element form, finally obtain texture recovery information, and feed the texture recovery information into the multi-scale feature extraction branch in the enhancement backbone network; the texture auxiliary branch in the enhancement backbone network adopts a multi-level convolution structure to extract texture auxiliary recovery information; The multi-scale feature extraction branch receives the recovery information from the three recovery sub-networks to enhance the low-light image, and finally outputs an enhanced image I out In the training process of the low-light image enhancement neural network model using the training set, the low-light image enhancement neural network model is made to converge by making the target loss function minimum, and a trained low-light image enhancement neural network model is obtained. Then, the low-light images in the test set are input into the enhancement backbone network in the above trained low-light image enhancement neural network model for model evaluation, and finally a multi-branch feature fusion low-light image enhancement neural network model is obtained.

Images