Feature-driven region-aware multi-scale low-light image enhancement method

Abstract

The application belongs to the technical field of image processing. The application provides a feature-driven region perception multi-scale low-light image enhancement method. The disclosed embodiment can dynamically and accurately locate the feature distribution of a low-light image, and implement region-adaptive differential enhancement, thereby significantly enhancing and protecting the detail features while strongly suppressing the noise in flat regions, so as to solve the core technical problem that it is difficult to balance between detail recovery and noise suppression.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to a feature-driven region-aware multi-scale low-light image enhancement method. Background Technology

[0002] In applications such as autonomous driving, security monitoring, medical imaging, and mobile photography, acquiring high-quality images under low-light conditions is crucial. Low-light images typically suffer from low brightness, poor contrast, color distortion, and severe sensor noise. Traditional image enhancement methods, such as histogram equalization (HE) and contrast-limited adaptive histogram equalization (CLAHE), improve contrast by adjusting pixel intensity distribution, but they are prone to amplifying noise and producing unnatural over-enhancement effects. Methods based on Retinex theory decompose the image into reflectance and illuminance components and enhance it by adjusting the illuminance map, but they rely on manually designed priors, are insufficient in suppressing complex noise, and are prone to producing halo artifacts.

[0003] In recent years, deep learning-based methods have made significant progress. One type of method combines Retinex theory with deep networks (such as Retinex-Net, KinD) to attempt to learn physically interpretable decompositions; another type employs end-to-end network architectures (such as Multi-branch Low-Light Enhancement Network (MBLLEN), EnlightenGAN) to directly learn the mapping from low-light to normal-light images. However, existing deep learning methods often adopt a "one-size-fits-all" global enhancement strategy, failing to fully consider the differences in degradation patterns between different regions within the image (such as feature regions rich in edge texture and smooth homogeneous regions). This leads to a common trade-off: aggressive processing to restore details in feature regions amplifies noise in smooth regions; while smoothing operations to suppress noise erase important details. Some studies on 3D point cloud denoising also show that preserving the sharp geometric features of the model while removing noise remains a persistent challenge.

[0004] Furthermore, although some methods have attempted to introduce attention mechanisms or perform preliminary region segmentation, feature detection and image enhancement are often performed as two separate steps, lacking synergistic optimization, which limits the improvement of the final results.

[0005] Therefore, it is necessary to improve one or more of the problems existing in the above-mentioned related technical solutions.

[0006] It should be noted that this section is intended to provide background or context for the technical solutions of this disclosure as set forth in the claims. The description herein does not constitute an admission that it is prior art simply because it is included in this section. Summary of the Invention

[0007] The purpose of this disclosure is to provide a feature-driven region-aware multi-scale low-light image enhancement method, thereby overcoming at least to some extent one or more problems caused by the limitations and defects of related technologies.

[0008] According to embodiments of this disclosure, a feature-driven region-aware multi-scale low-light image enhancement method is provided, comprising: Step S1: Construct the FRMNet model; wherein the FRMNet model includes a feature detection module, a feature region enhancement network, a non-feature region enhancement network, and a fusion module; Step S2: Acquire a low-light image and perform adaptive gamma correction on it to obtain a corrected image; Step S3: Use the feature detection module to perform feature detection and image denoising on the corrected image to obtain a feature mask and a denoised image; Step S4: Using the feature mask as a region guide, the feature region enhancement network is used to sharpen and restore details of the denoised image to obtain the first enhanced image; Step S5: Using the feature mask as a region guide, the non-feature region enhancement network is used to enhance the brightness, contrast and suppress noise of the denoised image to obtain the second enhanced image; Step S6: Based on the feature mask, the first enhanced image and the second enhanced image are subjected to pixel-level adaptive weighted fusion using the fusion module to obtain the enhanced image.

[0009] Furthermore, step S3 specifically includes: Step S31: Fix the current corrected image and update the feature regions using gradient and connectivity conditions: Based on the gradient vector field of the low-light image, calculate the gradient magnitude field of the current corrected image estimate and normalize it to obtain the normalized gradient magnitude. Based on the normalized gradient magnitude, an adaptive threshold is determined, and feature candidate regions are obtained. Erosion and dilation operations are performed sequentially on the candidate feature regions to obtain the updated feature regions; Step S32: Fix the updated feature regions, update the denoised image by solving a convex optimization problem, and obtain the final denoised image after iterative convergence; wherein, the convex optimization problem is:

[0010] in, For the first The denoised image obtained in the next iteration. For the optimized low-light image, The original, noisy, low-light image. For regularization parameters, The gradient vector field of the low-light image. For the updated feature region, For vectors The square of the norm; Step S33: Define feature masks for the feature regions based on the denoised image:

[0011]

[0012] in, As a feature mask, For feature masks at pixel coordinates The value at that location, For pixel domain, The height of the denoised image, The width of the denoised image.

[0013] Furthermore, the feature region enhancement network includes: The system includes a multi-scale feature extraction module, a feature enhancement network, a scale-aware feature aggregation module, and a progressive feature enhancement module; among which, The feature enhancement network employs a symmetric encoder-decoder structure; The scale-aware feature aggregation module includes an attention weight calculation submodule, a multi-head attention submodule, a feature aggregation submodule, and a residual connection submodule.

[0014] Furthermore, step S4 specifically includes: Step S41: Using the feature mask as a region guide, extract features from the denoised image using a multi-scale feature extraction module. Extract features from the feature regions in the image to generate a 1 / 4 scale feature representation. and 1 / 8 scale feature representation Combined with denoised images To generate multi-scale feature maps ; Step S42: Utilize the encoder of the feature enhancement network to process the multi-scale feature maps. Each scale in After basic feature extraction, the features corresponding to each layer of the encoder and the corresponding layers of the decoder are fused through skip connections to generate the original scale features. 1 / 4 scale features and 1 / 8 scale features ;in, B Batch size; Step S43: Calculate the original scale features using the attention weight submodule. Features at 1 / 4 scale Perform downsampling projection to extract the original scale features. Features at 1 / 4 scale Align to the same resolution to compute queries. Q s and keys K ; Step S44: Based on query Q s and keys K， The multi-head attention mechanism of the multi-head attention submodule is used to extract the original scale features. Features at 1 / 4 scale Divided along the channel dimension Each head is assigned a number of heads, and the attention weight for each head is calculated using a scaled dot product attention mechanism. Step S45: Use the feature aggregation submodule to perform weighted aggregation on the features output from all the headers to generate aggregated features. ; Step S46: Use the residual connection submodule to aggregate features The original resolution was restored through bilinear upsampling and compared with the original features. Perform residual connections to generate enhanced features. ; Step S47: Enhance features The image is decomposed into multiple components and progressively enhanced through residual connections to generate a first enhanced image. .

[0015] Furthermore, the non-feature region enhancement network includes: FEM module, EM module and FM module; among them The EM module consists of an initial enhancement block and multiple cascaded enhancement blocks.

[0016] Furthermore, step S5 specifically includes: Step S51: Using the feature mask as a region guide, utilize the FEM module to extract from the denoised image Features are extracted from non-feature regions to generate initial features. ; Step S52: Apply initial enhancement blocks to the initial features Perform initial enhancements to generate enhanced features. :

[0017] Step S53: Apply multiple cascaded enhancement blocks to the initial features Perform cascaded enhancements to generate enhanced features. :

[0018] in, For the initial enhancement block, For the first An enhancement block, The SiLU activation function is used. For the first The features obtained by convolution of the output of each enhancement block For the first The features obtained by convolution of the output of each enhancement block The size of the nucleus is Convolution operation, These are the weight parameters for the inter-module convolution; Step S54: After concatenating all enhancement features along the channel dimension using the FM module, feature fusion is performed through convolution to generate the second enhanced image. .

[0019] Furthermore, step S6 specifically includes: Based on feature mask , the first enhanced image Second enhanced image Pixel-level adaptive weighted fusion is performed to obtain the final enhanced image. :

[0020] in, This is element-wise multiplication.

[0021] Furthermore, the total loss function of the FRMNet model is:

[0022] in, These are the first adjustable hyperparameter, the second adjustable hyperparameter, the third adjustable hyperparameter, the fourth adjustable hyperparameter, and the fifth adjustable hyperparameter. For mean square error loss, for Norm loss, For structural similarity loss, For color consistency loss, The loss is a regional distribution consistency regularization loss based on KL divergence.

[0023] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: In the embodiments of this disclosure, the feature-driven region-aware multi-scale low-light image enhancement method described above, on the one hand, designs a collaboratively optimized framework, FRMNet model, which unifies region decomposition, detail enhancement, and noise suppression in an end-to-end training process, making the three mutually reinforcing rather than sequential independent steps. This method achieves intelligent region decomposition of low-light images, accurately distinguishing feature regions requiring detail enhancement (such as edges and textures) from smooth regions requiring robust denoising. While significantly improving the overall brightness, contrast, and color naturalness of the image, this method maximizes the recovery of details in feature regions and effectively suppresses noise in smooth regions, achieving an optimal balance between detail and purity. On the other hand, a comprehensive multi-task loss function is also designed, jointly optimizing pixel fidelity, structural similarity, color consistency, and region distribution harmony to ensure that the enhancement network can simultaneously achieve high-fidelity reconstruction and high-quality perception during training. The innovative region distribution consistency constraint is key to ensuring the naturalness of the results. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0025] Figure 1 The diagram illustrates the steps of a feature-driven region-aware multi-scale low-light image enhancement method according to an exemplary embodiment of this disclosure. Figure 2 This diagram illustrates the overall framework of the FRMNet model in an exemplary embodiment of this disclosure. Figure 3 A schematic diagram illustrating the workflow of the feature detection module in an exemplary embodiment of this disclosure is shown. Figure 4 This diagram illustrates the workflow of the feature region enhancement network in an exemplary embodiment of this disclosure. Figure 5 This diagram illustrates the workflow of the feature region enhancement network in an exemplary embodiment of this disclosure. Figure 6 Visual comparison results of various methods in exemplary embodiments of this disclosure on the SLL dataset are shown. Detailed Implementation

[0026] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0027] Furthermore, the accompanying drawings are merely illustrative diagrams of embodiments of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0028] This example implementation provides a feature-driven, region-aware, multi-scale low-light image enhancement method. (Reference) Figure 1 As shown, this feature-driven region-aware multi-scale low-light image enhancement method may include: Step S1: Construct the FRMNet model; wherein the FRMNet model includes a feature detection module, a feature region enhancement network, a non-feature region enhancement network, and a fusion module; Step S2: Acquire a low-light image and perform adaptive gamma correction on it to obtain a corrected image; Step S3: Use the feature detection module to perform feature detection and image denoising on the corrected image to obtain a feature mask and a denoised image; Step S4: Using the feature mask as a region guide, the feature region enhancement network is used to sharpen and restore details of the denoised image to obtain the first enhanced image; Step S5: Using the feature mask as a region guide, the non-feature region enhancement network is used to enhance the brightness, contrast and suppress noise of the denoised image to obtain the second enhanced image; Step S6: Based on the feature mask, the first enhanced image and the second enhanced image are subjected to pixel-level adaptive weighted fusion using the fusion module to obtain the enhanced image.

[0029] The aforementioned feature-driven region-aware multi-scale low-light image enhancement method employs two main approaches. First, a collaboratively optimized framework, FRMNet, is designed to unify region decomposition, detail enhancement, and noise suppression within a single end-to-end training process, ensuring that these three processes mutually reinforce each other rather than being independent steps. This method achieves intelligent region decomposition of low-light images, accurately distinguishing between feature regions requiring detail enhancement (such as edges and textures) and smooth regions requiring robust denoising. While significantly improving overall image brightness, contrast, and color naturalness, this method maximizes the recovery of detail in feature regions and effectively suppresses noise in smooth regions, achieving an optimal balance between detail and purity. Second, a comprehensive multi-task loss function is designed. By jointly optimizing pixel fidelity, structural similarity, color consistency, and region distribution harmony, this ensures that the enhancement network achieves both high-fidelity reconstruction and high-quality perception during training. The innovative region distribution consistency constraint is crucial for guaranteeing the naturalness of the results.

[0030] Below, we will refer to Figures 1 to 6 The steps of the feature-driven region-aware multi-scale low-light image enhancement method described in this example embodiment will be explained in more detail.

[0031] In step S1, an FRMNet model is constructed; wherein the FRMNet model includes a feature detection module, a feature region enhancement network, a non-feature region enhancement network, and a fusion module.

[0032] Specifically, such as Figure 2 The diagram shown is the overall framework of the FRMNet model; as shown below. Figure 3 The diagram shown illustrates the workflow of the feature detection module. Figure 4 The diagram shown illustrates the workflow of a feature region enhancement network. Figure 5 The diagram shown illustrates the workflow of a non-feature region enhancement network.

[0033] Loss Function Design: During the model training phase, to comprehensively and accurately guide the optimization of the Feature-Driven Region-Aware Network (FRMNet), this application designs and implements a comprehensive multi-task loss function system. This system not only ensures that the output image closely approximates the real reference image in pixel values, but also imposes constraints from multiple dimensions such as perceptual quality, color fidelity, and region consistency, which is one of the keys to achieving the excellent enhancement effect of this invention. This total loss function system integrates five core components through a weighted summation method, and its overall definition is as follows:

[0034] in, For adjustable hyperparameters used to balance the contributions of various losses, for example, a set of effective typical values ​​are: The following provides a detailed explanation of each loss component: 1) Mean Squared Error Loss (MSE Loss) This is the basic pixel-level fidelity loss, designed to minimize the enhancement image. Compared with real normal light images The pixel-by-pixel difference between them. It is defined as:

[0035] in, This represents the total number of pixels in the image (height × width × number of channels). This loss function drives the network to perform basic global brightness and color corrections, providing a stable gradient signal for overall reconstruction.

[0036] 2) Norm loss ( Loss)—— To further improve reconstruction accuracy and enhance edge sharpness, the following is introduced: Loss. Compared to the MSE loss, which is more sensitive to outliers, The loss function is more robust to noise and outliers, contributing to the generation of sharper images. It is defined as:

[0037] and The combined use of these technologies ensures the consistency of the overall reconstruction while enhancing the ability to preserve detailed structures.

[0038] 3) Structural Similarity Loss (SSIM Loss) To simulate the human visual system's (HVS) perception of image structural information, a perceptual loss based on the Structural Similarity Index (SSIM) is introduced. SSIM better measures the perceptual quality of an image by comparing the luminance, contrast, and structure of local image regions. The loss is defined as the negative value of the SSIM exponent.

[0039] in, In two image blocks and The calculation above is:

[0040] in, To minimize the mean of the augmented blocks, This represents the mean of a real, normal light image patch. and Let the first variance and the second variance be... For covariance, and where represents the first and second stability constants. This loss directly optimizes the local structure of the image, making the enhancement result visually more natural and in line with human visual perception.

[0041] 4) Color Consistency Loss To prevent color distortion or color cast during enhancement, a color consistency loss was designed. This loss constrains the enhanced image to maintain the global statistical mean of each color channel of the real image, thereby preserving the overall tone of the image. Its calculation formula is as follows:

[0042] in, These are the coordinates in pixel space.

[0043] This effectively avoids the problem of sacrificing color accuracy in pursuit of increased brightness.

[0044] 5) KL-Regularization Loss Based on Regional Distribution Consistency — This is a key innovation in the design of the loss function in this invention. To address the potential for brightness and contrast inconsistencies or "blockiness" between feature regions and non-feature (smooth) regions due to differential processing, a region distribution consistency regularization term based on Kullback-Leibler (KL) divergence is introduced:

[0045] in,

[0046]

[0047] ,

[0048]

[0049] in, l This is the level index after brightness discretization. , The total number of levels for brightness discretization.

[0050] Here, the Bin() function quantizes continuous brightness values ​​into L discrete levels. This is an indicator function; its value is 1 when the condition within the parentheses is true, and 0 otherwise. This loss function forces the network, during the enhancement process, not only to handle feature regions and smooth regions separately, but also to actively bring the overall illumination statistics of these two regions closer together, thereby generating high-quality results with globally unified tones, natural transitions, and no abrupt blocks. This loss function directly depends on and feeds back to optimize the accuracy of the feature detection module.

[0051] Training process: During the training phase, a large-scale paired dataset (such as LOL, MIT-Adobe FiveK) is used, with the aforementioned total loss. To optimize the target, the Adam or AdamW optimizer is used for end-to-end joint training of the entire FRN network (including feature detection and dual-path augmentation subnetworks). The learning rate is typically scheduled using a cosine annealing strategy with a warm-up. Hyperparameters such as batch size and number of epochs are set according to the specific dataset and computational resources. Guided by this comprehensive loss function, the network can collaboratively optimize the accuracy of feature detection and the quality of region augmentation, ultimately learning to achieve a perfect balance between detail and denoising under complex low-light conditions.

[0052] In step S2, a low-light image is acquired and subjected to adaptive gamma correction to obtain a corrected image.

[0053] Specifically, input a low-light RGB image (i.e., a low-light image), and process the low-light RGB image... Adaptive gamma correction preprocessing is performed to initially improve the overall brightness, resulting in a preprocessed image. The number of iterations K and the feature region ratio threshold p are set.

[0054] More specifically, this method first involves basic image representation and operations. A digital image can be represented as a two-dimensional matrix. ,in H and W These represent the height and width of the image, respectively. C Indicates the number of channels (for RGB images) The image is in coordinates. Location, passage The pixel value is denoted as The gradient vector field of an image is defined as follows: Its amplitude, i.e., gradient strength, is:

[0055] Input and preprocessing: Acquire low-light RGB images First, for low-light RGB images... Adaptive gamma correction is performed to improve image brightness, resulting in a gamma-corrected image. :

[0056] in, It is the enhanced pixel value, and:

[0057] in, It is the average intensity of the image. It is the baseline value. It is a small constant to prevent the denominator from being zero.

[0058] Step S3: Use the feature detection module to perform feature detection and image denoising on the corrected image to obtain a feature mask and a denoised image.

[0059] Specifically, based on the current image estimation (in the first iteration...) The gradient magnitude field is calculated. An initial set of candidate feature pixels is obtained by applying an adaptive threshold (dynamically determined based on the maximum proportion p of the preset feature regions). Then, morphological connectivity constraints are applied (specifically, an erosion operator is used to remove isolated noise points with excessively small areas, and a dilation operator is used to connect neighboring separated feature blocks), resulting in an optimized, connected binary feature mask. A value of 1 represents a feature region (such as edges or textures), and a value of 0 represents a non-feature (smooth) region. Through iteration, the feature mask becomes increasingly accurate as the image quality improves, and a denoised image is obtained.

[0060] More specifically, first, fix the current image estimation Update feature regions using gradient and connectivity conditions The specific update steps are as follows: (1) Calculate the gradient magnitude field: .

[0061] (2) Normalized gradient magnitude: .

[0062] (3) Determine the adaptive threshold , making .

[0063] (4) Obtaining feature candidates: .

[0064] (5) Apply connectivity constraints: Among them, the erosion operator Used to remove small connected regions: .

[0065] in, This is a preset minimum connected region size threshold.

[0066] Inflation operator Used to connect regions with similar features: , in, It is the shortest path connecting the separated feature regions.

[0067] Secondly, fixed feature regions Update the image with reduced noise. The specific update is achieved by solving the following convex optimization problem: .

[0068] in, The complement symbol, i.e. This is a non-feature region; The Euler-Lagrange equation for this problem is: , in, To restrict the gradient operator to non-feature regions, for The adjoint operator; The linear system is obtained by rearranging: , in, It is an identity operator.

[0069] Finally, define the feature mask for the feature region. :

[0070] Simultaneously, a standard deviation is applied to the non-feature regions. A Gaussian filter with a value of 0.25 is used for noise reduction. The process is iteratively updated until a convergence condition is met, at which point the feature region is obtained. Corresponding feature mask Compared with the denoised image .

[0071] In steps S4 and S5, guided by the feature mask, the feature region enhancement network is used to sharpen and restore details of the denoised image to obtain a first enhanced image; guided by the feature mask, the non-feature region enhancement network is used to enhance brightness, contrast and suppress noise of the denoised image to obtain a second enhanced image.

[0072] Specifically, dual-path region adaptive image enhancement. Using the feature mask obtained in step S3 as a region guide, the current denoised image is input into two parallel enhancement sub-networks (i.e., a feature region enhancement network and a non-feature region enhancement network) for parallel processing: Feature Region Enhancement Path: A lightweight feature enhancement network (FeatEnHancer) is used to process the image. This network employs multi-scale feature extraction, depthwise separable convolution, and channel attention mechanisms to focus on sharpening and detail restoration of the feature regions indicated by the mask (i.e., feature mask), resulting in the first enhanced image.

[0073] Non-feature region enhancement pathway: An improved version of the multi-branch learning (MBLLEN)-based enhancement network is used to process the image. This network focuses on robust brightness enhancement, contrast enhancement, and noise suppression in smooth regions indicated by the mask, resulting in a second enhanced image.

[0074] More specifically, the current denoised image and feature mask Input dual-path enhancement network.

[0075] 1) Feature Region Enhancement Network (Lightweight Feature Enhancer): This network is specifically designed to enhance high-frequency details such as edges and textures. (See reference...) Figure 4 The overall network architecture comprises three core modules: multi-scale feature extraction, feature enhancement network, scale-aware feature aggregation, and progressive feature enhancement. The specific processing flow is as follows: ① Multi-scale feature extraction: Let the input low-light RGB image be... ,in B For batch size, H and W These represent the height and width of the image, respectively. Multi-scale downsampling is specifically described as follows: the network first constructs feature representations at quarter-scale and eighth-scale.

[0076] in, Indicates the kernel size as The enhanced depthwise separable convolution operation is specifically represented as follows:

[0077] in, For the input image, D Represents depthwise convolution. P This represents pointwise convolution, and BN represents batch normalization. This represents the SiLU activation function. Compared to ordinary convolution, depthwise separable convolution significantly reduces the number of parameters and computational cost, while ensuring training stability through batch normalization.

[0078] ② Feature Enhancement Network: The feature enhancement network adopts a fully convolutional architecture and performs feature extraction in parallel at three scales. It consists of two parts: basic feature extraction and symmetric skip connection feature fusion.

[0079] a) Basic feature extraction: For each scale The feature extraction process can be represented as:

[0080] in, express Convolution operation, These are the first, second, third, and fourth learnable parameters, respectively. All convolutional layers use the ReLU activation function to introduce non-linear expressive power.

[0081] b) Symmetrical skip connection feature fusion: The network adopts a symmetrical encoder-decoder structure and fuses multi-level features through skip connections:

[0082] in, This indicates a channel-level concatenation operation. These are the fifth, sixth, and seventh learnable parameters, respectively. This design allows the network to establish direct connections between different feature levels, alleviating the vanishing gradient problem and enhancing feature propagation efficiency.

[0083] ③ Scale-aware feature aggregation: The scale-aware feature aggregation module is key to multi-scale information fusion. It dynamically adjusts the relative importance of features at different scales to achieve adaptive feature weighting fusion. It mainly includes three parts: attention weight calculation, multi-head attention mechanism, and feature aggregation and residual connection.

[0084] a) Attention weight calculation: Let the original scale feature be... A quarter-scale feature is First, align the two to the same resolution using downsampling projection:

[0085] in, Q and K These represent the query and the key representation, respectively. This represents a standard convolution with a kernel size of 3 and a stride of 2. Batch normalization and the SiLU activation function are preserved in the key path to ensure feature quality.

[0086] b) Multi-head attention mechanism: To capture richer feature relationships, features are segmented along the channel dimension. One head:

[0087] The attention weights for each head are calculated using a scaled dot product attention mechanism:

[0088] in, .

[0089] This multi-head design allows the network to learn feature relationships in different representation subspaces, enhancing the model's expressive power.

[0090] c) Feature aggregation and residual connection: The weighted aggregated features are restored to their original resolution through bilinear upsampling and then residually connected to the original features.

[0091] in, This represents a bilinear upsampling operation. The introduction of residual connections ensures that the original feature information is not lost during the aggregation process, while providing an identity mapping path, which is beneficial for gradient flow and model training.

[0092] ④ Progressive Feature Enhancement: The progressive feature enhancement module optimizes image features in stages to achieve a smooth feature enhancement process.

[0093] Feature decomposition and cumulative augmentation: The final augmented features are decomposed into 8 components, each responsible for feature optimization at a specific level.

[0094] in, For fixed input features, The hyperbolic tangent activation function is used. This is a channel-dimensional splitting operation.

[0095] Each component The original image is gradually enhanced through residual connections, forming a cumulative enhancement chain:

[0096] The final enhancement result is This progressive enhancement strategy allows the network to focus on optimizing different types of features at different stages, avoiding artifacts and unnaturalness that may result from a single large adjustment.

[0097] After the above four processes, the final enhanced image of the feature region is: .

[0098] 2) Non-Feature Region Enhancement Network (Improved MBLLEN): This network is specifically designed for the overall enhancement of smooth regions. The network architecture consists of three components: a feature extraction module (FEM module), multiple enhancement modules (EM modules), and finally, a fusion module (FM module) to generate the enhanced image. (See reference...) Figure 5 The specific process is as follows: ① FEM (Feature Extraction Module): The feature extraction module is a simple convolutional layer used to extract initial features from non-feature regions of the input low-light image. Let the input low-light image be... ,in, B For batch size, H and W These represent the height and width of the image, respectively. The mathematical expression for the FEM module is:

[0099] in, Indicates use Convolution kernel, convolution operation padded with 1s. These are learnable weight parameters. Represents the ReLU activation function. Output feature map. It can be used as input for subsequent model enhancement.

[0100] ② EM (Enhancement Module): The enhancement module employs a symmetrical encoder-decoder structure, responsible for extracting multi-scale information from the feature map and reconstructing the enhanced image. The input to each EM module is the feature map. The output is an enhanced color image. .

[0101] a) Encoding path: The encoding path extracts high-level features step by step through a series of convolutional layers: , in, Indicates the kernel size as The convolution operation uses valid convolution (padding=0), resulting in a gradual decrease in feature map size. , ; , ; , .

[0102] b) Decoding path: The decoding path gradually restores spatial resolution through transposed convolution.

[0103] in, Indicates the kernel size as The transpose convolution operation is then performed. Finally, a center-cropping operation is applied. Ensure that the output dimensions match the input dimensions: .

[0104] c) Multi-scale augmentation architecture: The network employs a hierarchical augmentation strategy, comprising an initial EM module (i.e., the initial augmentation block) and multiple cascaded augmentation blocks. Let the total number of augmentation modules be... ,in P For the number of blocks, Q This represents the number of EM modules in each block.

[0105] Initial enhancements: .

[0106] Cascaded enhancement: for the first The first block The enhancement process for each EM module is as follows:

[0107] in, , These are the weight parameters for inter-module convolution.

[0108] ③ FM (Fusion Module): This module integrates the outputs of all enhancement modules to generate the final enhanced image. First, it stitches together all EM outputs along the channel dimension:

[0109] Then through Convolutional features are fused, which is equivalent to performing a learnable weighted combination of the various enhancement results:

[0110] in, To output the weight parameters of the convolution, the sigmoid function ensures that the output range is [value range missing]. It conforms to the standard range of image pixel values.

[0111] After the above three modules, the final enhanced image of the non-feature region is: .

[0112] In step S6, based on the feature mask, the first enhanced image and the second enhanced image are subjected to pixel-level adaptive weighted fusion using the fusion module to obtain the enhanced image.

[0113] Specifically, based on feature masks The output results of the two enhancement pathways and Perform pixel-level adaptive weighted fusion:

[0114] This results in the final enhanced image. .

[0115] In one specific embodiment, to verify the effectiveness of the proposed FRMNet model, experiments were conducted on the publicly available SLL (Synthetic Low-Light) dataset. The SLL dataset contains low-light images with synthetic noise and their corresponding normal-light reference images, specifically designed to evaluate the joint performance of low-light enhancement and denoising.

[0116] Experimental conditions: The hardware platform consisted of eight NVIDIA RTX 4090 GPUs (24GB VRAM), the operating system was Linux, and the deep learning framework was PyTorch 2.4.1. The Adam optimizer was used during training, with an initial learning rate of 1×10⁻⁶. - ³, weight decay 1×10 -5 A two-stage learning rate scheduling strategy combining exponential decay and ReduceLROnPlateau was employed, with 1000 training epochs and a batch size of 32. The input image was cropped into 256×256 blocks with an overlap rate of 0.5. The feature detection module underwent alternating iterations T=3 times.

[0117] Comparison Methods: Nine mainstream methods were selected, including the traditional LIME method, illumination learning methods (RetinexNet, Zero-DCE, UREtinex-Net, UREtinex-Net++, SCI, RUAS), and end-to-end CNN methods (MBLLEN, AGLLNet). Seven metrics were used for quantitative evaluation: PSNR, SSIM, VIF, LOE, TMQI, AB, and LPIPS.

[0118] Table 1 Quantitative Comparison Results of SLL Datasets

[0119] In this chart, ↓ indicates that the smaller the value of the indicator, the better the image quality.

[0120] Experimental Results: As shown in Table 1, FRMNet, proposed in this application, achieved the best peak signal-to-noise ratio of 21.04 dB on the SLL dataset, which is about 3 dB higher than the second-best method, AGLLNet (18.02 dB). Its structural similarity index reached 0.828, significantly outperforming all comparison methods. The tone mapping quality index was 0.798, the luminance fidelity error AB was 0.054, and the perceptual similarity index was 0.119, all of which were optimal. These quantitative results fully demonstrate the superiority of FRMNet in the joint task of low-light enhancement and denoising.

[0121] like Figure 6 As shown, the visual comparison results of each method on the SLL dataset are presented. While the LIME method improves overall brightness, it leaves significant residual synthetic noise; RetinexNet and Zero-DCE amplify noise in dark areas; MBLLEN and AGLLNet over-smooth texture details; UREtinex-Net and its improved versions still have residual noise. In contrast, FRMNet in this application effectively removes noise while clearly preserving image edges and fine textures, resulting in enhancements that most closely resemble the real reference image. This verifies the effectiveness of feature-driven region-aware design in addressing the core technical challenge of balancing detail restoration and noise suppression.

[0122] The experimental results above demonstrate that the feature-driven region-aware multi-scale low-light image enhancement method proposed in this application outperforms existing mainstream methods in terms of quantitative indicators, visual effects, and region-awareness accuracy. It can effectively solve the core technical problem of balancing detail restoration and noise suppression in low-light images.

[0123] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0124] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0125] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A feature-driven, region-aware, multi-scale low-light image enhancement method, characterized in that, include: Step S1: Construct the FRMNet model; wherein the FRMNet model includes a feature detection module, a feature region enhancement network, a non-feature region enhancement network, and a fusion module; Step S2: Acquire a low-light image and perform adaptive gamma correction on it to obtain a corrected image; Step S3: Use the feature detection module to perform feature detection and image denoising on the corrected image to obtain a feature mask and a denoised image; Step S4: Using the feature mask as a region guide, the feature region enhancement network is used to sharpen and restore details of the denoised image to obtain the first enhanced image; specifically including: Step S41: Using the feature mask as a region guide, extract features from the denoised image using a multi-scale feature extraction module. Extract features from the feature regions in the image to generate a 1 / 4 scale feature representation. and 1 / 8 scale feature representation Combined with denoised images To generate multi-scale feature maps ; Step S42: Utilize the encoder of the feature enhancement network to process the multi-scale feature maps. Each scale in After basic feature extraction, the features corresponding to each layer of the encoder and the corresponding layers of the decoder are fused through skip connections to generate the original scale features. 1 / 4 scale features and 1 / 8 scale features ;in, B Batch size; Step S43: Calculate the original scale features using the attention weight submodule. Features at 1 / 4 scale Perform downsampling projection to extract the original scale features. Features at 1 / 4 scale Align to the same resolution to compute queries. Q s and keys K ; Step S44: Based on query Q s and keys K， The multi-head attention mechanism of the multi-head attention submodule is used to extract the original scale features. Features at 1 / 4 scale Divided along the channel dimension Each head is assigned a number of heads, and the attention weight for each head is calculated using a scaled dot product attention mechanism. Step S45: Use the feature aggregation submodule to perform weighted aggregation on the features output from all the headers to generate aggregated features. ; Step S46: Use the residual connection submodule to aggregate features The original resolution was restored through bilinear upsampling and compared with the original features. Perform residual connections to generate enhanced features. ; Step S47: Enhance features The image is decomposed into multiple components and progressively enhanced through residual connections to generate a first enhanced image. ; Step S5: Using the feature mask as a region guide, the non-feature region enhancement network is used to enhance the brightness, contrast, and suppress noise of the denoised image to obtain the second enhanced image; wherein, the non-feature region enhancement network includes a feature extraction module, an enhancement module, and a fusion module, and the enhancement module includes an initial enhancement block and multiple cascaded enhancement blocks; Specifically, it includes: Step S51: Using the feature mask as a region guide, extract features from the denoised image using the feature extraction module. Features are extracted from non-feature regions to generate initial features. ; Step S52: Apply initial enhancement blocks to the initial features Perform initial enhancements to generate enhanced features. : Step S53: Apply multiple cascaded enhancement blocks to the initial features Perform cascaded enhancements to generate enhanced features. : in, For the initial enhancement block, For the first An enhancement block, The SiLU activation function is used. For the first The features obtained by convolution of the output of each enhancement block For the first The features obtained by convolution of the output of each enhancement block The size of the nucleus is Convolution operation, These are the weight parameters for the inter-module convolution; Step S54: After concatenating all enhanced features along the channel dimension using the fusion module, feature fusion is performed through convolution to generate the second enhanced image. ; Step S6: Based on the feature mask, the first enhanced image and the second enhanced image are subjected to pixel-level adaptive weighted fusion using the fusion module to obtain the enhanced image.

2. The feature-driven region-aware multi-scale low-light image enhancement method according to claim 1, characterized in that, Step S3 specifically includes: Step S31: Fix the current corrected image and update the feature regions using gradient and connectivity conditions: Based on the gradient vector field of the low-light image, calculate the gradient magnitude field of the current corrected image estimate and normalize it to obtain the normalized gradient magnitude. Based on the normalized gradient magnitude, an adaptive threshold is determined, and feature candidate regions are obtained. Erosion and dilation operations are performed sequentially on the candidate feature regions to obtain the updated feature regions; Step S32: Fix the updated feature regions, update the denoised image by solving a convex optimization problem, and obtain the final denoised image after iterative convergence; wherein, the convex optimization problem is: in, For the first The denoised image obtained in the next iteration. For the optimized low-light image, The original, noisy, low-light image. For regularization parameters, The gradient vector field of the low-light image. For the updated feature region, For complement symbols, For vectors The square of the norm; Step S33: Define feature masks for the feature regions based on the denoised image: in, As a feature mask, For feature masks at pixel coordinates The value at that location, For pixel domain, The height of the denoised image, The width of the denoised image.

3. The feature-driven region-aware multi-scale low-light image enhancement method according to claim 2, characterized in that, Feature region enhancement networks include: The system includes a multi-scale feature extraction module, a feature enhancement network, a scale-aware feature aggregation module, and a progressive feature enhancement module; among which, The feature enhancement network employs a symmetric encoder-decoder structure; The scale-aware feature aggregation module includes an attention weight calculation submodule, a multi-head attention submodule, a feature aggregation submodule, and a residual connection submodule.

4. The feature-driven region-aware multi-scale low-light image enhancement method according to claim 3, characterized in that, Step S6 specifically includes: Based on feature mask , the first enhanced image Second enhanced image The final enhanced image is obtained by performing pixel-level adaptive weighted fusion. : in, This is element-wise multiplication.

5. The feature-driven region-aware multi-scale low-light image enhancement method according to claim 4, characterized in that, The total loss function of the FRMNet model is: in, These are the first adjustable hyperparameter, the second adjustable hyperparameter, the third adjustable hyperparameter, the fourth adjustable hyperparameter, and the fifth adjustable hyperparameter. For mean square error loss, for Norm loss, For structural similarity loss, For color consistency loss, The loss is a regional distribution consistency regularization loss based on KL divergence.

Images