Low-illumination image enhancement method based on concave manifold projection and local entropy weight arbitration

Abstract

The invention discloses a low-illumination image enhancement method based on concave manifold projection and local entropy weight arbitration, and belongs to the technical field of digital image processing. According to the method, a deep enhancement network of concave manifold projection and local entropy weight arbitration is constructed, and the network comprises an encoder module and a decoder module. In an encoder part, a concave manifold projection mechanism based on physical constraints is introduced, a four-dimensional input vector is constructed by extracting a texture density map and an RGB channel, physical parameter features are predicted by using a semantic guidance type physical manifold parameterization module, and an illumination enhanced parameterization model is established to reconstruct illumination distribution. In a decoder part, a local entropy weight arbitration mechanism is introduced, multi-scale features are captured by using a group of parallel cavity convolution branches with different expansion rates, and adaptive weighted fusion is performed on the features through a space attention weight map generated by a local entropy guidance type multi-scale arbitration module, so that noise amplification is suppressed while image details are recovered. According to the invention, enhancement of the low-illumination image can be effectively realized.

Description

A Low-Light Image Enhancement Method Based on Concave Manifold Projection and Local Entropy Weight Arbitration Technical Field

[0001] This invention belongs to the field of digital image processing technology, and specifically relates to a low-light image enhancement method based on concave manifold projection and local entropy weight arbitration. Background Technology

[0002] With the rapid development of digital imaging technology, image acquisition equipment is increasingly being used in fields such as intelligent monitoring, autonomous driving, night-time imaging assistance, and consumer electronics. However, limited by the physical size and aperture of imaging sensors, images captured at night or in low-light environments often suffer from low brightness, poor contrast, color distortion, and severe noise interference. These degradation phenomena not only affect the subjective visual quality of the images but also seriously hinder the accuracy of subsequent advanced visual tasks (such as object detection and face recognition).

[0003] To improve the quality of low-light images, existing research mainly falls into two categories: traditional methods and deep learning-based methods. Traditional methods are mostly based on histogram equalization or Retinex theory. Retinex-based methods assume that an image can be decomposed into reflection and illumination components, and recover image details by estimating the illumination component. However, these methods often rely on hand-designed prior assumptions, resulting in poor generalization ability when dealing with complex lighting scenes. More seriously, traditional methods often ignore the inherent Poisson-Gaussian noise of the sensor during the brightness enhancement process, which inevitably amplifies the noise while enhancing dark details, producing obvious artifacts and color shifts. Deep learning-based image enhancement methods have become mainstream due to their powerful feature representation capabilities. Early deep enhancement networks mostly adopted end-to-end "black box" projection strategies, directly learning the transformation from low-light images to normal images. Although these methods achieved certain improvements in objective metrics, they still have significant drawbacks. Most networks treat brightness enhancement as a simple nonlinear transformation of pixel values, ignoring the physical laws of illumination imaging (such as the Weber-Fechner law). This leads to enhanced images being prone to local overexposure, color banding, or inversion artifacts that violate physical monotonicity. Existing convolutional neural networks (such as CNNs) often use a uniform convolution kernel across the entire feature map. However, strong denoising is needed in flat areas of the image, while details need to be preserved in high-frequency texture areas. Existing network structures struggle to achieve this adaptive decision-making, often resulting in over-denoising that blurs the texture, or leaving noise behind in an attempt to preserve texture. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing low-light image enhancement techniques and provide a low-light image enhancement method based on concave manifold projection and local entropy weight arbitration. This method aims to solve key problems in the enhancement process of existing deep learning methods, such as color inversion artifacts caused by the lack of physical model constraints and the difficulty in balancing noise suppression and detail preservation.

[0005] This invention includes the following main steps:

[0006] A low-light image enhancement method based on concave manifold projection and local entropy weight arbitration, characterized by the following steps:

[0007] (1) Construct a deep augmentation network with concave manifold projection and local entropy weight arbitration, including the following sub-steps:

[0008] The deep augmentation network that constructs concave manifold projection and local entropy weight arbitration is a deep neural network model based on an encoder-decoder structure. The main architecture of the model includes an encoder module, a decoder module, and a feature transfer path between the encoder and the decoder.

[0009] (1.1) The encoder module is configured as a concave manifold projector based on physical constraints. Its core technology lies in using Taylor expansion to construct a quadratic surface model of illumination response. By learning the scene-adaptive physical parameters, it can realize the transition from the low illumination domain to the normal illumination domain.

[0010] (1.2) The decoder module is configured as a texture restorer based on local entropy weight arbitration. Its core technology is to focus on solving noise and detail problems in the process of low illumination enhancement. By introducing a set of parallel dilated convolution branches with different receptive field scales, and in conjunction with the local entropy-guided multi-scale arbitration module, the spatial attention weight map is generated by calculating the information energy dispersion of the feature space, and pixel-level adaptive fusion of features at each scale is performed.

[0011] (1.3) A feature transfer path is established between the encoder and the decoder to transfer the encoded features to the decoding stage and supplement the lost spatial information.

[0012] (2) The physical constraint-based concave manifold projection mechanism is executed in the encoder module, including the following sub-steps:

[0013] (2.1) The input low-light image is processed by a classification network and a high-level semantic feature extraction network (e.g., VGG-19 and DINOv2) to extract encoded features and high-level semantic features. The encoded features are then used to generate a single-channel texture density map through a manifold topology density-aware module (MTD-SM). ).

[0014] (2.2) Combine the R, G, and B channel data of the original low-light image with the texture density map ( Concatenate the data along the channel dimension to construct a four-dimensional input vector. .

[0015] (2.3) The high-level semantic features are used as conditional inputs and sent to the semantically guided physical manifold parameterization module (SG-PMPM) to generate physical parameter features with 45 channels.

[0016] (2.4) Utilizing the physical parameter characteristics and input vector Illumination adjustment residuals are generated using a concave constrained secondary illumination projection module (CC-QIPM). ( The intermediate enhanced image is output by pixel-level additive overlay onto the original image. ).

[0017] (3) In the decoder module section, a local entropy weight arbitration mechanism is executed, including the following sub-steps:

[0018] (3.1) In the decoder module, the intermediate enhanced image is received. The input consists of the sampled and upsampled encoded features, which are concatenated along the channel dimension and then fed into the feature fusion preprocessing unit. The output of the feature fusion preprocessing unit is... This serves as the input for subsequent parallel multi-scale dilated convolutions and local entropy-guided multi-scale arbitration modules (LEG-MAM).

[0019] (3.2) The stream is split into nine parallel dilated convolution branches with different dilation rates to perform multi-scale decomposition of image features, capturing feature representations from minute textures to global context.

[0020] (3.3) The input local entropy-guided multi-scale arbitration module (LEG-MAM) analyzes the local entropy distribution of the feature map, dynamically generates a spatial attention weight map, adaptively weights and fuses the multi-scale features based on the weight map, and then sends the fused high-dimensional feature map into the input... The convolutional layer reprojects the feature space back into the 3D RGB pixel space to generate a restored residual map. The restored residual map () ) and intermediate enhanced image ( The pixels are stacked to produce a low-light enhanced image.

[0021] Furthermore, in step (2), the texture density map ( ) and four-dimensional input vector The construction specifically includes:

[0022] The encoder integrates a manifold topology density sensing module (MTD-SM), which takes the encoded features extracted by a classification network (e.g., VGG-19) as input, and processes them sequentially through layer normalization, ... Convolutional layer, GELU activation function, popular gradient-aware unit, statistical density-aware unit, splicing, Convolutional layers and the sigmoid activation function at the end are used to generate a single-channel texture density map. ). This indicates that the pixel is located at a high-frequency edge or in a region with rich texture. This indicates that the pixel is located in a flat area.

[0023] The manifold gradient sensing unit is configured with a geometric topology that uses central difference convolution to capture features. Unlike ordinary convolution that calculates a pixel-weighted sum, this unit calculates the feature map locally. The relative gradient response within the neighborhood. For a position on the feature map... and its neighborhood Output The calculation is as follows:

[0024]

[0025] in, Defined as learnable convolutional kernel weight coefficients, this operation forces the network to ignore flat regions (with a difference of 0) and only produce a high response to abrupt changes in the manifold surface (such as edges, texture boundaries), which mathematically captures the geometric topology of the texture.

[0026] The statistical density sensing unit is configured to capture the statistical density features of the texture using a local variance pooling operator, and the local second-order statistics of the feature map are computed using a parameterless sliding window statistical method. Calculate the local standard deviation within the window. :

[0027]

[0028] in, To prevent numerically unstable minima, this operation directly quantifies the information entropy or energy dispersion of local regions. In the perceptual feature space, the variance of smooth backgrounds is extremely low, while the variance of complex texture regions is significantly higher, which statistically captures the information density of the texture.

[0029] Extract the red (R), green (G), and blue (B) channel data from the original low-light image and compare them with the texture density map. Cascaded along the channel dimension for each pixel location in the image. Construct a four-dimensional input vector .

[0030] Furthermore, the generation of physical parameter features in step (2) specifically includes:

[0031] The encoder integrates a semantically guided physical manifold parameterization module (SG-PMPM), which receives high-level semantic features extracted by a high-level semantic feature extraction network (e.g., DINOv2) as input, and then processes them sequentially. Convolutional layers, LeakyRELU activation function Convolutional layers, entering two respectively Convolutional layers generate pixel-level scale factors. ) and bias factor ( Then, the original low-light image was adaptively downsampled and processed... The output of the convolutional layer ( The input spatial features are transformed and modulated together; then, they are sequentially processed through dilated convolution, ReLU activation function, grouped feature projection, and Tanh activation function to generate physical parameter features with 45 channels.

[0032] In the spatial feature transformation modulation unit, the input is received. Scale factor ( ) and bias factor ( This is used to perform element-wise affine transformation operations. The formula is expressed as:

[0033]

[0034] in, It represents the Hadamardi (or Hadama) stack; It acts as a pixel-level amplifier gain. It acts as a pixel-level DC bias. This is a "gating" process where high-level semantic features determine which areas of physical illumination information should be amplified and which should be preserved.

[0035] Furthermore, in step (2), the illumination adjustment residual... ( The generation of ) specifically includes:

[0036] The encoder integrates a concave constrained secondary illumination projection module (CC-QIPM), which receives physical parameter features and an input vector. Input; The concave constrained secondary illumination projection module (CC-QIPM) is composed of a cascaded physical parameter decoupling and Hessian matrix construction unit and a secondary manifold illumination response generation unit.

[0037] The physical parameter decoupling and Hessian matrix construction unit receives physical feature parameters and performs the parsing and constraint construction of these parameters. Specifically, this unit decouples the 45 channels by grouping them along the channel dimension, dividing them into three groups on average, corresponding to the mapping parameters of the R, G, and B color channels respectively. For any color channel... Its parameter set includes: lower triangular matrix parameters used to construct the Hessian matrix. (10-dimensional) Jacobian vector used to control linear brightness (4-dimensional) and global bias terms (1-dimensional); using the formula Constraint Construction The symmetric Hessian matrix, where for The transpose of the matrix, symbol " "Used to enforce a semi-negative definite matrix."

[0038] After obtaining the physically constrained parameters, the secondary manifold illumination response generation unit generates the illumination response for each pixel and each color channel. Using the constructed four-dimensional input vector X, a projection operation based on a quadratic manifold is performed to calculate the illumination adjustment residual. :

[0039]

[0040] in, Given the four-dimensional input vector, using the Hessian matrix... The off-diagonal elements in the code implement the R, G, B color channels and texture factors. Nonlinear coupling between them is used to enhance brightness while maintaining color consistency and to adjust anisotropy according to texture density.

[0041] Finally, the calculated color residual maps of the three channels ( The intermediate enhanced image is output by concatenating the images along the channel dimension and performing element-wise addition with the original low-light input image using a residual learning mechanism. ).

[0042] Furthermore, in step (3), the specific structure and calculation logic of the local entropy weight arbitration specifically include:

[0043] The encoded features are first upsampled to make their spatial resolution match that of the intermediate enhanced image. The encoder output is consistent with the decoder, which then receives the intermediate enhanced image from the encoder output. The input consists of the encoded features and cascaded operations along the channel dimension, which are then fed into the feature fusion preprocessing unit.

[0044] In the feature fusion preprocessing unit, it sequentially goes through... Convolutional layers Convolutional layer, GELU activation function, layer normalization, output ( This serves as the input for subsequent parallel dilated convolution branches and local entropy-guided multi-scale arbitration modules.

[0045] In the local entropy-guided multi-scale arbitration module, the local entropy map generation unit and the local entropy map ( )and splicing Convolutional layers, GELU activation function, Convolutional layers and a softmax function are used to generate nine spatial attention weight maps. ,satisfy .

[0046] In the local entropy map generation unit, the receiver... Perform element-wise multiplication on it:

[0047]

[0048] Then and The data are fed into two identical average pooling layers with kernel size k=3, stride s=1, and padding p=1. The first branch outputs a local first-order moment. (i.e., local mean), this feature represents the background brightness or low-frequency floor of a local region; the second branch outputs the local second-order moment. (i.e., local squared mean), this feature represents the total energy intensity of a local region. Finally, a local entropy map is generated. Generate, calculated according to the following formula:

[0049]

[0050] in This indicates averaging along the channel dimension. To prevent numerical instability and local minima, a formula is used. Calculate the local variance and use it as an approximate measure of local entropy (information content). The generated local entropy map... It achieves blind detection at the physical level: high response values ​​accurately anchor high-frequency textures and noise areas, while low response values ​​lock flat background areas, providing objective physical evidence for subsequent arbitration.

[0051] The data is split and fed into a parallel dilated convolution branch, which outputs a feature map. The spatial attention weight map is used to perform a weighted summation of feature maps at different scales. The calculation formula is as follows: ,in ⊕ indicates element-wise multiplication, and ⊕ indicates weighted fusion.

[0052] After the weighted fusion Enter to the last one Convolution, generating the restored residual map ( ), to restore the residual map ( ) and intermediate enhanced image ( Pixel-level overlay is performed to output a low-light enhanced image. Attached Figure Description

[0053] Figure 1 shows the overall network structure of the low-light image enhancement method based on concave manifold projection and local entropy weight arbitration.

[0054] Figure 2 shows the manifold topology density sensing module.

[0055] Figure 3 is a diagram of the semantically guided physical manifold parameterization module.

[0056] Figure 4 shows the concave constraint secondary illumination projection module.

[0057] Figure 5 shows a diagram of a local entropy-guided multi-scale arbitration module.

[0058] Figure 6 shows the feature fusion preprocessing unit diagram. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0060] The overall implementation process of this method includes four steps: constructing and preprocessing high dynamic range paired datasets for multiple scenarios; constructing a deep augmentation network with concave manifold projection and local entropy weight arbitration; performing concave manifold projection at the encoder end; and performing local entropy weight arbitration at the decoder end. Each step will be described in great detail below.

[0061] (1) Constructing and preprocessing high dynamic range paired datasets for multiple scenarios, including the following sub-steps:

[0062] (1.1) In order to capture true low-light photon noise and sensor readout noise, the scheme of synthesizing low-light images through Gamma correction was abandoned. Instead, a full-frame high dynamic range camera with a wide dynamic range and a large aperture fixed-focus lens was used. The camera was fixed on a heavy-duty carbon fiber tripod to eliminate the slight displacement caused by wind or ground vibration. A shutter release cable was used to trigger the shooting to eliminate the camera shake caused by pressing the shutter button.

[0063] (1.2) For each static scene, strictly follow the “double exposure” acquisition process to obtain the input image and the ground truth image.

[0064] Low-light input image acquisition:

[0065] The aperture was fixed at (f / 5.0 - f / 5.6) to obtain sufficient depth of field and ensure that objects at different distances were sharp; the ISO was set to a high value (ISO 3200 - 6400), and the high ISO introduced analog gain, thus realistically introducing the sensor's unique Poisson-Gaussian noise distribution; the shutter speed was set to a short exposure (1 / 60s - 1 / 100s), which resulted in extremely low light intake, with the histogram concentrated in the dark areas, simulating real-world nighttime shooting scenarios.

[0066] Normal exposure true image acquisition:

[0067] The aperture is kept strictly constant to ensure that the depth of field and optical distortion are completely consistent with the input image; the ISO is set to the camera's native minimum value (ISO 100) to reduce noise and provide a clean color reference. The shutter speed is significantly extended (1s-10s) to accumulate photons through long exposure, resulting in a uniform histogram distribution and complete dynamic range coverage.

[0068] (1.3) To prevent the network from overfitting to specific types of scenarios, the constructed dataset covers four challenging typical environments:

[0069] High-frequency texture scenes (accounting for about 30%): These include dense foliage, rough brick walls, and distant urban or rural building complexes. These scenes are used to train the network's ability to retain high-frequency information and prevent over-denoising.

[0070] Low-frequency smooth scenes (accounting for about 30%): These include night skies, calm water surfaces, and smooth highway surfaces. These scenes are very prone to revealing noise and are used to train the network's smoothing and denoising capabilities.

[0071] Extreme lighting scenarios (approximately 20%): The image contains both extremely dark areas and extremely bright light sources (such as direct headlights or streetlights). This is used to test whether the network can utilize the "concave function" property of the concave manifold projection to suppress highlight clipping when enhancing dark areas.

[0072] Extremely dark scenes (approximately 20%): almost dark environments with ambient illumination below 0.1 Lux (such as rural roads without streetlights), challenging the model's extreme sensitivity.

[0073] (1.4) The raw data (RAW format) after acquisition must undergo rigorous processing:

[0074] Even with a tripod, wind-blown trees or passing vehicles can cause localized blurring. The SIFT optical flow algorithm is used to calculate the displacement vector between image pairs. If the average displacement error exceeds 0.5 pixels or there is significant ghosting, the image pair is discarded.

[0075] The RAW data is de-mosaiced and converted to RGB images using an ISP pipeline, but no noise reduction or sharpening is performed, preserving the original features.

[0076] Image data normalization is performed by linearly projecting pixel values ​​from the integer domain [0, 255] to the floating-point domain [0, 1], using the following formula: .

[0077] (2) Construct a deep augmentation network with concave manifold projection and local entropy weight arbitration, including the following sub-steps:

[0078] As shown in Figure 1, the deep augmentation network constructed by the present invention with concave manifold projection and local entropy weight arbitration is a deep neural network model based on an encoder-decoder structure. The main architecture of the model includes an encoder module, a decoder module, and a feature transfer path between the encoder and decoder.

[0079] (2.1) The purpose of the encoder module is to understand the illumination distribution from a macroscopic semantic perspective, predict the physical parameters used to reconstruct the illumination, and realize the transition from the low illumination domain to the normal illumination domain.

[0080] (2.2) The decoder module receives the intermediate enhancement results from the encoder and the encoded features passed through the feature transfer path, focusing on solving noise and detail problems.

[0081] (2.3) The role of the feature transfer path is to supplement the lost spatial information.

[0082] (3) Perform concave manifold projection at the encoder end, including the following sub-steps:

[0083] This is the core technical feature of the invention, which simplifies illumination to linear multiplication in the traditional Retinex model. To address the shortcomings of traditional methods, this embodiment introduces the Taylor expansion concept and uses quadratic surfaces to approximate complex illumination response functions.

[0084] (3.1) In order to achieve anisotropic enhancement (less enhancement in flat areas to prevent noise amplification, and more enhancement in textured areas to reveal details), a priori metric is needed for low-light images. First, encoded features and high-level semantic features are extracted using a classification network and a high-level semantic feature extraction network (e.g., VGG-19 and DINOv2), respectively. The encoded features are then fed into a specially designed manifold topology density-aware module (Figure 2). First, layer normalization is applied to unify the mean and variance of the feature distribution under different illumination intensities, eliminating the interference of global brightness shifts on local texture judgment. Then, a 1×1 convolutional layer performs channel dimension compression and feature fusion. Finally, the GELU activation function is used to introduce non-linear activation to improve the model's fitting accuracy for complex illumination distributions. The output result is then obtained. Input the manifold gradient sensing unit and the statistical density sensing unit respectively.

[0085] The manifold gradient sensing unit is configured with a geometric topology that uses central difference convolution to capture features. Unlike ordinary convolution that calculates a pixel-weighted sum, this unit calculates the feature map locally. The relative gradient response within the neighborhood. For a position on the feature map... and its neighborhood Output The calculation is as follows:

[0086]

[0087] in, Defined as learnable convolutional kernel weight coefficients, this operation forces the network to ignore flat regions (with a difference of 0) and only produce a high response to abrupt changes in the manifold surface (such as edges, texture boundaries), which mathematically captures the geometric topology of the texture.

[0088] The statistical density sensing unit is configured to capture the statistical density features of the texture using a local variance pooling operator, and the local second-order statistics of the feature map are computed using a parameterless sliding window statistical method. Calculate the local standard deviation within the window. :

[0089]

[0090] To prevent numerically unstable minima, this operation directly quantifies the information entropy or energy dispersion of local regions. In the perceptual feature space, the variance of smooth backgrounds is extremely low, while the variance of complex texture regions is significantly higher, which statistically captures the information density of the texture.

[0091] Will The features are concatenated along the channel dimension, then fused and mapped to a single-channel output through a 1×1 convolutional layer. Finally, the output value is strictly compressed to the [0, 1] interval using a Sigmoid activation function to generate a single-channel texture density map. ). This indicates that the pixel is located at a high-frequency edge or in a region with rich texture. This indicates that the pixel is located in a flat area.

[0092] (3.2) Traditional color correction is only performed in the RGB three-dimensional color space. In this embodiment, a four-dimensional input vector is constructed, which combines the R, G, and B channel data of the original low-light image with the texture density map ( ) Stitching is performed along the channel dimension, targeting each pixel position in the image. ), construct the input vector :

[0093]

[0094] This step enables the early fusion of physical photometric values ​​and texture semantic values. Subsequent projection functions will be based on this four-dimensional input vector, meaning that lighting adjustments will be jointly determined by color and texture.

[0095] (3.3) The semantically guided physical manifold parameterization module (Figure 3) uses the input high-level semantic features and the original low-light image to predict the physical parameter features used to define the quadratic surface. First, the original low-light image is adaptively downsampled to align its spatial resolution with the high-level semantic features. Then, through... Convolutional layers expand the number of channels to a dimension that matches the semantic features, and the output is ( ).

[0096] High-level semantic features are sequentially processed The convolutional layers project and increase the dimensionality of the channels, expanding the number of feature channels to 64 dimensions to enhance the network's implicit representation ability of nonlinear lighting features. The LeakyRELU activation function introduces nonlinear activation to improve the model's fitting accuracy to complex lighting distributions while allowing negative features to pass through. Convolutional layers introduce local spatial context. Finally, they enter two... Convolutional layers linearly map high-dimensional semantic context features to the physical modulation parameter space, generating pixel-level scale factors. ) and bias factor ( ). Scale factor ( This branch is used to correspond to gain or contrast stretching in the physical model. For example, for a semantic region of "low contrast," this branch will generate a larger value. (e.g., >1), to stretch the dynamic range of the feature; for "high-noise" regions, generate a smaller [size / size]. To suppress noise amplification. Bias factor ( This corresponds to ambient light compensation in the physical model; for example, for "extremely dark shadow" regions, this branch will generate a positive [path / path]. This directly increases the base brightness; for the "center of the light source" region, it generates a brightness close to 0. Avoid overexposure.

[0097] In the spatial feature transformation modulation unit, the input is received. Scale factor ( ) and bias factor ( This is used to perform element-wise affine transformation operations. The formula is expressed as:

[0098]

[0099] in, It represents the Hadamardi (or Hadama) stack; It acts as a pixel-level amplifier gain. It acts as a pixel-level DC bias. This is a "gating" process where high-level semantic features determine which areas of physical illumination information should be amplified and which should be preserved.

[0100] Entering configuration expansion rate of 2 Dilated convolutional layers effectively expand the receptive field, capturing global long-range correlations in illumination distribution without increasing the number of parameters, and imposing implicit spatial smoothness constraints on the feature manifold to avoid discontinuous block artifacts in the enhanced image. Then, a nonlinear rectification mechanism is introduced through the ReLU activation function. On the one hand, the feature map is sparsified through one-sided suppression, effectively filtering out background noise and invalid texture responses in low-light environments; on the other hand, it endows the network with the ability to fit complex nonlinear illumination manifolds, ensuring that the mapping from semantic context to physical control parameters has sufficient expressive dimension, while its non-saturation characteristic guarantees gradient propagation efficiency during training. Next, grouped feature projection is performed, using a group size of 3... Grouped convolution maps features to a 45-channel physical parameter space. This topology mathematically enforces orthogonal decoupling of the R, G, and B channel physical attributes, effectively blocking erroneous interference between color channels to ensure color fidelity. Finally, the Tanh activation function at the end strictly constrains the final generated physical parameter values ​​within the symmetrical range of [-1, 1], thus preventing numerical explosion that could lead to image overexposure while ensuring the numerical stability of the generated higher-order physical adjustment curves. The physical parameter features are then output after passing through the Tanh activation function.

[0101] (3.4) In order to transform the abstract features predicted by the network into adjustment coefficients that conform to the laws of physical illumination, this embodiment designs a concave constraint secondary illumination projection module (Figure 4). This module is composed of a cascaded physical parameter decoupling and Hessian matrix construction unit and a secondary manifold illumination response generation unit. The specific execution logic is as follows:

[0102] The physical parameter decoupling and Hessian matrix construction unit receives the 45-channel physical parameter features output by the semantically guided physical manifold parameterization module and performs physical parameter parsing and constraint construction. Specifically, this unit decouples the 45 channels by channel dimension, dividing them into three groups on average, corresponding to the mapping parameters of the R, G, and B color channels respectively. Taking the red channel (R) as an example, the required 15 parameters are defined as follows:

[0103] Hessian matrix parameters (10 dimensions) are used for construction. The Hessian matrix, because The matrix has 16 elements, but the Hessian matrix is ​​symmetric and is constructed using the lower triangular matrix method, so only 10 parameters are needed.

[0104] Jacobian vector (4-dimensional) Corresponding to the first-order coefficient, controlling the linear brightness gain.

[0105] Bias term (1-dimensional) Corresponding constant term, controlling the basic brightness offset.

[0106] According to the Weber-Fechner law, the human eye's perception of brightness is logarithmic, meaning that sensitivity decreases as brightness increases (diminishing marginal returns). Therefore, the illumination enhancement function must be a strictly concave function to prevent overexposure caused by excessive magnification of highlight areas. Mathematically, the concavity of a quadratic function is determined by the positive definiteness of its quadratic coefficient matrix (Hessian matrix). Therefore, physical parameter decoupling and the construction of Hessian matrix units do not directly predict... Instead, it is constructed by forcibly using the following formula:

[0107]

[0108] in It was predicted by the network. A lower triangular matrix (whose elements are filled with the aforementioned 10-dimensional parameters); for any real matrix L, the product... It must be a positive semi-definite matrix, that is, for any vector v, we have .therefore, It must be a semi-negative definite matrix.

[0109] This constraint ensures that... The defined illumination response surface is a downward-opening parabola in four-dimensional space. Regardless of how the network parameters are learned, this mathematical structure ensures that the enhancement curve automatically reduces gain in high-brightness regions, fundamentally eliminating artifacts and overexposure.

[0110] After obtaining the physically constrained parameters, the secondary manifold illumination response generation unit generates the illumination response for each pixel and each color channel. Using the constructed four-dimensional input vector X, a projection operation based on a quadratic manifold is performed to calculate the illumination adjustment residual. :

[0111]

[0112] Expand the formula:

[0113] From the quadratic term (such as ): Adjust the contrast of a single channel.

[0114] Cross quadratic terms (e.g.) Adjusting the color coupling between channels resolves color cast issues.

[0115] Texture coupling terms (e.g.) ):because Participation in the calculation means that for the same red pixel value R, if it is in the texture area... and in flat areas Its enhancement results It's completely different; this achieves adaptive enhancement of texture awareness.

[0116] Finally, the illumination adjustment residuals calculated by the secondary manifold illumination response generation unit are used... The original low-light image is superimposed using pixel-level addition to obtain an intermediate enhanced image. At this point, the overall brightness distribution of the image has been corrected to the normal visible range. However, due to the increase in illumination gain, the sensor noise hidden in the dark areas is also amplified simultaneously. Therefore, the image still exhibits obvious graininess and needs to be further processed by a subsequent decoder.

[0117] (4) Perform local entropy weight arbitration at the decoder end, including the following sub-steps:

[0118] The decoder's task is to process Noise and details in low-light images. The challenge of low-light images lies in the fact that noise in flat areas requires smoothing with large convolutional kernels, while subtle textures need to be preserved with small convolutional kernels, which traditional CNNs with fixed convolutional kernels cannot handle simultaneously. This embodiment designs a decoder to address this problem and utilizes local entropy weight arbitration to solve it.

[0119] (4.1) The encoded features are first upsampled to make their spatial resolution match that of the intermediate enhanced image. The encoder output is consistent with the decoder, which then receives the intermediate enhanced image from the encoder output. The inputs are coded features and coded features, which are then cascaded along the channel dimension and finally fed into the feature fusion preprocessing unit (Figure 6).

[0120] In the feature fusion preprocessing unit, firstly, using Pointwise convolution performs cross-channel manifold projection, linearly recombining heterogeneous input channels into a unified latent feature space, eliminating semantic gaps between modalities; then, the input... Deep convolution performs spatial dimension context calibration, utilizing local neighborhood information to smooth and filter the fused features, correcting spatial discontinuities and alignment errors caused by direct concatenation. Subsequently, a nonlinear activation function, GELU, is introduced, leveraging its smooth gradient properties to protect weak detail signals under low light conditions. Finally, layer normalization is used to renormalize the output features, suppressing internal covariate shifts and providing statistically stable and spatially aligned standardized feature inputs for the subsequent local entropy-guided multi-scale arbitration module. The output of the feature fusion preprocessing unit is ( ).

[0121] (4.2) Output results of the feature fusion preprocessing unit ( The data is split into 9 parallel convolutional branches. To expand the receptive field without increasing the number of parameters, dilated convolution is used.

[0122] The expansion rates of the 9 branches are set as follows: .

[0123] Small expansion rate ( ): Feel the Wild Small, focusing on capturing high-frequency, minute textures in images.

[0124] medium expansion rate ( ): The field of view is moderate, capturing the edges and shapes of objects.

[0125] Large expansion rate ( ): Extremely large receptive field, capable of covering a large image area.

[0126] Its function is to utilize long-range contextual information to perform consistency checks on flat regions, thereby effectively identifying and suppressing isolated noise points and artifacts. Each branch outputs a feature map. .

[0127] (4.3) The local entropy-guided multi-scale arbitration module (Figure 5) analyzes the local entropy of the feature map. The output of the feature fusion preprocessing unit ( After inputting, First, enter the local entropy map generation unit.

[0128] In the local entropy map generation unit, the receiver... Perform element-wise multiplication on it:

[0129]

[0130] The features are nonlinearly mapped from the "amplitude response space" to the "energy response space". Since the calculation of variance (texture dispersion) depends on the expected value of the square of the variables, this step lays the mathematical foundation for subsequent capture of the energy fluctuations of the signal.

[0131] Then and The data are fed into two identical average pooling layers with kernel size k=3, stride s=1, and padding p=1. The first branch outputs a local first-order moment. (i.e., local mean), this feature represents the background brightness or low-frequency floor of a local region; the second branch outputs the local second-order moment. (i.e., local squared mean), this feature represents the total energy intensity of a local region. Both branches maintain... constant.

[0132] Finally, the local entropy map is generated, calculated according to the following formula:

[0133]

[0134] in This indicates averaging along the channel dimension. To prevent numerical instability and local minima, a formula is used. Calculate the local variance and use it as an approximate measure of local entropy (information content). The generated local entropy map ( It achieves blind detection at the physical level: high response values ​​accurately anchor high-frequency textures and noise areas, while low response values ​​lock flat background areas, providing objective physical evidence for subsequent arbitration.

[0135] Output from the local entropy map generation unit and Stacking along the channel dimension (Dim=1) results in the following dimensional transformation:

[0136]

[0137] Achieving spatial alignment of features enables the neural network to process each spatial coordinate ( When ), it possesses the "semantic context" of that point ( It also possesses the "statistical complexity" of that point. This enables early fusion of dual-source information; then, the hybrid tensor is fed into... Convolutional layer The convolutional kernel is equivalent to a weighted linear combination of "entropy" and "semantics" to learn specific feature patterns; then, it passes through the GELU activation function, which introduces the idea of ​​random regularization, preserving the weak responses in the negative value region; then it passes through a second... The convolutional layer projects the high-dimensional features into nine independent decision logic values. These nine channels physically correspond one-to-one with nine externally parallel dilated convolutional branches with different dilation rates, representing the network's "recommendation strength" for each branch. Finally, the softmax function generates nine spatial attention weight maps. ,satisfy .

[0138] (4.4) Final decoding output features The result is calculated by weighted sum of the features of each branch:

[0139]

[0140] The weighted fusion output Enter directly to the last one. The convolutional layer acts as a non-linear projector, directly projecting the high-dimensional feature space onto the three-dimensional RGB pixel space to generate a restored residual map. The restored residual map () ) and intermediate enhanced image ( This method performs pixel-level overlay and employs a global residual learning strategy. This allows the network to focus on predicting the amount of texture detail restoration and noise reduction, rather than reconstructing the image from scratch. This achieves high-fidelity image quality enhancement while maintaining consistent illumination. Ultimately, the output is a low-light enhanced image.

[0141] In summary, this invention, through the above-described embodiments, mathematically guarantees that the enhancement results conform to the Weber-Fechner law by constructing a semi-negative definite Hessian matrix, naturally suppressing highlights while enhancing dark areas. It also breaks the limitation of fixed convolutional kernels by utilizing a local entropy weight arbitration mechanism, achieving pixel-level adaptive receptive field. Because of the introduction of physical model constraints, the network not only fits data but also learns the physical laws of lighting and imaging, thus performing excellently even in unseen real-world scenes without exhibiting common artifacts or color casts.

Claims

1. A low-light image enhancement method based on concave manifold projection and local entropy weight arbitration, characterized in that... The following steps are taken: (1) Constructing a deep enhancement network based on concave manifold projection and local entropy weight arbitration, including the following sub-steps: The construction of the deep enhancement network based on concave manifold projection and local entropy weight arbitration is based on a deep neural network model with an encoder-decoder structure. The main architecture of the model includes an encoder module, a decoder module, and a feature transfer path between the encoder and the decoder. (1.1) The encoder module is configured as a concave manifold projector based on physical constraints. Its core technology is to use the Taylor expansion idea to construct a quadratic surface model of illumination response. By learning the scene-adaptive physical parameters, it can realize the transition from the low illumination domain to the normal illumination domain. (1.2) The decoder module is configured as a texture restorer based on local entropy weight arbitration. Its core technology is to focus on solving noise and detail problems in the low illumination enhancement process. By introducing a set of parallel dilated convolution branches with different receptive field scales, and cooperating with the local entropy-guided multi-scale arbitration module to calculate the information energy dispersion of the feature space, a spatial attention weight map is generated, and pixel-level adaptive fusion of features at each scale is performed. (1.3) A feature transfer path is established between the encoder and the decoder to transfer the encoded features to the decoding stage to supplement the lost spatial information. (2) A concave manifold projection mechanism based on physical constraints is executed in the encoder module, including the following sub-steps: (2.1) The input low-light image is processed by a classification network and a high-level semantic feature extraction network (e.g., VGG-19 and DINOv2) to extract encoded features and high-level semantic features, respectively. The encoded features are used to generate a single-channel texture density map through the manifold topology density sensing module (MTD-SM). (2.2) Combine the R, G, and B channel data of the original low-light image with the texture density map ( Concatenate the data along the channel dimension to construct a four-dimensional input vector. (2.3) The high-level semantic features are used as conditional inputs and fed into the semantically guided physical manifold parameterization module (SG-PMPM) to generate physical parameter features with 45 channels. (2.4) The physical parameter features and the input vector are used... Lighting adjustment residuals are generated using the concave constrained secondary illumination projection module (CC-QIPM). ( The intermediate enhanced image is output by pixel-level additive overlay onto the original image. (3) In the decoder module, a local entropy weight arbitration mechanism is executed, including the following sub-steps: (3.1) In the decoder module, the intermediate enhanced image is received ( The input consists of the sampled and upsampled encoded features, which are concatenated along the channel dimension and then fed into the feature fusion preprocessing unit. The output of the feature fusion preprocessing unit is... This serves as the input for subsequent parallel multi-scale dilated convolutions and local entropy-guided multi-scale arbitration modules (LEG-MAM). (3.2) The image features are split into nine parallel dilated convolution branches with different dilation rates to perform multi-scale decomposition, capturing feature representations from minute textures to global context. (3.3) The input local entropy-guided multi-scale arbitration module (LEG-MAM) analyzes the local entropy distribution of the feature map, dynamically generates a spatial attention weight map, adaptively weights and fuses the multi-scale features based on the weight map, and then sends the fused high-dimensional feature map into the input... The convolutional layer reprojects the feature space back into the 3D RGB pixel space to generate a restored residual map. The restored residual map () ) and intermediate enhanced image ( The pixels are stacked to produce a low-light enhanced image.

2. The method according to claim 1, characterized in that... The texture density map in step (2) ) and four-dimensional input vector The construction specifically includes: (1) The encoder integrates a manifold topology density sensing module (MTD-SM), which takes the encoded features extracted by a classification network (e.g., VGG-19) as input, and passes them through layer normalization, Convolutional layer, GELU activation function, popular gradient-aware unit, statistical density-aware unit, splicing, Convolutional layers and the sigmoid activation function at the end are used to generate a single-channel texture density map. )。 This indicates that the pixel is located at a high-frequency edge or in a region with rich texture. This indicates that the pixel is located in a flat region. (2) The geometric topology of the manifold gradient sensing unit is configured to capture features using central difference convolution. Unlike ordinary convolution which calculates pixel weighted sums, this unit calculates feature maps locally. The relative gradient response within the neighborhood. For a position on the feature map... and its neighborhood Output The calculation is as follows: in, Defined as learnable convolutional kernel weight coefficients, this operation forces the network to ignore flat regions (difference of 0) and only produce a high response to abrupt changes in the manifold surface (such as edges, texture boundaries), which mathematically captures the geometric topology of the texture. (3) The statistical density sensing unit is configured to capture the statistical density features of the texture using a local variance pooling operator, and the local second-order statistics of the feature map are calculated using a parameterless sliding window. Calculate the local standard deviation within the window. ： in, To prevent numerically unstable minimum values, this operation directly quantifies the information entropy or energy dispersion of the local region. In the perceptual feature space, the variance of the smooth background is extremely low, while the variance of the complex texture region is significantly higher, which statistically captures the information density of the texture. (4) Extract the red (R), green (G), and blue (B) three-channel data of the original low-light image and compare them with the texture density map ( Cascaded along the channel dimension for each pixel location in the image. Construct a four-dimensional input vector 。 3. The method according to claim 1, characterized in that... The generation of physical parameter features in step (2) specifically includes: (1) The encoder integrates a semantically guided physical manifold parameterization module (SG-PMPM), which receives high-level semantic features extracted by a high-level semantic feature extraction network (e.g., DINOv2) as input, and sequentially passes through... Convolutional layers, LeakyRELU activation function Convolutional layers, entering two respectively Convolutional layers generate pixel-level scale factors. ) and bias factor ( Then, the original low-light image was adaptively downsampled and processed... The output of the convolutional layer ( (2) In the spatial feature transformation modulation unit, the input is fed together with the spatial feature transformation modulation; then it is sequentially processed by dilated convolution, ReLU activation function, grouped feature projection and Tanh activation function to generate physical parameter features with 45 channels. Scale factor ( ) and bias factor ( This is used to perform element-wise affine transformation operations. The formula is expressed as: in, It represents the Hadamardi (or Hadama) stack; It acts as a pixel-level amplifier gain. It acts as a pixel-level DC bias. This is a "gating" process, where high-level semantic features determine which areas of physical illumination information should be amplified and which should be preserved.

4. The method according to claim 1, characterized in that... The illumination adjustment residual in step (2) ( The generation of ) specifically includes: (1) The encoder integrates a concave constraint secondary illumination projection module (CC-QIPM), which receives physical parameter features and input vectors. Input; The concave constraint secondary illumination projection module (CC-QIPM) is composed of a physical parameter decoupling and Hessian matrix construction unit and a secondary manifold illumination response generation unit cascaded together. (2) The physical parameter decoupling and Hessian matrix construction unit receives physical feature parameters and performs the analysis and constraint construction of physical parameters. Specifically, this unit decouples the 45 channels in the channel dimension and divides them into three groups on average, corresponding to the mapping parameters of the three color channels R, G, and B respectively. For any color channel Its parameter set includes: lower triangular matrix parameters used to construct the Hessian matrix. (10-dimensional) Jacobian vector used to control linear brightness (4-dimensional) and global bias terms (1-dimensional); using the formula Constraint Construction The symmetric Hessian matrix, where for The transpose of the matrix, symbol " "Used to force the constraint to be a semi-negative definite matrix. (3) After obtaining the above physically constrained parameters, the quadratic manifold illumination response generation unit generates each pixel and each color channel. Using the constructed four-dimensional input vector X, a projection operation based on a quadratic manifold is performed to calculate the illumination adjustment residual. ： in, Given the four-dimensional input vector, using the Hessian matrix... The off-diagonal elements in the code implement the R, G, B color channels and texture factors. Nonlinear coupling between them is used to enhance brightness while maintaining color consistency and to perform anisotropic adjustment based on texture density. Finally, the calculated color residual maps of the three channels are used. The intermediate enhanced image is output by concatenating the images along the channel dimension and performing element-wise addition with the original low-light input image using a residual learning mechanism. )。 5. The method according to claim 1, characterized in that... In step (3), the specific structure and calculation logic of the local entropy weight arbitration include: (1) the encoded features are first upsampled to make their spatial resolution match that of the intermediate enhanced image. The encoder output is consistent with the decoder, which then receives the intermediate enhanced image from the encoder output. (2) The coded features are used as initial inputs and cascaded in the channel dimension before being input to the feature fusion preprocessing unit. Convolutional layers Convolutional layer, GELU activation function, layer normalization, output ( (3) In the local entropy-guided multi-scale arbitration module, it passes through the local entropy map generation unit and the local entropy map ( ) and splicing Convolutional layers, GELU activation function, Convolutional layers and a softmax function are used to generate nine spatial attention weight maps. ,satisfy In the local entropy map generation unit, the receiver... Perform element-wise multiplication on it: Then and The data is fed into two identical average pooling layers with kernel size k=3, stride s=1, and padding p=1. The first branch outputs a local first-order moment. (i.e., local mean), this feature represents the background brightness or low-frequency floor of a local region; the second branch outputs the local second-order moment. (i.e., local squared mean), this feature represents the total energy intensity of a local region. Finally, a local entropy map is generated. Generate, calculated according to the following formula: in This indicates averaging along the channel dimension. To prevent numerical instability and local minima, a formula is used. Calculate the local variance and use it as an approximate measure of local entropy (information content). The generated local entropy map ( This achieves blind detection at the physical level: high response values ​​accurately anchor high-frequency textures and noise areas, while low response values ​​lock flat background areas, providing objective physical evidence for subsequent arbitration. (4) The data is split and fed into a parallel dilated convolution branch, which outputs a feature map. The spatial attention weight map is used to perform a weighted summation of feature maps at different scales. The calculation formula is as follows: ,in ⊕ indicates element-wise multiplication, and ⊕ indicates weighted fusion. (5) After the weighted fusion, Enter to the last one Convolution, generating the restored residual map ( ), to restore the residual map ( ) and intermediate enhanced image ( Pixel-level overlay is performed to output a low-light enhanced image.

Images