A pixel self-adaptive low-illumination image enhancement method based on numerical stable decomposition

Abstract

The application discloses a pixel self-adaptive low-illumination image enhancement method based on numerical stability decomposition, obtains a low-illumination input image to be enhanced, a pixel value matrix and brightness data thereof, and forms an illumination matrix based on original pixel row and column coordinates of the low-illumination input image; performs Retinex decomposition processing on the pixel value matrix, the initial illumination matrix and a pixel-level self-adaptive stability term, obtains a noisy reflection component, constructs a residual noise prediction mapping function, and performs noise compensation on the noisy reflection component to obtain a denoising reflection component; inputs the pixel value matrix and the illumination matrix into an enhancement parameter prediction model, and performs brightness gain adjustment on the denoising reflection component by using a plurality of pixel-level enhancement factors to output an enhanced image. The application can complete cooperative processing through a step-by-step link of decomposition, denoising and enhancement, does not need a large number of paired training samples, and improves the generalization capability of the enhancement parameter prediction model to a complex real scene.

Description

Technical Field

[0001] This invention relates to the field of low-light image processing technology, and more specifically, to a pixel-adaptive low-light image enhancement method based on numerically stable decomposition. Background Technology

[0002] Low-light images are acquired by security monitoring, vehicle sensing, consumer electronics and other devices in low-light environments. They usually need to be image enhanced to meet the needs of human visual observation and subsequent machine vision analysis.

[0003] However, existing low-light image enhancement techniques suffer from several drawbacks when enhancing low-light images of realistic and complex scenes. These include unstable calculations in dark area decomposition leading to noise amplification, and susceptibility of illumination estimation to local noise interference. Furthermore, some post-processing methods for denoising can damage image texture edges and cause color shifts, making it difficult to achieve differentiated enhancement for different lighting regions, resulting in overexposure and color distortion. In addition, the high cost of obtaining paired training samples, insufficient model generalization ability, and the failure of training constraints to simultaneously address decomposition stability, reflectance domain noise, and the synergistic optimization of the enhancement process all contribute to the computational burden and training difficulties of some solutions, hindering the development of low-light image processing technology.

[0004] Based on this, this application proposes a pixel-adaptive low-light image enhancement method based on numerically stable decomposition. Summary of the Invention

[0005] This invention aims to overcome the problems of unstable dark area decomposition values ​​causing noise amplification, susceptibility of illuminance estimation to noise interference, easy generation of texture and color distortion in post-denoising, difficulty in achieving differentiated enhancement by region, high cost of obtaining paired training samples, limited generalization ability, lack of decomposition stability in training constraints, and lack of synergistic consideration of reflectance domain noise and enhancement process, as well as the high cost and training difficulty of some solutions in low-light image enhancement under real complex scenes.

[0006] To achieve the above objectives, the present invention provides a pixel-adaptive low-light image enhancement method based on numerically stable decomposition, comprising: The process involves acquiring the low-light input image to be enhanced, its pixel value matrix, and brightness data. The initial illuminance value is calculated for each pixel location based on the pixel value matrix and a predefined illuminance estimation formula. An illuminance value matrix is ​​then formed based on the original pixel row and column coordinates of the low-light input image. Based on the brightness data and the predefined calculation formula of the pixel-level adaptive stabilization term, a pixel-level adaptive stabilization term is generated. Retinex decomposition is performed on the pixel value matrix, the initial illuminance, and the pixel-level adaptive stabilization term to obtain the noisy reflection component. The noisy observation sub-map is obtained by sampling the noisy reflection component using a sampling operator, and a residual noise prediction mapping function is constructed to obtain the predicted noise residual. Based on the predicted noise residual, noise compensation is performed on the noisy reflection component to obtain the denoised reflection component. The pixel value matrix and illumination matrix are input into the enhancement parameter prediction model to output several pixel-level enhancement factors. The brightness gain of the denoised reflection component is adjusted using these pixel-level enhancement factors to output an enhanced image.

[0007] Optionally, the process of acquiring the low-light input image to be enhanced, its pixel value matrix, and brightness data; calculating the initial illuminance measurement corresponding to each pixel position based on the pixel value matrix and a predefined illuminance estimation formula; and forming an illuminance measurement matrix based on the original pixel row and column coordinates of the low-light input image specifically includes: The low-light input image to be enhanced, as well as the pixel value matrix and brightness data of the low-light input image, are extracted using an external image acquisition device. Retrieve a predefined illuminance estimation formula to calculate the initial illuminance value corresponding to each pixel position in the pixel value matrix; Based on the original pixel row and column coordinates of the low-light input image, a two-dimensional row and column arrangement is performed on the initial illumination metric to obtain the illumination metric matrix.

[0008] Optionally, the step of extracting the low-light input image to be enhanced, as well as the pixel value matrix and brightness data of the low-light input image, through an external image acquisition device specifically includes: The low-light input image to be enhanced is extracted using an external image acquisition device. Preprocessing is then performed on the original low-light input image to normalize its format. Alternatively, the low-light input image to be enhanced and normalized is extracted using an external image acquisition device. Map all pixel values ​​in the low-light input image to The interval is used to obtain a pixel value matrix with the same size as the low-light input image; Extract the pixel position corresponding to each matrix coordinate in the pixel value matrix, and perform channel fusion calculation on the pixel value of each color channel of the low-light input image to obtain the brightness channel value of each pixel position, thereby obtaining brightness data.

[0009] Optionally, the step of retrieving a predefined illuminance estimation formula to calculate the initial illuminance value corresponding to each pixel position in the pixel value matrix specifically includes: Retrieve a predefined illuminance estimation formula to calculate the initial illuminance for each pixel position in the pixel value matrix. : In the formula: Represented as color channels; Indicates color channel Red Channel Green Channel Or blue channel ; Represented as matrix coordinates for each pixel position; Represented as a low-light input image in the color channel The pixel value on the screen.

[0010] Optionally, the step of generating a pixel-level adaptive stabilization term based on luminance data and a predefined pixel-level adaptive stabilization term calculation formula, and performing Retinex decomposition on the pixel value matrix, initial illuminance, and pixel-level adaptive stabilization term to obtain a noisy reflection component, specifically includes: Pixel-level adaptive stabilization terms are generated based on brightness data and a predefined formula for calculating pixel-level adaptive stabilization terms. : In the formula: It is represented as the luminance channel value corresponding to the target pixel position in the low-light input image, or the average value of the luminance channel values ​​within a preset neighborhood range of the target pixel position. This is represented as a preset stable lower bound; This is represented as a preset stable upper bound; Based on the Retinex imaging relationship, the pixel value matrix is ​​defined as the product of the illuminance component and the reflectance component, and a noise term is introduced to calculate the noisy reflectance component based on the initial illuminance and pixel-level adaptive stabilization term, combined with the initialization method.

[0011] Optionally, based on the Retinex imaging relationship, the pixel value matrix is ​​defined as the product of the illuminance component and the reflectance component, and a noise term is introduced to calculate the noisy reflectance component based on the initial illuminance and pixel-level adaptive stabilization term, combined with the initialization formula. Specifically, this includes: Based on Retinex imaging relationships, the pixel value matrix Defined as illuminance component With reflection component The product of and gives: ; Let the noise term of the low-light input image be... Then we get: ; Based on the initial illumination metric and pixel-level adaptive stabilization term, combined with the initialization formula, the noisy reflection component is calculated. : In the formula: It is represented as a positive-valued stable function, and , Then we get: .

[0012] Optionally, the step of sampling the noisy reflection component using a sampling operator to obtain a noisy observation sub-map, constructing a residual noise prediction mapping function to obtain a predicted noise residual, and performing noise compensation on the noisy reflection component based on the predicted noise residual to obtain a denoised reflection component specifically includes: Based on the first sampling operator Second sampling operator For the noisy reflection component respectively Perform sampling operations to obtain the first noise observation subgraph. Second noise observation subgraph Its expression is: In the formula: ; A residual noise prediction mapping function is constructed based on an image mapping convolutional network model, and the first noise observation sub-image and the second noise observation sub-image are input into the residual noise prediction mapping function. The reflection noise loss function is constructed based on the output of the residual noise prediction mapping function, and parameter optimization training is performed on the residual noise prediction mapping function. The reflection noise loss function includes at least one of the residual consistency constraint function, the symmetric consistency constraint function, and the smoothness constraint function. Input the noisy reflection component into the trained residual noise prediction mapping function and output the prediction noise residual; In the reflection component domain, noise compensation is performed on the noisy reflection component based on the predicted noise residual to obtain the denoised reflection component. : In the formula: It is represented as the residual noise prediction mapping function after training.

[0013] Optionally, the step of inputting the pixel value matrix and illumination matrix into the enhancement parameter prediction model to output several pixel-level enhancement factors, and using these pixel-level enhancement factors to perform brightness gain adjustment on the denoised reflection component to output an enhanced image, specifically includes: The pixel value matrix and illumination matrix are input into the augmentation parameter prediction model, which consists of several convolutional layers, nonlinear activation functions, and output mapping layers. The enhancement parameter prediction model outputs several pixel-level enhancement factors: ; The denoised reflection component is defined as the initial reflection component of the iteration. And set the first The corresponding pixel-level enhancement factor is Then we obtain the enhancement curve function: In the formula: This is represented as the enhancement curve intensity parameter; This is represented as an exposure balance parameter; And based on a preset number of iterations When the enhancement curve function is iterated to After this, the optimized reflection component matrix is ​​obtained; Based on the pixel value matrix, the denoised reflection component, and the optimized reflection component matrix, a no-reference loss function is constructed, and the no-reference loss function is used to perform end-to-end joint parameter updates on the enhanced parameter prediction model and the residual noise prediction mapping function. The optimized reflection component matrix can be converted into an enhanced image and output, or the optimized reflection component matrix can be multiplied point-by-point with the illumination metric matrix to obtain an enhanced image and output.

[0014] Optionally, the construction of a reference-free loss function based on the pixel value matrix, the denoised reflectance component, and the optimized reflectance component matrix specifically includes: A no-reference loss function is constructed based on the pixel value matrix, the denoised reflection component, and the optimized reflection component matrix. : In the formula: Represented as the structure preservation loss function; Represented as the illuminance constraint loss function; Represented as the color consistency loss function; Represented as the reflection noise loss function; , and These represent the weights of the illuminance constraint loss function, the color consistency loss function, and the reflection noise loss function, respectively.

[0015] Optionally, the step of converting the optimized reflectance component matrix into an enhanced image and outputting it, or multiplying the optimized reflectance component matrix point-by-point with the illumination metric matrix to obtain an enhanced image and outputting it, specifically includes: Based on the original pixel row and column coordinates of the low-light input image, a two-dimensional row and column arrangement is performed on the optimized reflectance component matrix to obtain and output the enhanced image. or, Based on the original pixel row and column coordinates of the low-light input image, the product matrix obtained by multiplying the optimized reflectance component matrix and the illumination matrix point by point is arranged in two dimensions to obtain the enhanced image and output it.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The present invention proposes a pixel-adaptive low-light image enhancement method based on numerically stable decomposition. This method employs a predefined illuminance estimation formula to calculate the initial illuminance from the pixel matrix, achieving pixel-level estimation of the initial illuminance and reducing the interference of local noise on the illuminance estimation results. Then, by introducing a predefined pixel-level adaptive stabilization term calculation formula, a pixel-level adaptive stabilization term is generated to participate in Retinex decomposition processing, achieving numerical stability in the decomposition process and effectively suppressing noise amplification during dark area solving. Furthermore, a sampling operator is used to obtain a noisy observation sub-image, and noise compensation is completed using a residual noise prediction mapping function, achieving accurate denoising in the reflectance component domain, reducing the damage to image texture edges caused by denoising operations, and avoiding color shift phenomena. Additionally, by inputting the pixel matrix and illuminance matrix together into the enhancement parameter prediction model, pixel-level enhancement factors are output to achieve differentiated brightness gain adjustment in different illumination areas, suppressing overexposure and color distortion. Thus, collaborative processing is achieved through a step-by-step chain of decomposition, denoising, and enhancement, eliminating the need for a large number of paired training samples, reducing sample acquisition costs, improving the generalization ability of the enhancement parameter prediction model to complex real-world scenes, and simultaneously considering decomposition stability, reflectance domain noise suppression, and pixel-differentiated enhancement, controlling overall computational overhead, and significantly improving the processing effect of low-illuminance images.

[0017] As can be seen from the above, the technical solution of the present invention can effectively solve the problems of unstable dark area decomposition values ​​causing noise amplification, easy noise interference in illumination estimation, easy texture and color distortion in post-denoising, difficulty in achieving differentiated enhancement by region, high cost of obtaining paired training samples, limited generalization ability, lack of decomposition stability in training constraints, and lack of synergistic consideration of reflectance domain noise and enhancement process, and some solutions have high overhead and difficult training.

[0018] Other features and advantages of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the pixel-adaptive low-light image enhancement method based on numerically stable decomposition, according to an embodiment of the present invention. Detailed Implementation

[0020] To enable those skilled in the art to more fully understand the technical solutions of the present invention, exemplary embodiments of the present invention will be described more comprehensively and in detail below with reference to the accompanying drawings. Obviously, the one or more embodiments of the present invention described below are merely one or more specific ways to implement the technical solutions of the present invention, and are not exhaustive. It should be understood that other ways belonging to a general inventive concept can be used to implement the technical solutions of the present invention, and should not be limited to the embodiments described exemplary. Based on one or more embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0021] Reference Figure 1 The present invention provides a pixel-adaptive low-light image enhancement method based on numerically stable decomposition, comprising: Step S1: Obtain the low-light input image to be enhanced, its pixel value matrix, and brightness data. Calculate the initial illuminance corresponding to each pixel position based on the pixel value matrix and the predefined illuminance estimation formula, and form an illuminance matrix based on the original pixel row and column coordinates of the low-light input image. Step S2: Based on the brightness data and the predefined calculation formula for the pixel-level adaptive stabilization term, generate the pixel-level adaptive stabilization term. Perform Retinex (retina-cortex theory) decomposition on the pixel value matrix, the initial illuminance, and the pixel-level adaptive stabilization term to obtain the noisy reflection component. Step S3: Sample the noisy reflection component using the sampling operator to obtain the noisy observation sub-map, construct the residual noise prediction mapping function to obtain the predicted noise residual, and perform noise compensation on the noisy reflection component based on the predicted noise residual to obtain the denoised reflection component; Step S4: Input the pixel value matrix and illumination matrix into the enhancement parameter prediction model to output several pixel-level enhancement factors. Use these pixel-level enhancement factors to perform brightness gain adjustment on the denoised reflection component to output an enhanced image.

[0022] This invention proposes a pixel-adaptive low-light image enhancement method based on numerically stable decomposition. It employs a predefined illumination estimation formula to calculate the initial illumination from the pixel matrix, achieving pixel-level estimation of the initial illumination and reducing interference from local noise in the illumination estimation results. Furthermore, it introduces a predefined pixel-level adaptive stabilization term calculation formula to generate a pixel-level adaptive stabilization term that participates in Retinex decomposition, achieving numerical stability in the decomposition process and effectively suppressing noise amplification during dark area solving. Finally, it utilizes a sampling operator to obtain a noisy observation sub-image and uses a residual noise prediction mapping function to complete noise compensation, achieving accurate de-noising in the reflectance component domain. This method reduces noise, minimizes the damage to image texture edges caused by denoising operations, avoids color shift phenomena, and achieves differentiated brightness gain adjustment for different lighting regions by inputting the pixel matrix and illumination metric matrix into the enhancement parameter prediction model to output pixel-level enhancement factors. This suppresses overexposure and color distortion, thus completing collaborative processing through a step-by-step chain of decomposition, denoising, and enhancement. It eliminates the need for a large number of paired training samples, reduces sample acquisition costs, and improves the generalization ability of the enhancement parameter prediction model to complex real-world scenes. At the same time, it takes into account decomposition stability, reflection domain noise suppression, and pixel-differentiated enhancement, controls overall computational overhead, and significantly improves the processing effect of low-light images.

[0023] In one embodiment, in step S1, the low-light input image to be enhanced, its pixel value matrix, and luminance data are acquired. An initial illuminance measure corresponding to each pixel position is calculated based on the pixel value matrix and a predefined illuminance estimation formula. An illuminance measure matrix is ​​then formed based on the original pixel row and column coordinates of the low-light input image. Specifically, this includes: Step S11: Extract the low-light input image to be enhanced, as well as the pixel value matrix and brightness data of the low-light input image, through an external image acquisition device; Step S12: Retrieve the predefined illuminance estimation formula to calculate the initial illuminance value corresponding to each pixel position in the pixel value matrix; Step S13: Based on the original pixel row and column coordinates of the low-light input image, perform a two-dimensional row and column arrangement on the initial illumination metric to obtain the illumination metric matrix.

[0024] Specifically, the initial illuminance calculated pixel by pixel is assembled into an illuminance matrix according to the original two-dimensional row and column coordinates of the low-illuminance input image. This can adapt to the illumination differences in different regions within the image, avoid the problem of local illumination distortion caused by global uniform illuminance calculation, thereby reducing the interference of local image noise on the illuminance solution process, and providing accurate, pixel-aligned illuminance input data for subsequent Retinex decomposition processing, reducing the instability of subsequent decomposition solutions caused by illuminance estimation deviations from the source.

[0025] In one specific embodiment, in step S11, the low-light input image to be enhanced, as well as the pixel value matrix and brightness data of the low-light input image, are extracted by an external image acquisition device, specifically including: Step S111: Extract the low-light input image to be enhanced using an external image acquisition device, and perform preprocessing on the original low-light input image to normalize its format; alternatively, extract the low-light input image to be enhanced and normalized using an external image acquisition device. Map all pixel values ​​in the low-light input image to The interval is used to obtain a pixel value matrix with the same size as the low-light input image; Step S112: Extract the pixel position corresponding to each matrix coordinate in the pixel value matrix, and perform channel fusion calculation on the pixel value of each color channel of the low-light input image to obtain the brightness channel value of each pixel position, so as to obtain brightness data.

[0026] Specifically, the external image acquisition device can be a security monitoring device, a vehicle-mounted sensing device, etc. The low-light input image can be an RGB color image or a grayscale image. It is worth noting that if it is a grayscale image, its pixel value is directly taken as the brightness channel value to obtain the brightness data.

[0027] Low-light input images are acquired using an external image acquisition device, and pixel values ​​are mapped to [a specific format] after format normalization. The pixel value matrix obtained from the interval can eliminate the differences in the numerical range of the output images from different imaging devices, ensure the consistency of the input data for the subsequent illumination estimation formula, and also support the direct reception of image data that has already been normalized to reduce the overhead of repeated calculations.

[0028] Then, based on the pixel position corresponding to each matrix coordinate in the pixel value matrix, channel fusion processing such as weighted operation is performed on each color channel of the low-light input image to calculate and extract the brightness channel value of each pixel position, thereby obtaining brightness data. This provides a reliable pixel-level brightness input basis for the subsequent predefined illumination estimation formula, avoiding errors in the initial illumination calculation caused by inconsistent scale of the low-light input image and deviation in brightness information extraction.

[0029] In one specific embodiment, step S12 involves retrieving a predefined illuminance estimation formula to calculate the initial illuminance value corresponding to each pixel position in the pixel value matrix, specifically including: Step S121: Retrieve the predefined illuminance estimation formula to calculate the initial illuminance corresponding to each pixel position in the pixel value matrix. : In the formula: Represented as color channels; Indicates color channel Red Channel Green Channel Or blue channel ; Represented as matrix coordinates for each pixel position; Represented as a low-light input image in the color channel The pixel value on the screen.

[0030] Specifically, by retrieving a predefined illuminance estimation formula, the initial illuminance is solved pixel by pixel based on the matrix coordinates of each pixel position and the pixel values ​​of the color channels of the low-illuminance input image. Illuminance estimation can be completed directly using the original image channel information, achieving pixel-level initial illuminance solution. This effectively improves the precision of illuminance estimation, reduces local illumination estimation distortion caused by global unified calculation, and reduces the deviation of subsequent Retinex decomposition caused by local brightness differences in the input image. This provides an accurate and reliable basic input for the subsequent generation of pixel-level adaptive stable terms and the decomposition to obtain noisy reflection components.

[0031] It is worth noting that the initial illumination metric can also be calculated by directly using the luminance channel value corresponding to each pixel position. Since the luminance channel value is obtained by performing channel fusion calculation on the pixel values ​​of each color channel of the low-light input image, it can directly reflect the brightness of the pixel itself. The luminance channel value is directly equivalent to the incident light intensity at the pixel position, which is simple to calculate, has low computational overhead, and is suitable for scenes with gradual changes in illumination.

[0032] The initial illumination metric can also be calculated based on a preset neighborhood range, by taking the average of the brightness channel values ​​corresponding to local pixel positions in the pixel value matrix. Given that incident light in real-world scenes generally changes continuously and smoothly within a preset neighborhood range, averaging the brightness within the target pixel's neighborhood range can smoothly suppress single-pixel noise interference, filter random noise in low-light images, and obtain more stable illumination estimation results.

[0033] The preset neighborhood range is determined based on a combination of the spatial resolution, local brightness variation scale, and noise level of the low-light input image. In this embodiment, the preset neighborhood range is a 3×3 or 5×5 square neighborhood centered on the target pixel position. The neighborhood can be appropriately increased when noise is strong, and appropriately decreased when texture edges are dense, to balance noise suppression and local structure preservation. The target pixel position is represented by the pixel position currently used for initial illumination measurement calculation, determined pixel-by-pixel according to the original pixel row and column coordinates of the low-light input image; for each target pixel, the preset neighborhood range is determined centered on its pixel position.

[0034] In one embodiment, after performing a two-dimensional row and column arrangement on the initial illumination metric based on the original pixel row and column coordinates of the low-light input image in step S12 to obtain the illumination metric matrix, the method further includes: The illumination metric matrix is ​​mapped to a two-dimensional image representation, and the initial illumination map is output.

[0035] Specifically, by mapping the illuminance matrix to the original pixel row and column coordinates and converting it into a visualized initial illuminance map, the illuminance matrix retains the original calculated values ​​for subsequent algorithm calculations, while the initial illuminance map can intuitively display the illuminance distribution at each pixel location. This not only facilitates the observation of illuminance estimation results and the location of local abnormal pixels during the debugging phase, but also does not change the data content of the illuminance matrix itself. The algorithm can still directly call the illuminance matrix to participate in subsequent Retinex decomposition operations, avoiding the impact of visualization conversion on calculation accuracy.

[0036] In one embodiment, in step S2, a pixel-level adaptive stabilization term is generated based on luminance data and a predefined pixel-level adaptive stabilization term calculation formula. Retinex decomposition is then performed on the pixel value matrix, initial illuminance, and pixel-level adaptive stabilization term to obtain a noisy reflection component. Specifically, this includes: Step S21: Generate a pixel-level adaptive stabilization term based on the brightness data and a predefined formula for calculating the pixel-level adaptive stabilization term. : In the formula: It is represented as the luminance channel value corresponding to the target pixel position in the low-light input image, or the average value of the luminance channel values ​​within a preset neighborhood range of the target pixel position. This is represented as a preset stable lower bound; This is represented as a preset stable upper bound; Step S22: Based on the Retinex imaging relationship, the pixel value matrix is ​​defined as the product of the illuminance component and the reflectance component, and a noise term is introduced to calculate the noisy reflectance component based on the initial illuminance and pixel-level adaptive stabilization term, combined with the initialization method.

[0037] Specifically, a stable lower bound is preset. Used to constrain division by zero or near-zero amplification problems in extremely dark regions, with a preset stable upper bound. This is used to limit the compensation intensity, so that the normal brightness area remains relatively undisturbed, and the obtained pixel-level adaptive stabilization term is a pixel-level parameter, so different stabilization intensities can be obtained at different spatial locations.

[0038] Preset stable lower bound and preset stable upper bound The settings can be adjusted based on the brightness distribution of the low-light input image, the sensor noise level, and the enhancement intensity requirements. For low-light input images with significant noise, the stability lower bound can be appropriately increased, while for images with rich details and low noise, the stability lower bound can be decreased to preserve more local texture.

[0039] In one specific embodiment, in step S22, based on the Retinex imaging relationship, the pixel value matrix is ​​defined as the product of the illuminance component and the reflectance component, and a noise term is introduced to calculate the noisy reflectance component based on the initial illuminance and pixel-level adaptive stabilization term, combined with the initialization formula. Specifically, this includes: Step S221: Based on the Retinex imaging relationship, the pixel value matrix is... Defined as illuminance component With reflection component The product of and gives: ; Step S222: Let the noise term of the low-light input image be... Then we get: ; Step S223: Based on the initial illumination measurement and pixel-level adaptive stabilization term, and combined with the initialization formula, calculate the noisy reflection component. : In the formula: It is represented as a positive-valued stable function, and , Then we get: .

[0040] Specifically, the introduction of noise terms for low-light input images is based on photon shot noise and circuit readout noise generated by the image sensor of the external image acquisition device when acquiring images in low-light environments. These are real interferences that exist in the hardware acquisition process. By explicitly writing them in rather than ignoring them, we can match the actual imaging of real low-light images and avoid the mismatch between the ideal noise-free Retinex and the actual image, which would cause excessive amplification of noise in dark areas. Together with the pixel-level adaptive stabilization term, we can suppress the runaway gain in dark areas and balance the enhancement of dark area information with the preservation of the original content in bright areas.

[0041] Based on the Retinex imaging relationship, the pixel value matrix is ​​modeled as the product of illuminance and reflectance components. A noise term is introduced to construct a noisy imaging function model that closely matches the actual acquisition scene. Combining the initial illuminance measurement and a pixel-level adaptive stabilization term, the noisy reflectance component is solved through an initialization formula. A positive value stabilization function achieves dual constraints of input-side compensation and denominator-side stabilization. In low-illuminance dark areas, the pixel-level adaptive stabilization term increases the effective value of the denominator, thereby constraining the gain of the noisy reflectance component and preventing excessive noise amplification. In high-brightness areas, the pixel-level adaptive stabilization term is constrained by a preset upper bound, preventing excessive intervention in enhancement processing and maximizing the preservation of original scene information. Therefore, the noisy reflectance component obtained in this way provides a reliable foundation for subsequent reflectance denoising, thereby reducing the computational burden of handling abnormal noise and suppressing color artifacts in subsequent processing stages.

[0042] In one embodiment, in step S3, a noisy observation sub-map is obtained by sampling the noisy reflection component using a sampling operator, and a residual noise prediction mapping function is constructed to obtain a predicted noise residual. Noise compensation is then performed on the noisy reflection component based on the predicted noise residual to obtain a denoised reflection component. Specifically, this includes: Step S31: Based on the first sampling operator Second sampling operator For the noisy reflection component respectively Perform sampling operations to obtain the first noise observation subgraph. Second noise observation subgraph Its expression is: In the formula: ; Step S32: Construct a residual noise prediction mapping function based on an image mapping convolutional network model, and input the first noise observation sub-image and the second noise observation sub-image into the residual noise prediction mapping function; Step S33: Construct a reflection noise loss function based on the output of the residual noise prediction mapping function, and perform parameter optimization training on the residual noise prediction mapping function. The reflection noise loss function includes at least one of the residual consistency constraint function, symmetric consistency constraint function, and smoothness constraint function. Step S34: Input the noisy reflection component into the trained residual noise prediction mapping function and output the prediction noise residual; Step S35: In the reflection component domain, perform noise compensation on the noisy reflection component based on the predicted noise residual to obtain the denoised reflection component. : In the formula: It is represented as the residual noise prediction mapping function after training.

[0043] Specifically, this embodiment utilizes two distinct first sampling operators. Second sampling operator Perform downsampling, misaligned sampling, or subgraph sampling operations on the noisy reflection components to obtain the first noise observation subgraph. Second noise observation subgraph The two sets of sub-images correspond to the reflection information of the same scene, but the noise performance is different. Based on the prior that the noise has inherent consistency after the same noisy reflection component is sampled in different ways, a noise observation sub-image is constructed on the reflection component, so that the denoising target is focused on the texture and noise mixed area after low illumination decomposition. This forms the observation samples required to construct the residual noise prediction mapping function, and the denoising model can be trained without the need to obtain a pair of clean reference images.

[0044] It is worth noting that the first sampling operator Second sampling operator These are one of the following operations: downsampling, misaligned sampling, or subgraph sampling.

[0045] Image mapping convolutional network models can be multi-layer convolutional neural networks, lightweight residual networks, or other equivalent network models capable of image mapping. The downsampling operation, misaligned sampling operation, or subgraph sampling operation, as well as the specific structure or principle of the network model, are all existing technologies, and will not be elaborated further in this invention.

[0046] In a preferred embodiment, a reflection noise loss function is constructed by predicting the noise residual, which includes at least one of the residual consistency constraint function, the symmetric consistency constraint function, and the smoothing constraint function. The residual consistency constraint drives the noise prediction results of different sampling sub-graphs to approximate each other, the symmetric consistency constraint ensures that the denoising effect under different sampling directions remains consistent in the sampling space, and the smoothing constraint suppresses the generation of structural artifacts in the prediction results. The network parameters of the mapping function are iteratively updated through backpropagation of the joint loss to complete the training convergence of the residual noise prediction mapping function.

[0047] Let the residual consistency constraint function be... ,get: ; Let the symmetric consistency constraint function be... ,but: In the formula: Represents the squared L2 norm; Let the smoothing constraint function be... ,but: In the formula: Represents the L1 norm; Represented as the horizontal gradient operator; Represented as the vertical gradient operator; Let the reflection noise loss function be Lnoise, then: (1) or, (2) In the formula: , and These represent the weights of the residual consistency constraint function, the symmetric consistency constraint function, and the smoothing constraint function, respectively. Each weight is a non-negative number, and at least one weight is greater than 0. Weights for unselected constraints are set to 0. Each weight is determined based on the numerical magnitude of the corresponding function in the early stages of training, as well as the noise suppression and texture preservation effects. Weights are increased if there are significant differences in the residuals of the noisy observation sub-images. When there are significant differences in denoising results across different sampling directions, the efficiency of denoising can be improved. Improved when predicting noise with local abrupt changes or structural artifacts. When there is significant loss of texture detail, appropriately reduce It is preferable to use all three constraints simultaneously and... Normalized to 1, corresponding to the reflection noise loss function (2).

[0048] This effectively avoids the inherent noise of low-light images being further amplified by subsequent enhancement processing, making the obtained denoised reflection components more consistent with the texture and color information of the real scene, and reducing the burden of processing abnormal noise and color artifacts in subsequent stages.

[0049] In one embodiment, in step S4, the pixel value matrix and illumination matrix are input into the enhancement parameter prediction model to output several pixel-level enhancement factors. These pixel-level enhancement factors are then used to adjust the brightness gain of the denoised reflection component to output an enhanced image. Specifically, this includes: Step S41: Input the pixel value matrix and illumination matrix into the enhancement parameter prediction model, wherein the enhancement parameter prediction model consists of several convolutional layers, nonlinear activation functions and output mapping layers; Step S42: Based on the enhancement parameter prediction model, output several pixel-level enhancement factors: ; Step S43: Define the denoised reflection component as the initial reflection component of the iteration. And set the first The corresponding pixel-level enhancement factor is Then we obtain the enhancement curve function: In the formula: This is represented as the enhancement curve intensity parameter; This is represented as an exposure balance parameter; And based on a preset number of iterations When the enhancement curve function is iterated to After this, the optimized reflection component matrix is ​​obtained; Step S44: Based on the pixel value matrix, the denoised reflection component, and the optimized reflection component matrix, construct a no-reference loss function, and use the no-reference loss function to perform end-to-end joint parameter updates on the enhanced parameter prediction model and the residual noise prediction mapping function; Step S45: Convert the optimized reflectance component matrix into an enhanced image and output it, or multiply the optimized reflectance component matrix point by point with the illumination metric matrix to obtain an enhanced image and output it.

[0050] Specifically, an end-to-end joint parameter update is performed on the enhancement parameter prediction model and the residual noise prediction mapping function using a no-reference loss function, ensuring that the reflection domain denoising results are compatible with the subsequent pixel-level enhancement targets. The residual noise prediction mapping function is used to suppress noise that might be further amplified by the enhancement curve. The enhancement parameter prediction model predicts pixel-level enhancement factors based on the pixel value matrix and illumination metric matrix, and applies these pixel-level enhancement factors to the denoised reflection components for local gain adjustment. After training, the model parameters are fixed, and only forward computation is performed during the inference phase, thus ensuring that noise suppression, structure preservation, and brightness enhancement remain consistent.

[0051] In addition, the intensity parameters of the enhanced curve The exposure balance parameter is used to control the overall perturbation amplitude. It is determined based on the target exposure range of the low-light input image, the allowable brightness change amplitude in a single iteration, and iteration stability, while also being constrained to avoid pixel overexposure, overexposure, or brightness oscillation caused by a single update. The equilibrium position for enhancing the curve function perturbation is set based on the brightness statistics of a normally exposed image.

[0052] It is worth noting that the validation samples are low-light image samples or image patches used to evaluate the enhancement effect under different candidate iteration numbers; for the validation samples, the enhancement parameter prediction model outputs several pixel-level enhancement factors corresponding to the candidate iteration number.

[0053] The preset number of iterations, n, is predetermined based on the convergence degree of the enhancement effect of the enhancement parameter prediction model and the computational cost. Preferably, the number of iterations is gradually increased on the validation samples, and convergence criteria are set based on the normalized pixel range, the average pixel change of adjacent iterations in the validation samples, and the stabilization of brightness, structure, and color indices. When the average pixel change between two adjacent iterations meets the convergence criteria, or when the improvement in brightness, structure, and color indices tends to saturate after further increasing the number of iterations, the corresponding number is determined as the preset number of iterations.

[0054] The enhanced parameter prediction model extracts local spatial features of the image using several convolutional layers. A nonlinear activation function introduces nonlinear expressive power, and the output mapping layer converts the features into the required pixel-level parameters. The pixel value matrix and illumination matrix are input into the enhanced parameter prediction model, which combines the original image information and illumination information to predict suitable enhancement parameters for each pixel location. The specific structure and working principle of the enhanced parameter prediction model are existing technologies and will not be elaborated upon further in this invention.

[0055] By enhancing the parameter prediction model to output pixel-level enhancement factors, pixel-adaptive differential brightness gain is achieved. The optimized reflection component is obtained through iterative enhancement curves. End-to-end joint training of the enhancement parameter prediction model and residual noise prediction mapping is completed using no-reference loss. No external reference image is required. It can effectively improve the brightness of low-light images, suppress overexposure, preserve image texture details, and reduce reflection component noise. At the same time, it supports direct output of optimized reflection component images or reconstruction of enhanced images by combining illumination components, which can adapt to the output requirements of different practical application scenarios.

[0056] In one specific embodiment, in step S44, a no-reference loss function is constructed based on the pixel value matrix, the denoised reflectance component, and the optimized reflectance component matrix, specifically including: Step S441: Construct a no-reference loss function based on the pixel value matrix, the denoised reflectance component, and the optimized reflectance component matrix. : In the formula: Represented as the structure preservation loss function; Represented as the illuminance constraint loss function; Represented as the color consistency loss function; Represented as the reflection noise loss function; , , as well as These represent the weights of the structure preservation loss function, the illuminance constraint loss function, the color consistency loss function, and the reflection noise loss function, respectively.

[0057] Specifically, , , as well as All values ​​are non-negative, and at least one weight is greater than 0; the weight corresponding to a loss function not selected is 0. Preferably, four joint constraints—structure preservation, illuminance constraint, color consistency, and reflection noise—are used simultaneously, and these constraints can be further refined. Normalized to 1. Each weight is determined based on the magnitude of the corresponding loss function in the early training phase, as well as the degree of exposure bias, color shift, noise residue, and texture preservation in the validation samples, ensuring that the contribution of the corresponding loss function to the update of the no-reference loss function is comparable; when exposure anomalies are obvious, the weight is increased. Improve when color shift is obvious Increase when noise residue is significant Improve when texture or edge retention is insufficient .

[0058] It is worth noting that the structure preservation loss function for: In the formula: This represents the number of pixels involved in the calculation of the structure preservation loss; Represented as pixel position index; Represents absolute value; Represented as the first Optimized reflection components at each pixel location; Represented as the first Denoising reflection components at each pixel location; Illumination-constrained loss function for: In the formula: This is represented as optimizing the reflection component; Represented as a local region index; This is represented as optimizing the reflection component. In the The average brightness within a local area; This is represented as the target exposure value determined based on the brightness statistics of the normal exposure area or the training set. Represented as the number of local regions; Color consistency loss function for: In the formula: , and These represent the average values ​​of the three color channels R, G, and B in the enhanced result.

[0059] By allocating the weights of the structure-preserving loss function, illumination constraint loss function, color consistency loss function, and reflection noise loss function to the proportions of each sub-loss, it is possible to simultaneously constrain the denoising effects of texture edges, overall brightness, color performance, and reflection components. This enhances the brightness of low-light input images while preserving the original image texture details, avoiding local abnormal brightness, suppressing color shift during the enhancement process, and constraining the network to reasonably predict noise without destroying the real image structure. By adjusting the weights of each loss, it is also possible to flexibly balance multiple optimization objectives such as image brightness, color, and noise reduction, achieving end-to-end collaborative optimization of the enhancement parameter prediction model and the residual noise prediction module.

[0060] In one specific embodiment, in step S45, the optimized reflectance component matrix is ​​converted into an enhanced image and output, or the optimized reflectance component matrix is ​​multiplied point-by-point with the illumination metric matrix to obtain an enhanced image and output, specifically including: Based on the original pixel row and column coordinates of the low-light input image, a two-dimensional row and column arrangement is performed on the optimized reflectance component matrix to obtain and output the enhanced image. or, Based on the original pixel row and column coordinates of the low-light input image, the product matrix obtained by multiplying the optimized reflectance component matrix and the illumination matrix point by point is arranged in two dimensions to obtain the enhanced image and output it.

[0061] Specifically, this embodiment uses the above two image enhancement output modes to ensure pixel position alignment based on the original pixel row and column coordinates. It can directly output optimized reflection information or restore the complete scene image by combining the illuminance component, adapting to different image output needs.

[0062] In one embodiment, after step S45, which involves converting the optimized reflectance component matrix into an enhanced image and outputting it, or multiplying the optimized reflectance component matrix point-by-point with the illumination metric matrix to obtain an enhanced image and outputting it, the method further includes: Perform pixel cropping or normalization on the enhanced image to ensure that the pixels of the enhanced image are within a preset display range.

[0063] Specifically, the preset display range is determined based on the normalization method of the low-light input image and the output data format: when the low-light input image is processed in normalized floating-point form, the preset display range is [0,1]; when the output is an 8-bit image, [0,1] is linearly mapped to [0,255] and cropped to avoid out-of-bounds pixel values ​​during the enhancement process.

[0064] By maintaining the same spatial dimensions as the low-light input image, the enhanced image can be used for visual observation, image quality improvement, or subsequent visual analysis.

[0065] While one or more embodiments of the present invention have been described above, those skilled in the art will recognize that the present invention can be implemented in any other form without departing from its spirit and scope. Therefore, the embodiments described above are illustrative and not restrictive, and many modifications and substitutions will be apparent to those skilled in the art without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A pixel-adaptive low-light image enhancement method based on numerically stable decomposition, characterized in that, include: The process involves acquiring the low-light input image to be enhanced, its pixel value matrix, and brightness data. The initial illuminance value is calculated for each pixel location based on the pixel value matrix and a predefined illuminance estimation formula. An illuminance value matrix is ​​then formed based on the original pixel row and column coordinates of the low-light input image. Based on the brightness data and the predefined calculation formula of the pixel-level adaptive stabilization term, a pixel-level adaptive stabilization term is generated. Retinex decomposition is performed on the pixel value matrix, the initial illuminance, and the pixel-level adaptive stabilization term to obtain the noisy reflection component. The noisy observation sub-map is obtained by sampling the noisy reflection component using a sampling operator, and a residual noise prediction mapping function is constructed to obtain the predicted noise residual. Based on the predicted noise residual, noise compensation is performed on the noisy reflection component to obtain the denoised reflection component. The pixel value matrix and illumination matrix are input into the enhancement parameter prediction model to output several pixel-level enhancement factors. The brightness gain of the denoised reflection component is adjusted using these pixel-level enhancement factors to output an enhanced image.

2. The pixel-adaptive low-light image enhancement method based on numerically stable decomposition according to claim 1, characterized in that, The process of acquiring the low-light input image to be enhanced, its pixel value matrix, and brightness data; calculating the initial illuminance measurement corresponding to each pixel position based on the pixel value matrix and a predefined illuminance estimation formula; and forming an illuminance measurement matrix based on the original pixel row and column coordinates of the low-light input image specifically includes: The low-light input image to be enhanced, as well as the pixel value matrix and brightness data of the low-light input image, are extracted using an external image acquisition device. Retrieve a predefined illuminance estimation formula to calculate the initial illuminance value corresponding to each pixel position in the pixel value matrix; Based on the original pixel row and column coordinates of the low-light input image, a two-dimensional row and column arrangement is performed on the initial illumination metric to obtain the illumination metric matrix.

3. The pixel-adaptive low-light image enhancement method based on numerically stable decomposition according to claim 2, characterized in that, The step of extracting the low-light input image to be enhanced, as well as the pixel value matrix and brightness data of the low-light input image, through an external image acquisition device specifically includes: The low-light input image to be enhanced is extracted using an external image acquisition device. Preprocessing is then performed on the original low-light input image to normalize its format. Alternatively, the low-light input image to be enhanced and normalized is extracted using an external image acquisition device. Map all pixel values ​​in the low-light input image to The interval is used to obtain a pixel value matrix with the same size as the low-light input image; Extract the pixel position corresponding to each matrix coordinate in the pixel value matrix, and perform channel fusion calculation on the pixel value of each color channel of the low-light input image to obtain the brightness channel value of each pixel position, thereby obtaining brightness data.

4. The pixel-adaptive low-light image enhancement method based on numerically stable decomposition according to claim 3, characterized in that, The process of retrieving the predefined illuminance estimation formula to calculate the initial illuminance value corresponding to each pixel position in the pixel value matrix specifically includes: Retrieve a predefined illuminance estimation formula to calculate the initial illuminance for each pixel position in the pixel value matrix. : In the formula: Represented as color channels; Indicates color channel Red Channel Green Channel Or blue channel ; Represented as matrix coordinates for each pixel position; Represented as a low-light input image in the color channel The pixel value on the screen.

5. The pixel-adaptive low-light image enhancement method based on numerically stable decomposition according to claim 4, characterized in that, The process involves generating a pixel-level adaptive stabilization term based on luminance data and a predefined formula, then performing Retinex decomposition on the pixel value matrix, initial illuminance, and the pixel-level adaptive stabilization term to obtain a noisy reflection component. Specifically, this includes: Pixel-level adaptive stabilization terms are generated based on brightness data and a predefined formula for calculating pixel-level adaptive stabilization terms. : In the formula: It is represented as the luminance channel value corresponding to the target pixel position in the low-light input image, or the average value of the luminance channel values ​​within a preset neighborhood range of the target pixel position. This is represented as a preset stable lower bound; This is represented as a preset stable upper bound; Based on the Retinex imaging relationship, the pixel value matrix is ​​defined as the product of the illuminance component and the reflectance component, and a noise term is introduced to calculate the noisy reflectance component based on the initial illuminance and pixel-level adaptive stabilization term, combined with the initialization method.

6. The pixel-adaptive low-light image enhancement method based on numerically stable decomposition according to claim 5, characterized in that, The Retinex imaging relation defines the pixel value matrix as the product of the illuminance component and the reflectance component, and introduces a noise term to calculate the noisy reflectance component based on the initial illuminance and pixel-level adaptive stabilization term, combined with the initialization formula. Specifically, this includes: Based on Retinex imaging relationships, the pixel value matrix Defined as illuminance component With reflection component The product of and gives: ； Let the noise term of the low-light input image be... Then we get: ； Based on the initial illumination metric and pixel-level adaptive stabilization term, combined with the initialization formula, the noisy reflection component is calculated. : In the formula: It is represented as a positive-valued stable function, and , Then we get: 。 7. The pixel-adaptive low-light image enhancement method based on numerically stable decomposition according to claim 6, characterized in that, The process involves sampling the noisy reflection component using a sampling operator to obtain a noisy observation sub-map, constructing a residual noise prediction mapping function to obtain a predicted noise residual, and performing noise compensation on the noisy reflection component based on the predicted noise residual to obtain a denoised reflection component. Specifically, this includes: Based on the first sampling operator Second sampling operator For the noisy reflection component respectively Perform sampling operations to obtain the first noise observation subgraph. Second noise observation subgraph Its expression is: In the formula: ; A residual noise prediction mapping function is constructed based on an image mapping convolutional network model, and the first noise observation sub-image and the second noise observation sub-image are input into the residual noise prediction mapping function. The reflection noise loss function is constructed based on the output of the residual noise prediction mapping function, and parameter optimization training is performed on the residual noise prediction mapping function. The reflection noise loss function includes at least one of the residual consistency constraint function, symmetric consistency constraint function, and smoothness constraint function. Input the noisy reflection component into the trained residual noise prediction mapping function and output the prediction noise residual; In the reflection component domain, noise compensation is performed on the noisy reflection component based on the predicted noise residual to obtain the denoised reflection component. : In the formula: It is represented as the residual noise prediction mapping function after training.

8. The pixel-adaptive low-light image enhancement method based on numerically stable decomposition according to claim 7, characterized in that, The process of inputting the pixel value matrix and illumination matrix into the enhancement parameter prediction model to output several pixel-level enhancement factors, and then using these pixel-level enhancement factors to adjust the brightness gain of the denoised reflection component to output an enhanced image, specifically includes: The pixel value matrix and illumination matrix are input into the augmentation parameter prediction model, which consists of several convolutional layers, nonlinear activation functions, and output mapping layers. The enhancement parameter prediction model outputs several pixel-level enhancement factors: ； The denoised reflection component is defined as the initial reflection component of the iteration. And set the first The corresponding pixel-level enhancement factor is Then we obtain the enhancement curve function: In the formula: This is represented as the enhancement curve intensity parameter; This is represented as an exposure balance parameter; And based on a preset number of iterations When the enhancement curve function is iterated to After this, the optimized reflection component matrix is ​​obtained; Based on the pixel value matrix, the denoised reflection component, and the optimized reflection component matrix, a no-reference loss function is constructed, and the no-reference loss function is used to perform end-to-end joint parameter updates on the enhanced parameter prediction model and the residual noise prediction mapping function. The optimized reflection component matrix can be converted into an enhanced image and output, or the optimized reflection component matrix can be multiplied point-by-point with the illumination metric matrix to obtain an enhanced image and output.

9. The pixel-adaptive low-light image enhancement method based on numerically stable decomposition according to claim 8, characterized in that, The construction of a no-reference loss function based on the pixel value matrix, the denoised reflectance component, and the optimized reflectance component matrix specifically includes: A no-reference loss function is constructed based on the pixel value matrix, the denoised reflection component, and the optimized reflection component matrix. : In the formula: Represented as the structure preservation loss function; Represented as the illuminance constraint loss function; Represented as the color consistency loss function; Represented as the reflection noise loss function; , and These represent the weights of the illuminance constraint loss function, the color consistency loss function, and the reflection noise loss function, respectively.

10. The pixel-adaptive low-light image enhancement method based on numerically stable decomposition according to claim 9, characterized in that, The process of converting the optimized reflectance component matrix into an enhanced image and outputting it, or multiplying the optimized reflectance component matrix point-by-point with the illumination metric matrix to obtain an enhanced image and outputting it, specifically includes: Based on the original pixel row and column coordinates of the low-light input image, a two-dimensional row and column arrangement is performed on the optimized reflectance component matrix to obtain and output the enhanced image. or, Based on the original pixel row and column coordinates of the low-light input image, the product matrix obtained by multiplying the optimized reflectance component matrix and the illumination matrix point by point is arranged in two dimensions to obtain the enhanced image and output it.

Images