Low-light large dynamic image enhancement method based on dynamic weight and pyramid fusion

Abstract

The invention provides a low-light large dynamic image enhancement method based on dynamic weight and pyramid fusion. The low-light large dynamic image enhancement method is suitable for complex illumination imaging in security monitoring. According to the method, the global weight and the detail weight of an image are obtained according to the global average brightness, the local brightness and the gradient of an image sequence, and the total weight is obtained through weighting. And the halo phenomenon easily occurring in the fusion process is reduced through guide filtering. And combining Gaussian pyramid and ratio pyramid decomposition and weighting to obtain a final fusion image. According to the method, the problems of detail loss and limited dynamic range in the low-light scene are effectively solved. Experiments show that the method is superior to the prior art in information entropy, average gradient, edge strength, spatial frequency and other key indexes. The technology has an all-time adaptive capability, is suitable for the fields of intelligent transportation, urban security and protection, low-illumination monitoring and the like, and has a wide market prospect.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing and computer vision, and in particular to a low-light large-dynamic image enhancement method based on dynamic weight and pyramid fusion. This method is based on dynamic weight calculation and pyramid fusion strategy to improve the imaging quality under complex lighting conditions and is applicable to application scenarios such as security monitoring, intelligent transportation, and urban remote sensing. Background Art

[0002] At present, significant progress has been made in low-light image enhancement research, which can be divided into three categories: traditional image enhancement methods, image fusion methods, and deep learning-based methods.

[0003] Early research mostly started from the perspective of image processing, such as histogram equalization (HE) and algorithms based on the Retinex theory. The Retinex theory decomposes the illumination component and reflectance component of an image to enhance contrast and eliminate the influence of non-uniform illumination, and is commonly used for low-light enhancement and color correction. The local histogram equalization method proposed by Pizer et al. (1987) improved the local contrast of medical images; Rahman et al. (1996) introduced multi-scale Retinex, which enhanced image details while suppressing noise and became an important theoretical cornerstone for subsequent research. Subsequently, Rahman et al. (2004), Setty et al. (2013), and Gonzales et al. (2015) made various improvements to the Retinex algorithm, further improving the contrast and denoising ability of images under low-light conditions. However, since most of these methods lack sufficient preservation of edge information, when encountering scenes with both strong and weak light such as low-light large-dynamic, halation amplification or overall over-brightness is likely to occur, affecting the visual quality of the image and the development of subsequent detection and recognition tasks.

[0004] To better preserve details and balance bright and dark regions, multi-exposure or multi-sensor image fusion techniques have received extensive attention. Yin et al. (2010) used the non-subsampled contourlet transform and fuzzy logic to fuse infrared and visible light images, improving visibility in low-light environments; Wang et al. (2011), Singh et al. (2019), Imran et al. (2019), and Wu et al. (2023) successively proposed image fusion ideas based on multi-scale pyramids. Among them, the pyramid decomposition method extracts feature information at different scales by decomposing images at multiple levels, which helps with detail enhancement and artifact suppression. This method has achieved the effect of reducing detail loss and artifacts in fields such as remote sensing, medical, and consumer cameras. Compared with traditional enhancement methods, image fusion pays more attention to information complementarity between different exposures or different source images, and can alleviate the problem of coexistence of overexposure and underexposure to a certain extent. However, under complex lighting conditions, how to efficiently calculate the fusion weights and simultaneously suppress halos and noise amplification in strong light regions remains a direction worthy of further research. In addition, multi-sensor image acquisition and registration increase the system cost and volume, which is contrary to the lightweight requirements of actual security monitoring deployments.

[0005] In recent years, deep learning has shown great potential in the field of image enhancement. Zhang et al. (2019) constructed a deep network called KinD based on the Retinex theory, Lv et al. (2021) introduced an attention mechanism into a multi-branch convolutional network, and Chi et al. (2022) designed a multi-scale feature fusion network with pyramid attention, PAMF-NET, all of which achieved adaptive enhancement in low-light or complex lighting scenarios, improving the detail expression and visual effect of images. Xing et al. (2023) combined adaptive learning with a convolutional neural network, showing good denoising and enhancement capabilities under complex lighting. However, deep learning methods rely on large-scale training data and hardware computing resources, and the complexity of the network structure also leads to high costs for model deployment and update.

[0006] In summary, existing low-light high-dynamic-range image enhancement techniques, whether based on traditional image processing, image fusion, or deep learning, have to some extent solved the problems of inability to balance bright and dark regions, easy loss of details, and halo amplification. However, in practical applications, they still face the following two challenges: one is the limitation of computing and hardware resources, which requires considering the enhancement effect of images with limited hardware costs; the other is the adaptability to variable lighting environments. The lighting differences in different scenes and time periods require the method to have higher robustness and self-adaptability. Further research is needed in this regard. Summary of the Invention

[0007] In view of the above-mentioned deficiencies of the prior art, the present invention provides a low-light large dynamic range image enhancement method based on dynamic weight and pyramid fusion, and provides a multi-exposure image fusion method that can balance the details of overexposed and underexposed areas under medium and low hardware configurations. The content of the present invention is as follows: The object of the present invention is to provide a low-light large dynamic range image enhancement method based on dynamic weight and pyramid fusion. The technical points thereof include the following steps: S1. Multi-exposure image acquisition: Acquire a sequence of multiple frames of images with different exposure degrees in the same scene, including at least one image with darker exposure and one image with brighter exposure; S2. Feature calculation: Calculate the global average brightness, local brightness, and gradient for each frame of image respectively; S3. Dynamic weight generation and smoothing: Based on the global average brightness, local brightness, and gradient, generate a global weight and a detail weight, and use guided filtering to smooth the weights to suppress halos and noise during the fusion process; S4. Pyramid decomposition: Decompose each frame of image into several scale subband coefficients through multi-scale pyramid decomposition; S5. Multi-scale weighted fusion: Based on the smoothed global weight and detail weight, perform weighted fusion on each layer of scale subband coefficients layer by layer, and gradually stack the fusion results from the low layer to the high layer; S6. Image reconstruction: Upsample the fused subband coefficients layer by layer to obtain a high-fidelity fused image.

[0008] To better implement the above technical solution, the calculation of the global average brightness in the low-light large dynamic range image enhancement method based on dynamic weight and pyramid fusion of the present invention includes: Statistically analyze the brightness distribution of multiple frames of input images to obtain the overall brightness level; Filter and smooth the overall brightness distribution to eliminate the influence of noise or abnormally exposed areas; Based on the smoothed brightness information, construct a global average brightness for distinguishing between over-dark scenes and over-bright scenes.

[0009] To better implement the above technical solution, the calculation of the local brightness in the low-light large dynamic range image enhancement method based on dynamic weight and pyramid fusion of the present invention includes: Use guided filtering to locally smooth the pixel brightness of each frame of image; Retain the brightness gradient difference between image regions, and enhance the visible details of the dark low-light regions before fusion; Use the smoothed result as the local brightness for distinguishing different brightness regions during the fusion process.

[0010] To better implement the above technical solution, the acquisition of gradients in the micro low-light large dynamic range image enhancement method based on dynamic weights and pyramid fusion of the present invention includes: Perform gradient calculation on the image to extract high-frequency components containing texture and detail information; Combine the high-frequency components with luminance information to distinguish high-contrast regions from flat regions; During multi-scale fusion, apply higher weights to detail regions based on gradients to enhance detail performance.

[0011] To better implement the above technical solution, the generation and smoothing of dynamic weights in the micro low-light large dynamic range image enhancement method based on dynamic weights and pyramid fusion of the present invention specifically include: Calculate the initial luminance weight according to the global average luminance and local luminance, which is used to balance the fusion ratio between the bright part and the dark part; Calculate the detail weight according to the gradient, which is used to enhance the fusion weight of high-frequency details; Use guided filtering to smooth the luminance weight and detail weight respectively, suppressing the halo and over-enhancement phenomena.

[0012] To better implement the above technical solution, the process of multi-scale weighted fusion in the micro low-light large dynamic range image enhancement method based on dynamic weights and pyramid fusion of the present invention includes: Decompose the weight matrix through Gaussian pyramid decomposition, and decompose the input image through ratio pyramid decomposition; Apply weight smoothing in the bright-dark transition region to avoid halos and white edges; Upsample level by level from the low level to the high level, and adjust the weights during high-scale fusion to maintain overall visual consistency.

[0013] To better implement the above technical solution, the method in the micro low-light large dynamic range image enhancement method based on dynamic weights and pyramid fusion of the present invention is applicable to the fields of smart cities, intelligent transportation, and urban street scene monitoring, for realizing all-day large dynamic range imaging and maintaining high-fidelity performance of dark part details in night or low-light environments.

[0014] Compared with the prior art, the micro low-light large dynamic range image enhancement method based on dynamic weights and pyramid fusion of the present invention has the following advantages: The method for enhancing micro low-light large dynamic images based on dynamic weight and pyramid fusion of the present invention does not require constructing a complex physical imaging model and does not rely on high-performance computing devices to obtain images with relatively high fusion quality. It can not only enhance the dark details in low-light environments but also effectively suppress overexposure and halo phenomena in high-brightness regions, achieving brightness balance and detail enhancement. Compared with traditional multi-scale fusion methods, the present invention has significant advantages in suppressing halos, retaining dark details, and adapting to various large dynamic lighting environments, meeting the imaging requirements for all-day and high-robustness in fields such as security monitoring and intelligent transportation.

[0015] The concept, specific structure, and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, features, and effects of the present invention. Brief Description of the Drawings

[0016] Figure 1 is the flowchart of the method for enhancing micro low-light large dynamic images based on dynamic weight and pyramid fusion of the present invention; Figure 2 is the overall comparison of processing results at different time points; Figure 3 is the line chart of six evaluation indicators of each method in 24 hours; Figure 4 is the detail presentation effect during a period with good lighting; (a) Scene at 7 a.m.; (b) Scene at 10 a.m.; (c) Scene at 1 p.m.; (d) Scene at 4 p.m.; Figure 5 is the detail presentation effect of the window during the low-light period; (a) Scene at 7 p.m.; (b) Scene at 10 p.m.; (c) Scene at 1 a.m.; (d) Scene at 4 a.m. Detailed Embodiments

[0017] The following introduces multiple preferred embodiments of the present invention with reference to the accompanying drawings of the specification to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the protection scope of the present invention is not limited to the embodiments mentioned in the text.

[0018] As Figure 1 shown, the main process of a multi-exposure image fusion method based on dynamic weight and pyramid fusion of the present invention includes the following steps: ‌1. Acquisition and preprocessing of multi-exposure images‌ Obtain an image sequence of multiple frames with different exposure levels in the same scene, including at least one underexposed frame (rich in dark details) and one overexposed frame (clear in bright details). Preprocess the images for alignment and denoising to ensure the accuracy of the fusion input.

[0019] 2. Dynamic Feature Calculation and Weight Generation Global Average Brightness Calculation: Statistically analyze the overall brightness distribution of the input images, and eliminate the interference of noise and abnormally exposed areas through guided filtering and smoothing processing to generate a global average brightness index (μ global ) for distinguishing over-dark / over-bright scenes.

[0020] Local Brightness Optimization: Use guided filtering to locally smooth the pixel brightness of each frame of the image, retain the brightness gradient difference, and enhance the visible details in the weak light areas of the dark part to obtain a refined local brightness distribution (l i >). [[ID=]14]

[0021] Gradient Feature Extraction: Extract the high-frequency components (such as edges and textures) of the image through gradient calculation, and distinguish high-contrast areas from flat areas in combination with brightness information.

[0022] [[ID=)18]3. Dynamic Weight Allocation and Smoothing Weight Generation: Based on the global average brightness and local brightness distribution, calculate the initial brightness weight (used to balance the ratio of the bright part and the dark part), and generate the detail weight in combination with the gradient feature (used to enhance high-frequency details).

[0023] Weight Smoothing: Use guided filtering to jointly optimize the brightness weight and detail weight, suppress the fusion artifacts (such as halos and white edges) caused by weight mutations, and at the same time reduce the influence of noise.

[0024] 4. Multi-scale Pyramid Decomposition and Fusion Pyramid Decomposition: Decompose each frame of the input image into multi-scale subband coefficients through ratio pyramid decomposition (the low scale retains the global brightness, and the high scale retains the details).

[0025] Layer-by-layer Weighted Fusion: On different scale subbands, dynamically adjust the fusion ratio according to the smoothed global weight and detail weight: Low-level Subband: Focus on global brightness balance, use the weight matrix of Gaussian pyramid decomposition to ensure smooth transition between bright and dark; High-level Subband: Focus on detail retention, decompose the image through ratio pyramid, and apply weight smoothing constraints at the bright-dark junction; Highest-level Fusion: Adjust the weight in combination with the overall visual effect to ensure that the output image has natural transition and consistency in day, night, and strong light / weak light scenes.

[0026] 5. Image Reconstruction and Optimization The upsampled and superimposed multi-scale subband coefficients after fusion are used to reconstruct the final high-fidelity fused image layer by layer. During the reconstruction process, the halos are suppressed by the multiplication strategy while the dark details and high-frequency textures are retained. Embodiment

[0027] As Figure 2 shown, to verify the effectiveness and capabilities of this method, the scene was photographed throughout the day and compared with three advanced image fusion methods. The following figure shows the overall effect. Generally speaking, the GFF method is prone to halos at the junction of buildings and the sky; the Li17 method makes the overall brightness darker in backlit scenes, such as during periods with large lighting contrasts like 11 a.m., and there are cases where the overall brightness transition of the image is inconsistent due to changes in cloud thickness (11 p.m., 12 p.m.) at night; the effect of the Li20 method during the day is similar to that of Li17, and it is slightly insufficient in terms of brightness transition in the image. However, during the low-light period at night, its brightness and contrast are good, but in scenes with LED light strips (7 p.m., 8 p.m.), the image is darker than in nearby time periods. In contrast, this method can well transition the overall brightness of each time period during the day, will not get dark when backlit, and has no obvious halo problem; when there are both strong light and dark parts at night, it can also maintain a reasonable and natural transition, and there will be no abnormal brightness transition of the image due to environmental changes, demonstrating the strong adaptability and robustness of this method.

[0028] In terms of quantitative analysis, we use six evaluation metrics to compare the performance of the four methods throughout the day and night. Among them, the entropy (EN) reflects the amount of information and the richness of details in the image; the average gradient (AG) and edge intensity (EI) measure the structural clarity and edge sharpness of the image respectively; the standard deviation (SD) and spatial frequency (SF) measure the breadth of brightness distribution and the richness of textures in the image respectively; the human visual perception metric (Qcb) approximately simulates the human eye's perception of the overall visual consistency of the image. Figure 3 The line charts of the evaluation metrics in

[0029] Under daytime lighting conditions, both GFF and this method have relatively high EN values (both greater than 15), indicating that both are quite rich in information content. However, under low-light conditions, the EN of this method is significantly higher than that of the GFF method, with an improvement range of 9.76% to 20.28%. This shows that in low-light environments, this method can retain more texture and details. For the AG and EI metrics, this method also performs better than other methods during low-light periods, with an improvement of approximately 2% to 23.56% compared to the second-best method, demonstrating its advantage in maintaining the sharpness of low-light details. In terms of the SD metric, this method has a more obvious advantage in daytime scenes, while being slightly lower than Li20 in low-light scenes. The reason is that the application of guided filtering smooths the weights while sacrificing the brightness contrast to some extent. In terms of SF, GFF performs excellently during the day, while during low-light periods, this method obtains higher evaluation values, with an improvement of approximately 7.09% to 41.94% compared to the second-best method. The Qcb metric shows that this method has a better visual evaluation in daytime scenes, while being comparable to the GFF evaluation metric under low-light conditions, with a difference ranging from 0.07% to 4.05%. Generally speaking, this method shows the most prominent overall performance in each period, and is basically comparable to the best results of other methods in individual periods.

[0030] To further examine the detail presentation effect under good daytime lighting, Figure 4 four scenes at 7 a.m., 10 a.m., 1 p.m., and 4 p.m. were selected for local magnification, and the evaluation metrics of these four groups of scenes were compared using a radar chart ( Figure 5 ). GFF and Li17 can provide stronger edge sharpening in some areas, so the AG, EI, and SF metrics are generally high, but there are also slight halos or overly unnatural phenomena. The overall brightness of Li20 is relatively soft, but there is no fine processing in areas with rich details such as window frames, resulting in blurriness in the magnified details. This method processes the transition between strong light and shadow more evenly, avoiding the loss of details caused by the bright-dark boundary.

[0031] Higher requirements are placed on the robustness of the fusion algorithm for night-time low-light and large dynamic range scenes. Figure 5Shows a local detail magnification comparison of four night-time periods: 7 p.m., 10 p.m., 1 a.m., and 4 a.m. Among them, the 7 p.m. scene is a low-light dynamic scene, and the lighting in the 10 p.m., 1 a.m., and 4 a.m. scenes is even weaker. From the detailed magnification of the window, in the result image processed by this method, the lines and textures above the interior of the window are clearer, the contrast between light and dark inside the window is higher, and this method effectively avoids overexposure through multi-scale adaptive weights and guided filter smoothing, and the details in the dark areas can still be well preserved. Overall, this method has significantly better effects in more complex and important low-light scenes than other methods.

[0032] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations based on the concept of the present invention without creative labor. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field based on the concept of the present invention through logical analysis, reasoning, or limited experiments on the basis of the prior art should fall within the protection scope determined by the claims.

Claims

1. A low-light and large dynamic image enhancement method based on dynamic weight and pyramid fusion, characterized in that: It includes the following steps: S1. Multi-exposure image acquisition: Acquire a sequence of multiple frames of images with different exposure degrees in the same scene, including at least one image with darker exposure and one image with brighter exposure; S2. Feature calculation: Calculate the global average brightness, local brightness, and gradient for each frame of the image respectively; S3. Dynamic weight generation and smoothing: Based on the global average brightness, local brightness, and gradient, generate a global weight and a detail weight, and use guided filtering to smooth the weights to suppress halos and noise during the fusion process; S4. Pyramid decomposition: Decompose each frame of the image into several scale sub-band coefficients through multi-scale pyramid decomposition; S5. Multi-scale weighted fusion: Based on the smoothed global weight and detail weight, perform weighted fusion on each scale sub-band coefficient layer by layer, and gradually stack the fusion results from the low layer to the high layer; S6. Image reconstruction: Upsample the fused sub-band coefficients layer by layer to obtain a high-fidelity fused image.

2. The low-light-level dynamic image enhancement method based on dynamic weighting and pyramid fusion according to claim 1, characterized in that: The calculation of the global average brightness includes: Statistically analyze the brightness distribution of multiple frames of input images to obtain the overall brightness level; Filter and smooth the overall brightness distribution to eliminate the influence of noise or abnormally exposed areas; Based on the smoothed brightness information, construct a global average brightness for distinguishing between over-dark scenes and over-bright scenes.

3. The low-light-level dynamic image enhancement method based on dynamic weighting and pyramid fusion according to claim 1, characterized in that: The calculation of the local brightness includes: Use guided filtering to perform local smoothing on the pixel brightness of each frame of the image; Retain the brightness gradient difference between image regions, and enhance the visible details of the dark low-light regions before fusion; Take the smoothed result as the local brightness for distinguishing different brightness regions during the fusion process.

4. The low-light-level dynamic image enhancement method based on dynamic weighting and pyramid fusion according to claim 1, characterized in that: The acquisition of the gradient includes: Perform gradient calculation on the image to extract high-frequency components containing texture and detail information; Combine the high-frequency components with the brightness information to distinguish high-contrast regions from flat regions; During multi-scale fusion, apply a higher weight to the detail regions based on the gradient to enhance the detail performance.

5. The low-light-level dynamic image enhancement method based on dynamic weighting and pyramid fusion according to claim 1, characterized in that: The dynamic weight generation and smoothing specifically include: Calculate an initial brightness weight based on the global average brightness and local brightness for balancing the fusion ratio between the bright part and the dark part; Calculate a detail weight based on the gradient for enhancing the fusion weight of high-frequency details; Use guided filtering to smooth the brightness weight and detail weight respectively to suppress halos and over-enhancement phenomena.

6. The low-light-level dynamic image enhancement method based on dynamic weighting and pyramid fusion according to claim 1, characterized in that: The process of multi-scale weighted fusion includes: Perform Gaussian pyramid decomposition on the weight matrix and ratio pyramid decomposition on the input image; Apply weight smoothing in the bright-dark transition region to avoid halos and white edges; Upsample step by step from the low level to the high level, and adjust the weights during high-scale fusion to maintain overall visual consistency.

7. The low-light-level dynamic image enhancement method based on dynamic weighting and pyramid fusion according to claim 1, characterized in that: The method is applicable to the fields of smart cities, intelligent transportation, and urban street view monitoring, for realizing all-day large dynamic range imaging and maintaining high-fidelity performance of dark part details in night or low-light environments.

Images