An adaptive image enhancement method under unstable visual conditions

Abstract

This invention discloses an adaptive image enhancement method under unstable visual conditions, comprising the following steps: Step 1, image segmentation of the input image; Step 2, image enhancement through weight adjustment; Step 3, extraction of contrast in underexposed areas using AIES, and progressive enhancement following decomposition consistency; Step 4, achieving illumination smoothness and chromaticity balance through correlation guidance of high-frequency components in an encoder-decoder style illumination restoration network (IR-Net); Step 5, improvement of latency and constraint loss function through hyperparameters in Step 2, where the total variation (TV) plays a role in protecting boundaries and edges under the guidance of AIES. This invention employs pointwise hyperparameters to improve visibility, brightness, color, and contrast under unstable visual conditions through local and global adjustments, and embeds the AIES scheme into a degraded image enhancement network, achieving adaptive image enhancement through loss adjustment influence learning.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, and in particular to an adaptive image enhancement method under unstable visual conditions. Background Technology

[0002] Modern computer vision applications are built on high-quality images. The ability to capture these images with the latest digital devices has led to their widespread adoption. However, their advantages are limited to applications operating under optimal lighting and typical weather conditions. Achieving the same results in dim / backlit environments is a significant problem when camera settings cannot meet the requirements under unstable visual conditions (e.g., extreme darkness / backlight, fog, and haze). In-vehicle cameras / devices suffer from low brightness, low contrast, and color fading, ultimately distorting the structure and texture of the final image under unstable visual conditions (EVC). These terrible EVCs appear to be a major obstacle for vision applications from dawn to dusk. It is essential to develop new image augmentation strategies with minimal latency, independent of data type, with the ultimate goal of semantic scene understanding. Over the past decade, the computer vision community has made significant progress in augmenting low-quality images in various environments. Existing low-light image augmentation methods can be categorized into supervised learning (i.e., relying on image pairs), unsupervised learning (i.e., relying on large-scale datasets), and direct augmentation methods.

[0003] Direct enhancement methods distort color, contrast, and global structure, amplifying artifacts while increasing brightness. Network-based methods, on the other hand, rely heavily on the type of training data (i.e., training pairs, large-scale, and carefully crafted data). Directly capturing large datasets or training image pairs can lead to critical failures when dealing with unstable visual conditions in the field. Capturing image pairs in such scenarios presents a more comprehensive challenge than ever before.

[0004] In such scenarios, one solution is to use gamma correction to synthesize image pairs or consider expert-based retouching. However, gamma correction-based specification limits prediction if the required reservation level is lacking in the training data, and retouching requires resources and time. In either case, preparing large-scale datasets or relying on image pairs increases computational complexity and overall cost. Therefore, the proposed method is also robust to visual anomalies caused by unpredictable and unstable conditions, requiring repeated, arduous training if the required reservation level is missing from the training database. The failure of existing network-based methods in EVCs is primarily due to a lack of physical knowledge, making it difficult to interpret the true manifestations of intrinsic visual anomalies. A practical method must produce satisfactory results in the aforementioned scenarios and must be computationally efficient, operating with minimal latency and low memory footprint. Therefore, novel approaches are needed to interpret the practical significance of EVCs in potential real-world applications and consumer photography. Summary of the Invention

[0005] The purpose of this invention is to provide an adaptive image enhancement method under unstable visual conditions. This invention employs a novel adaptive image enhancement scheme that utilizes pointwise hyperparameters to improve visibility, brightness, color, and contrast under unstable visual conditions through local and global adjustments. Furthermore, it embeds the AIES scheme into a degraded image enhancement network, achieving adaptive image enhancement through loss-adjusted influence learning.

[0006] To solve this technical problem, the technical solution of the present invention is: an adaptive image enhancement method under unstable visual conditions.

[0007] Step 1: Perform image segmentation on the input image:

[0008] The input image is segmented into reflection R and illumination T by a multilayer image segmentation network (MIS-Net) to initialize subsequent enhancement operations.

[0009] Step 2: Enhance the image through weight adjustment: Using an effective weighting function to darken the haze image can benefit the image under degraded visual conditions in extremely dark background scenes.

[0010] Step 3: Extract contrast in underexposed areas using AIES and perform progressive enhancement following the principle of decomposition consistency: This includes reflection enhancement. and lighting upgrade The parameters are:

[0011] ,

[0012] Step 4: Achieve illumination smoothness and chromaticity balance through correlation guidance of high-frequency components in an encoder-decoder style illumination restoration network (IR-Net): The input reflections and illumination estimates of the encoder-decoder style IR-Net network are as follows: and Its specific expression is: ,

[0013] Step 5: The hyperparameters from Step 2 help improve the delay and constraint loss functions, where the total change (TV) plays a role in protecting the boundaries and edges under the guidance of AIES.

[0014] A multi-layer image segmentation network (MIS-Net) is preferred. MIS-Net starts by extracting features from a stack of 3×3 convolutional layers. The decomposed components are mapped using rectified linear units (ReLU), activation functions, and a stack of 3×3 convolutional layers. Hyperbolic tangent (tanh) is used to constrain specified ranges for reflection and illumination.

[0015] The preferred approach is AIES, which includes parameter tuning operations and loss adjustment strategies. Progressive enhancement of reflection upgrades is achieved through parameter tuning operations. Parameter and lighting upgrades Parameters. Loss adjustment strategies help suppress artifacts.

[0016] The optimal loss function in MIS-Net is selected. Image reconstruction loss is used. Reflection loss and light loss This describes the loss of the multi-layer segmentation network. Overall. It is represented as the sum of these three distance terms. ,in Constraining image reconstruction error, Remove reflection irregularities, where the reflection loss is constrained to... of Norm, The estimate is as follows. . It limits lighting inconsistencies and prevents the amplification of related structural and textural flaws.

[0017] AIES preferably uses exponential decay and growth to control the image gradient to suppress unwanted artifacts. This is achieved by adjusting the reflection... and lighting The residual image is weighted by introducing parameters, and then the consistency of reflection and illumination is decomposed.

[0018] ,

[0019] We preferentially utilize TV operations to preserve strong priors, enabling image recovery and enhancement in degraded visual scenes. This also eliminates the blindness of using total variation loss in the network loss and forces TV as a strong preservation prior. Image gradients are penalized to avoid boundary artifacts and preserve strong edges after enhancement operations.

[0020] The DIE network is preferably trained from scratch using the Adam Optimizer and backpropagation with the TensorFlow framework to enhance poorly lit input images. The network training lasts for 100 epochs with a learning rate of 1e. -3 The batch size is 16, and the image block size is 128x128.

[0021] By adopting the above technical solution, the beneficial effects of the present invention are:

[0022] This invention proposes a hybrid framework that can operate regardless of the type of training dataset. The AIES strategy of this invention, embedded in the network, can directly alter the illumination balance. The proposed hyperparameter tuning can directly induce local and global adjustments to maintain chromaticity balance and extract contrast.

[0023] Qualitative and quantitative comparative experiments on several benchmark datasets demonstrate that the proposed method outperforms existing state-of-the-art methods. The proposed framework improves performance under unstable visual conditions without relying on large-scale pairwise training datasets.

[0024] Thus, the above-mentioned objectives of the present invention are achieved. Attached Figure Description

[0025] Figure 1 This invention relates to an overall framework diagram of an adaptive image enhancement method under unstable visual conditions;

[0026] Figure 2 This describes the PSNR performance of the AIES network proposed in this invention on images in the UXOV dataset.

[0027] Figure 3 This invention upgrades total reflection and illumination. and The behavior of the parameters is used to improve the graph of intensity versus hyperparameter n;

[0028] Figure 4 This is the result of the present invention using enhancement operations to remove inconsistencies in the reflection components;

[0029] Figure 5 This invention uses lighting adjustment operations to achieve a lightmap with acceptable structure and texture. Detailed Implementation

[0030] To further explain the technical solution of the present invention, the present invention will be described in detail below through specific embodiments.

[0031] This embodiment discloses an adaptive image enhancement method under unstable visual conditions, and the specific model architecture diagram is shown below. Figure 1 As shown, it includes the following steps:

[0032] Step 1, such as Figure 1 As shown, image segmentation is performed on the input image:

[0033] The input image is segmented by a multi-layer image segmentation network (MIS-Net). The image is divided into reflection R and illumination T to initialize subsequent enhancement operations.

[0034] Step 2: Image Enhancement through Weight Adjustment: Using an effective weighting function to darken hazy images can benefit them under degraded visual conditions, even in extremely dark backgrounds. Darkening hazy images is beneficial for low-light images and can be achieved through inverse operations. Enhancing underexposed areas through smoothness balancing requires an effective and adaptive weighting function, achieved through the progression of transport components, where N = 1,.....,255.

[0035]

[0036] Step 3: Extract contrast in underexposed areas using AIES and perform progressive enhancement following the principle of decomposition consistency: This includes reflection enhancement. and lighting upgrade The parameters are:

[0037] , ,in .

[0038] Locally adjusted parameters m = 1, ..., 255 and adaptive global hyperparameters 0 < n ≤ 1. The required enhancements are initialized; in this work, m varies from 128 to 256, and n varies from 0.5 to 0.7. Figure 3 The effects of the aforementioned uphill parameters and adjacent global hyperparameter n are shown. These parameter adjustments summarize visibility under degraded visual conditions to maintain sharp reflection and lighting maps. (It can be found in...) Figure 4 (ah) and Figure 5 As seen in (ah), several reflection and lighting maps demonstrate a generalized approach to visualization under several degraded visual conditions. To achieve adaptive balance and lighting consistency, strict control of the image gradient is required in each case to preserve structure, texture, and boundaries. On the other hand, undesirable texture magnification degrades global structure and edges. Exponential decay and growth are employed to control the image gradient to suppress undesirable artifacts. This is achieved by applying reflective... and lighting This is achieved by introducing weighting parameters into the residual image and then decomposing the consistency of reflection and illumination. Reflection and lighting The expression is shown below.

[0039] ,

[0040] Step 4: Achieve illumination smoothness and chromaticity balance through correlation guidance of high-frequency components in an encoder-decoder style illumination restoration network (IR-Net): The input reflections and illumination estimates of the encoder-decoder style IR-Net network are as follows: and Its specific expression is: , The IR-Net network is an encoder-decoder unit with skip connections from downsampling to upsampling (3x3), following a ReLU activation function with a stride of 2. After each channel of the upsampling block, features are rescaled at multiple scales using nearest-neighbor interpolation. Rescaling follows a stack of convolutional layers with ReLU and a stride of 1. In contrast, downsampling blocks consist of a stack of convolutional layers with a stride of 2 and a ReLU.

[0041] Step 5: The hyperparameters from Step 2 help improve the latency and constraint loss function, where the total variation (TV), guided by AIES, acts to protect the boundaries and edges. This aims to reduce the latency and constraint loss function at pixel location p(x) along the illumination and reflection components. i To reduce distortion and black spots, the steepness of the image gradient is adjusted while minimizing TV. Per-pixel illumination and reflection adjustments on the horizontal axis p(h±1, v) and vertical axis p(h, v±1) are continuously updated. and .

[0042] ,

[0043] TV preserves structural and textural details through proposed exponential growth and decay to control changes in dramatic scenes. The final image is the product of lighting enhancement and reflection.

[0044] In this embodiment, the dataset, evaluation metrics, and network optimization settings are configured sequentially, as follows:

[0045] (1) Dataset

[0046] We chose LOL, a variable exposure dataset consisting of 500 pairs of images, and UXOV, a variable lighting condition dataset consisting of 1018 images of various objects taken in poorly lit scenes with predetermined camera and lighting conditions.

[0047] (2) Evaluation indicators

[0048] We conduct an objective comparison of several state-of-the-art methods. This objective comparison relies on metrics based on both full-reference and non-reference images. We use Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), Luminance Order Enhancement (LOE), Learned Perceptual Patch Similarity (LPIPS), and Naturalness Image Quality Evaluator (NIQE) as objective metrics. Higher PSNR and SSIM values ​​indicate higher visual quality, while lower LOE and NIQE values ​​and LPIPS indicate higher image quality, and vice versa.

[0049] (3) Network optimization settings

[0050] We trained the DIE network from scratch using the Adam Optimizer and backpropagation with the TensorFlow framework to augment poorly lit input images. Our network was trained for 100 epochs with a learning rate of 1e. -3 The batch size is 16, and the image patch size is 128x128. We tested DIE net on multiple images, including challenging indoor and outdoor datasets.

[0051] This embodiment verifies the role of each proposed component in the framework, namely the effectiveness of MIS-Net, IR-Net, and IAIES, through ablation experiments.

[0052] The results of the ablation study are shown in Table 1. The test-data-sample-C dataset was selected to evaluate the performance of the ablation under several degraded visual conditions. We removed the AIES module from the network to investigate the effect of each component. The comparison of PSNR and SSIM illustrates the network performance. Regarding PSNR, the overall performance of the network with and without AIES is as follows: Figure 2 As shown.

[0053] Objective Comparison: Section 2 shows a comparison between the proposed method and several existing methods, using the LOL dataset. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods in objective metrics such as PSNR, SSIM, NIQE, and MEF. Next, Table 3 illustrates the performance of our method on a variable illumination condition dataset, namely the UXOV dataset, for Average PSNR, SSIM, and Average SSIM metrics, where our method achieves numerical superiority.

[0054] Table 1 presents the ablation experiments conducted on the key modules MIS-Net, IR-Net, and AIES modules in DIENet.

[0055] frame PSNR SSIM MIS-Net+IR-Net 9.24 0.49 MIS-Net 7.84 0.43 MIS-Net+AIES 11.67 0.65 MIS-Net+AIES+IR-Net 15.69 0.72 Gain 4.45 0.23

[0056] Table 2 compares PSNR, SSIM, LOE, and NIQE on the LOL dataset.

[0057] method PSNR SSIM NIQE LOE LIME 16.7586 0.5644 8.3777 1909.5 FEW 18.7916 0.6422 8.8770 2051.7 SLIMER 13.8765 0.6577 5.8101 2025.5 SRIE 11.8552 0.4979 7.2869 1745.4 Retinex-Net 16.7740 0.5594 8.8785 2449.3 NPE 16.9697 0.5894 8.4390 2076.3 GLAD 19.7182 0.7035 6.4755 1795.5 KinD 20.8665 0.8022 5.1461 2012.2 DIE-Net 23.9574 0.8486 4.2017 1680.2

[0058]

[0059] This invention proposes a practical framework consisting of a multi-layer image segmentation network (MIS network) for segmenting an input image into reflections R and illumination T to initialize subsequent enhancement operations. The proposed AIES is customized within the DIE-Net network, where reflections are adjusted and illumination smoothness is improved to preserve structure and texture. The proposed strategy provides efficient weight upscaling while being embedded in the network to constrain scene radiance, where high-frequency reflection components extract contrast from the darkest areas. Furthermore, illumination smoothness and chromaticity balance are achieved through correlation guidance obtained from high-frequency components in an encoder-decoder style illumination inpainting network (IR-Net). Additionally, the hyperparameters in this scheme contribute to improving latency and constraining the loss function, where the total variation (TV), guided by AIES, acts to protect boundaries and edges. The final image is the product of illumination enhancement and reflection.

Claims

1. An adaptive image enhancement method under unstable visual conditions, characterized in that: Includes the following steps: Step 1: Perform image segmentation on the input image: The input image is segmented into a reflection component R(x) and an illumination component T(x) by a multi-layer image segmentation network to initialize subsequent enhancement operations; Step 2: Image Enhancement through Weight Adjustment: An effective weighting function is used to darken the input image, which can benefit the image under degraded visual conditions, even in extremely dark backgrounds. The transfer component... N = 1,...,255; Step 3: Extract the contrast of underexposed areas in the input image using an adaptive image enhancement strategy, and perform progressive enhancement following decomposition consistency: This includes reflection enhancement. and lighting upgrade The parameters are: , , where m is the local adjustment parameter and n is the adaptive global hyperparameter; where the adaptive global hyperparameter n helps to improve the delay and constraint loss function; Step 4: Achieve illumination smoothness and chromaticity balance through correlation guidance of high-frequency components in the encoder-decoder style illumination restoration network: The input reflection estimation and illumination estimation of the encoder-decoder style illumination restoration network are respectively... and Its specific expression is: , Where R(x) is the reflection component and T(x) is the illumination component; Step 5: The total change TV, guided by an adaptive image enhancement strategy, protects boundaries and edges. This total change TV preserves structural and texture details through proposed exponential growth and decay to control changes in drastic scenes, aiming to alleviate the impact of each pixel (p) along the illumination and reflection components at pixel location p(x). i To reduce distortion and black spots, the steepness of the image gradient is adjusted while minimizing TV. Per-pixel illumination and reflection adjustments on the horizontal axis p(h±1, v) and vertical axis p(h, v±1) are continuously updated. and , 、 。 2. The adaptive image enhancement method under unstable visual conditions as described in claim 1, characterized in that: A hybrid framework is proposed, which includes: MIS-Net, a multi-layer image segmentation network; AIES, an adaptive image enhancement strategy; and IR-Net, an illumination inpainting network.

3. An adaptive image enhancement method under unstable visual conditions as described in claim 2: the adaptive image enhancement strategy includes parameter adjustment operations and loss adjustment strategies.

4. The adaptive image enhancement method under unstable visual conditions as described in claim 3, characterized in that: The loss adjustment strategy in the adaptive image enhancement strategy helps suppress artifacts; image reconstruction loss is used. Reflection loss and light loss To describe the loss of a multi-layer segmentation network; overall It is expressed as the sum of these three distance terms. ,in Constraining image reconstruction error, Remove reflection irregularities, where the reflection loss is constrained to... of Norm, The estimate is as follows: ; It limits illumination inconsistency and prevents the amplification of related structural and textural defects, where R is the reflection component, T is the illumination component, and I is the input image.

5. The adaptive image enhancement method under unstable visual conditions as described in claim 4, characterized in that: The lighting is updated using the relevant consistency of reflections in the lighting restoration network to remove noise and artifacts.

6. The adaptive image enhancement method under unstable visual conditions as described in claim 4, characterized in that: The proposed method can introduce adaptive lighting balance into images without requiring a whole tedious training process for new scenes.

7. The adaptive image enhancement method under unstable visual conditions as described in claim 4, characterized in that: Image gradients are controlled using exponential decay and growth to suppress unwanted artifacts, and this is achieved by decomposing the consistency of reflection and illumination; where the reflection weighting function... Illumination weighting function The calculation is performed as follows: , Where N=1,...,255, n is the global hyperparameter, T(x) is the illumination component, and R(x) is the reflection component.

8. The adaptive image enhancement method under unstable visual conditions as described in claim 6, characterized in that: The DIE network was trained from scratch using the Adam Optimizer and backpropagation with the TensorFlow framework to enhance poorly lit input images; the method is as follows: the network was trained for 100 epochs with a learning rate of 1e. -3 The batch size is 16, and the image block size is 128x128.

Images