Lightweight illumination and structure calibration low-light image enhancement method based on hvi space

Abstract

The present application provides a lightweight lighting and structure calibration low-light image enhancement method based on HVI space, relates to the technical field of low-light image enhancement, and the present application decouples a low-light image to be calibrated into an HV color component and an I brightness component through an HVI color space variable, and respectively inputs a double-branch enhancement network; an I branch decoding stage generates a calibrated lighting feature through multi-scale average pooling, up-sampling, adaptive coefficient aggregation, exposure perception gating and residual retention operation; an HV branch decoding stage generates an enhanced HV branch intermediate feature through channel average pooling, channel maximum pooling, projection mapping, structure-sensitive attention modulation and residual refinement; the enhanced HV branch intermediate feature and the I feature are fused, an enhanced image is obtained through inverse HVI color space transformation, local lighting adaptive calibration and structure-sensitive region detail strengthening are realized, and the problems of local uneven exposure, overexposure of bright areas and weakening of texture edges of a low-light image are alleviated.

Description

Technical Field

[0001] This invention relates to the field of low-light image enhancement technology, specifically to a lightweight illumination and structure calibration low-light image enhancement method based on HVI space. Background Technology

[0002] Low-light images are usually caused by acquisition conditions such as nighttime, weak indoor lighting, backlight, backlight, or underexposure. The core task of low-light image enhancement technology is to recover images with sufficient brightness, natural colors, and clear structure from input images with insufficient lighting, and to provide reliable input for subsequent visual tasks such as target detection, recognition, tracking, and scene understanding.

[0003] In recent years, deep learning methods have significantly improved enhancement quality by learning the mapping relationship between low-light images and normal-light images using large-scale data. However, many networks still directly process brightness restoration, chromaticity preservation, and detail reconstruction simultaneously in the RGB space, leading to the coupled optimization of different degradation factors in the same feature space. This easily results in problems such as insufficient local exposure calibration, unstable texture structure restoration, and poor model interpretability. Among these, low-light image enhancement methods based on the HVI space provide an effective foundation for branched restoration of illumination and chromaticity detail components by explicitly decoupling the low-light image into HV and I components. However, existing HVI dual-branch networks may still have two shortcomings in the decoding stage: First, although the I branch can estimate illumination intensity, it lacks a local exposure calibration mechanism for spatially non-uniform degradation, which can easily lead to insufficient enhancement in dark areas or overexposure in bright areas; second, although the HV branch is responsible for chromaticity and detail restoration, it lacks explicit structure-sensitive guidance, which can easily lead to detail weakening in texture, edge, and object contour areas.

[0004] Therefore, how to further improve the local illumination calibration capability of the I branch and enhance the structural sensitivity refinement capability of the HV branch while maintaining the decoupling advantages of the HVI color space and the lightweight nature of the dual-branch network is a technical problem that urgently needs to be solved in the field of low-light image enhancement.

[0005] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a lightweight illumination and structure calibration low-light image enhancement method based on HVI space, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A lightweight illumination and structure calibration low-light image enhancement method based on HVI space, the specific steps of which include: Step 1: Convert the low-light image to be calibrated to the HVI color space, separate the HV color component and the I luminance component, and input the two types of components into the trained dual-branch enhancement network. After encoding and decoding, extract the intermediate features of the I branch used for illumination correction and the intermediate features of the HV branch used for structural detail preservation. Step 2: Perform multi-scale average pooling and upsampling restoration on the intermediate features of branch I to generate multi-scale low-frequency illumination response. Generate adaptive aggregation coefficients for each scale through projection mapping. Weighted fuse the multi-scale low-frequency illumination response to obtain the illumination calibration basis. Step 3: The illumination calibration base and the intermediate feature of the I branch are spliced ​​together to generate an exposure-sensing gating map that can identify overexposed, underexposed, and normal areas. At the same time, the residual preservation terms of the two are constructed. The calibrated illumination features are calculated by combining the exposure-sensing gating map and the residual preservation terms. Step 4: Perform channel averaging and max pooling on the intermediate features of the HV branch, and concatenate them to obtain spatial structure information. Generate a structure-sensitive attention map through the projection layer. At the same time, perform pointwise convolution, nonlinear activation and depthwise separable convolution lightweight processing on the intermediate features of the HV branch to generate residual refined features. Step 5: Modulate the residual thinning feature using a structure-sensitive attention map, and superimpose the modulated residual thinning feature onto the original HV branch intermediate feature to obtain the enhanced HV branch intermediate feature. Then, stitch the calibrated illumination feature with the enhanced HV branch intermediate feature, reconstruct the image through inverse HVI spatial transformation, and output the low-light enhanced image.

[0008] Furthermore, the low-light image to be calibrated is converted to the HVI color space, and the HV color component and I luminance component are separated. The specific logic is as follows: The conventional RGB low-light image to be calibrated is mapped to the HVI color space, decoupling the image color information from the brightness information, and independently outputting the HV joint color component that represents the image hue and saturation, as well as the I brightness component that represents the image brightness. The HV branch intermediate features are structural color features obtained by encoding and decoding HV color components through a dual-branch enhancement network, which are used to characterize image texture, edge and color information. The intermediate features of the I branch are illumination features obtained by encoding and decoding the I brightness component through a dual-branch enhancement network, which are used to characterize the brightness distribution information of the image.

[0009] Furthermore, multi-scale average pooling and upsampling are performed on the intermediate features of branch I to generate multi-scale low-frequency illumination responses. The specific logic is as follows: For the intermediate features of the I branch that characterize the brightness distribution of the image, downsampling is performed using average pooling operations of different scales to extract global and local low-frequency information of illumination under different receptive fields; The pooled features at each scale are upsampled and restored one by one to the original feature size, thus obtaining a low-frequency illumination response that adapts to different illumination ranges and corresponds to multiple scales. Adaptive aggregation coefficients at various scales are generated through projection mapping, and the multi-scale low-frequency illumination responses are weighted and fused to obtain the illumination calibration basis. The specific logic is as follows: Multi-scale low-frequency illumination responses are input into a linear projection layer for feature mapping learning. Based on the differences in the contribution of illumination responses at different scales to the overall illumination calibration of the image, aggregate weight coefficients for the corresponding scales are adaptively generated. The corresponding low-frequency illumination response is weighted by adaptive aggregation weight coefficients at each scale, and the low-frequency illumination response after weighting at all scales is fused to generate an illumination calibration base that adapts to the brightness distribution of the entire image and has adaptive correction capabilities.

[0010] Furthermore, the average pooling operation using different scales requires at least two average pooling processes of different scales, and includes at least 5×5 average pooling and 9×9 average pooling.

[0011] Furthermore, the illumination calibration base and the intermediate features of the I branch are stitched together to generate an exposure-sensing gating map that can identify overexposed, underexposed, and normal areas. The specific logic is as follows: The illumination calibration base and the intermediate features of the I branch, which characterize the brightness distribution information of the original image, are concatenated by channel dimension features to fuse global illumination prior information and local original brightness information. By comparing complementary dual feature information, the system automatically identifies overexposed, underexposed, and normally lit areas in an image, generating an exposure perception gating map with pixel-level exposure perception capabilities. The dual feature information consists of the illumination calibration base and the intermediate feature of the I branch.

[0012] Furthermore, residual preservation terms are constructed for both, and by combining the exposure-aware gating map and the residual preservation terms, the calibrated illumination characteristics are calculated. The specific logic is as follows: The two refer to the illumination calibration base and the intermediate feature of the I branch. Based on the global standard brightness distribution represented by the illumination calibration base, the difference operation is performed with the original pixel brightness information represented by the intermediate feature of the I branch to construct the residual preservation term, which effectively captures the original brightness details of the image and the deviation of the real illumination, and preserves the inherent local brightness texture and light and shadow layers of the image. Pixel-level differential illumination adjustment is achieved by relying on the exposure-sensing gating map, which adaptively corrects the brightness deviation of overexposed and underexposed areas of the image. At the same time, the brightness and darkness details of the illumination adjustment result are constrained and compensated by the residual preservation term. Finally, the calibrated illumination features with uniform illumination and complete brightness and darkness details are obtained by fusion calculation.

[0013] Furthermore, the refined features of the modulated residual are superimposed onto the original HV branch intermediate features to obtain the enhanced HV branch intermediate features. The specific logic is as follows: The modulated residual refinement feature is a structural compensation feature after being constrained by the weights of the structure-sensitive attention map. The modulated residual refinement feature is incorporated into the original HV branch intermediate feature by residual superposition. This completes and corrects the missing and blurred structural details in the original HV branch intermediate feature under low light conditions, while preserving the inherent color tone and basic structural information of the original HV branch intermediate feature without color shift or structural distortion. The result is an enhanced HV branch intermediate feature with clear texture, complete structure, and faithful color.

[0014] Furthermore, the image is reconstructed by stitching together the calibrated illumination features and the enhanced intermediate features of the HV branch, and then subjected to inverse HVI spatial transformation to output a low-light enhanced image. The specific logic is as follows: The calibrated illumination features with optimized brightness are combined with the intermediate features of the HV branch with enhanced structural color by channel dimension splicing and fusion to form a complete HVI space-optimized feature map. The HVI space optimized feature map is subjected to inverse HVI space transformation, and the decoupled and optimized brightness, color and structural information is remapped back to the original RGB image space to complete the overall reconstruction of the image pixel level. The final output is a high-quality low-light enhancement image with uniform lighting, true colors, clear texture details, and no obvious distortion artifacts.

[0015] Compared with the prior art, the beneficial effects of the present invention are: This invention extracts low-frequency illumination response by performing multi-scale average pooling on the intermediate features of the I branch, then projects and maps to generate adaptive aggregation coefficients at each scale, and generates an illumination calibration basis by weighted aggregation. The calibrated illumination features are obtained by processing the exposure-aware gating map and residual preservation terms, which effectively alleviates the problems of uneven local exposure, insufficient enhancement of dark areas and overexposure of bright areas in low-light images. This invention also performs average pooling and channel max pooling operations on the intermediate features of the HV branch, and after splicing, inputs them into the projection layer to obtain a structure-sensitive attention map. This allows the dual-branch enhancement network to prioritize the enhancement of areas with obvious changes in texture, edges, contours and local details, improving the effect of softened details and blurred boundaries during the enhancement process. Then, the structure-sensitive attention map is used to modulate the residual thinning features, and the modulated residual thinning features are superimposed on the original intermediate features of the HV branch, avoiding overcorrection, oversmoothing and structural damage caused by directly covering the original features. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall method flow of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0018] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0019] Example: Please see Figure 1 The present invention provides a technical solution: A lightweight illumination and structure calibration low-light image enhancement method based on HVI space, the specific steps of which include: Step 1: Convert the low-light image to be calibrated to the HVI color space, separate the HV color component and the I luminance component, and input the two types of components into the trained dual-branch enhancement network. After encoding and decoding, extract the intermediate features of the I branch used for illumination correction and the intermediate features of the HV branch used for structural detail preservation. In scenarios such as nighttime shooting, indoor low light, backlight, backlight, underexposure, low-light monitoring, or low-light mobile imaging, RGB images with problems such as overall low brightness, uneven local exposure, loss of detail in dark areas, risk of overexposure in bright areas, color shift, weakened texture edges, and blurred structural contours are collected as regular RGB low-light images to be calibrated. In order to ensure the stability of subsequent network processing, the low-light RGB images are first subjected to normalization preprocessing, which maps the image pixel values ​​from the original value range to a preset value range. For example, the RGB image pixel values ​​are normalized to the [0,1] interval to keep the pixel dimensions consistent between different images and reduce the numerical differences caused by different exposure conditions, different acquisition devices, and different scene brightness distributions. Based on the above scheme, the conventional RGB low-light image to be calibrated is mapped to the HVI color space, decoupling the image's color and brightness information. The HV joint color component, representing the image's hue and saturation, and the I brightness component, representing the image's lightness and darkness, are output independently. The I brightness component primarily carries the image's illumination intensity, brightness distribution, and exposure state, reflecting information such as insufficient illumination in dark areas, uneven local exposure, unnatural transitions between light and dark areas, and low-frequency illumination variations in the low-light image. The HV color component primarily carries the image's chroma, color distribution, texture edges, and structural details, reflecting information such as object contours, local textures, color variations, and boundary structures in the low-light image. Through this decoupling process, subsequent networks can perform differentiated processing for illumination restoration and chroma structure restoration respectively, avoiding the feature coupling problem caused by simultaneously optimizing brightness, color, and texture in the RGB space.

[0020] The HV branch intermediate features are structural color features obtained by encoding and decoding the HV color components through a dual-branch enhancement network, used to characterize image texture, edges, and color information; the I branch intermediate features are illumination features obtained by encoding and decoding the I brightness components through a dual-branch enhancement network, used to characterize image brightness distribution information. Based on the above, it should be noted that: Degradation in low-light images typically encompasses both illumination degradation and structural degradation. Illumination degradation manifests as overall darkness, underexposure in dark areas, unnatural transitions between bright and dark regions, and overexposure in bright areas. Structural degradation manifests as blurred edges, weakened textures, softened details, and unstable colors. Directly performing uniform enhancement in the RGB space requires a dual-branch enhancement network to simultaneously perform brightness enhancement, color preservation, and structural detail restoration within the same feature space, which can easily lead to interference between the enhancement targets. For example, excessive brightness enhancement may cause overexposure in bright areas and color drift, while excessive texture preservation may limit illumination restoration in dark areas. Therefore, this embodiment decomposes the low-light RGB image into HV chromaticity components and I luminance components through HVI color space transformation. This allows the I branch to primarily handle illumination restoration, while the HV branch handles chromaticity and structural detail restoration, providing a clearer and more stable feature base for subsequent branching processing.

[0021] The HV color component and I luminance component are input into a dual-branch enhancement network to extract intermediate features of the I branch and HV branch during the decoding process. The dual-branch enhancement network includes an HV branch and an I branch. The HV branch is used to encode, extract features, decode, and restore structural details of the HV color component, focusing on color, chroma, texture, edge, and contour information. The I branch is used to encode, extract features, decode, and restore illumination intensity of the I luminance component, focusing on brightness, exposure, low-frequency illumination trends, and local illumination distribution information. In this dual-branch enhancement network, the two branches are not completely independent during network processing. Instead, they exchange information through a cross-branch feature interaction mechanism, enabling the illumination branch to obtain edge, color change, and contour position references from the structure branch, and enabling the structure branch to obtain brightness distribution and exposure state references from the illumination branch.

[0022] Step 2: Perform multi-scale average pooling and upsampling restoration on the intermediate features of branch I to generate multi-scale low-frequency illumination response. Generate adaptive aggregation coefficients for each scale through projection mapping. Weighted fuse the multi-scale low-frequency illumination response to obtain the illumination calibration basis. First, the intermediate features of the I branch Multi-scale average pooling is performed to extract low-frequency illumination responses under different receptive fields. Due to changes in spatial size, the pooled features require further upsampling to restore them to the intermediate features of the I branch. Consistent spatial resolution facilitates subsequent scale alignment and feature aggregation. The formulas underlying the low-frequency illumination response at multiple scales are as follows: in This represents the intermediate features of the I-branch input to the lightweight illumination calibration module during the I-branch decoding stage. Indicates the core size is Average pooling operation, Indicates the first The pooling kernel size corresponding to each scale. Indicates the first Upsampling restoration operations corresponding to each scale Indicates the first Low-frequency lighting response at various scales Indicates the number of scales.

[0023] The multi-scale average pooling includes at least 5×5 average pooling and 9×9 average pooling. In an optional implementation, K=2. The 5×5 average pooling is used to extract illumination change trends within smaller local areas, responding more sensitively to exposure changes at local light-dark boundaries, small bright areas, and local dark areas. The 9×9 average pooling is used to extract low-frequency brightness distribution information within larger local areas, better describing large dark areas, overall insufficient brightness, and widespread illumination attenuation trends. By simultaneously employing both 5×5 and 9×9 average pooling scales, both fine-grained local exposure calibration and large-scale low-frequency illumination compensation can be achieved.

[0024] Based on the above, it should be noted that: Illumination degradation in low-light images typically exhibits spatial non-uniformity, meaning that overexposed, underexposed, and normally exposed areas coexist within the same image. If only a single scale is used to extract the illumination response, smaller scales tend to focus only on local variations while neglecting the overall brightness trend, while larger scales ignore exposure differences in small areas. Therefore, this embodiment employs multi-scale average pooling to obtain low-frequency illumination responses under different receptive fields, enabling the lightweight illumination calibration module to simultaneously perceive the brightness distribution of both local and larger areas, providing a more stable low-frequency prior for subsequent adaptive illumination calibration.

[0025] Based on the above embodiments, low-frequency illumination responses at multiple scales are input into the linear projection layer to adaptively generate response scores for each scale. Adaptive aggregation weight coefficients for each scale are then obtained based on Softmax normalization, using the following formula: in, This represents a linear projection layer, and is a lightweight projection function used for feature mapping of low-frequency illumination responses across multiple scales. This represents the response score corresponding to the k-th scale. This represents the aggregate weighting coefficient corresponding to the low-frequency lighting response at the k-th scale, where k represents the index of the scale.

[0026] Based on the above, it should be noted that: Different low-light images and different image regions have varying degrees of dependence on scale information. For large dark areas, larger-scale low-frequency illumination responses are more likely to reflect overall insufficient brightness; for areas where light and dark meet or locally bright areas, smaller-scale low-frequency illumination responses are more likely to reflect local exposure differences. Therefore, this embodiment does not simply average the low-frequency illumination responses at multiple scales, but instead uses a lightweight projection function to predict the response scores at each scale and obtains adaptive aggregation weights through Softmax normalization. This allows the network to automatically determine the contribution ratio of different scales in the current illumination calibration process based on the image content. Through this adaptive weight allocation method, the network can achieve a more flexible balance between dark area enhancement, bright area suppression, and transition area calibration.

[0027] Based on the above embodiments, the low-frequency lighting responses at multiple scales are weighted and aggregated according to the adaptive aggregation weights to obtain the formula on which the lighting calibration basis is based, as follows: in, This indicates element-wise multiplication. This represents the illumination calibration basis obtained by weighted aggregation of low-frequency illumination responses at multiple scales, and serves as the illumination prior in subsequent exposure calibration processes.

[0028] Based on the above, it should be noted that: The illumination calibration base is essentially a local brightness trend representation obtained by adaptively fusing illumination responses at different scales. This base contains both local exposure variation information within a smaller receptive field and overall brightness distribution information within a larger receptive field, providing a unified low-frequency illumination prior for dark area enhancement, bright area suppression, and smooth calibration of tonal transitions. Compared to directly enhancing the original I-component, calibration based on the illumination calibration base reduces the interference of noise and high-frequency textures on illumination estimation, resulting in more stable illumination recovery.

[0029] Step 3: The illumination calibration base and the intermediate feature of the I branch are spliced ​​together to generate an exposure-sensing gating map that can identify overexposed, underexposed, and normal areas. At the same time, the residual preservation terms of the two are constructed. The calibrated illumination features are calculated by combining the exposure-sensing gating map and the residual preservation terms. In this example, the illumination calibration base is not used directly. Instead of covering the original I-branch intermediate features, an exposure-aware gating map is generated based on the I-branch intermediate features and the illumination calibration basis, and residual preservation terms are constructed to control the illumination calibration intensity and retain the effective structural information in the original features.

[0030] The formula used to generate the exposure-sensing gating map by stitching together the intermediate features of the illumination calibration base and the I branch is as follows: In the formula, This represents the exposure-aware gating map. This represents the Sigmoid activation function. Represents a lightweight projection operator; The formula for constructing the residual preservation term is as follows: The global standard brightness distribution represented by the illumination calibration basis is interpolated with the original pixel brightness information represented by the intermediate features of the I branch. In the formula, This indicates the residual preservation term, which effectively captures the original brightness details of the image and the deviation from the actual lighting, preserving the inherent local brightness and shadow textures and light and shadow levels of the image. Pixel-level differential illumination adjustment is achieved based on the exposure-aware gating map, adaptively correcting brightness deviations in overexposed and underexposed areas of the image. Simultaneously, a residual preservation term constrains and compensates for brightness and darkness details in the illumination adjustment results. Finally, the calibrated illumination features with uniform illumination and complete brightness and darkness details are calculated. The formula used to calculate the calibrated illumination features is as follows: In the formula, and Represents the learnable coefficient. This indicates the illumination characteristics after calibration.

[0031] Based on the above, it should be noted that: If directly using the lighting calibration base Replace or cover intermediate features of branch I This could lead to the loss of effective details in the original lighting features, resulting in problems such as over-smoothing, overexposure in bright areas, or over-enhancement in dark areas. Therefore, this embodiment uses an exposure-sensing gating map. Adaptive control of calibration intensity. For areas requiring stronger lighting compensation. Able to improve lighting calibration base The degree of effect; for areas that were originally exposed relatively normally or had a relatively stable structure. It can reduce calibration intensity and utilize residual preservation terms. Effective brightness variations and structural stability information are preserved in the original features. and With two learnable coefficients, the network can automatically adjust the ratio of calibration contribution to residual preservation contribution during training, thereby achieving stable, moderate, and region-adaptive lighting calibration.

[0032] Step 4: Perform channel averaging and max pooling on the intermediate features of the HV branch, and concatenate them to obtain spatial structure information. Generate a structure-sensitive attention map through the projection layer. At the same time, perform pointwise convolution, nonlinear activation and depthwise separable convolution lightweight processing on the intermediate features of the HV branch to generate residual refined features. First, channel average pooling and channel max pooling are performed on the intermediate features of the HV along the channel dimension to obtain spatial structure information. Then, the spatial structure information is input into the projection layer, and a structure-sensitive attention map is generated by using the Sigmoid activation function, based on the following formula: in, This represents a structure-sensitive attention map. Represents a lightweight spatial projection function. This indicates a channel average pooling operation. This indicates a channel max pooling operation. This represents the intermediate features of the HV branch.

[0033] Building upon the above embodiments, channel average pooling is used to obtain the overall response distribution of HV branch intermediate features across the channel dimension, reflecting the average structural strength of a region across multiple channels. Channel max pooling is used to obtain the salient response distribution of HV branch intermediate features across the channel dimension, highlighting locations with significant local edges, texture abrupt changes, and contour variations. By concatenating or fusing the channel average pooling and channel max pooling results and inputting them into a lightweight spatial projection function, the generated structure-sensitive attention map can simultaneously possess both overall structure perception and local salient structure capture capabilities.

[0034] Based on the above, it should be noted that: In low-light image enhancement, simply increasing brightness does not necessarily restore sharp structures. Edges, textures, and contours in dark areas are often weakened due to underexposure. If the network lacks explicit structural guidance, the enhancement result may suffer from softened details, blurred boundaries, and texture discontinuities. Channel average pooling reflects the overall response level of HV branch intermediate features across all channels, making it suitable for describing the global average distribution of structural regions. Channel max pooling highlights strong response locations in certain channels, making it suitable for capturing significant structural information such as edges, contours, and texture abrupt changes. Using channel average pooling alone may weaken local salient structures, while using channel max pooling alone may overemphasize local strong responses and ignore overall structural continuity. Therefore, this embodiment combines channel average pooling and channel max pooling to generate a structure-sensitive attention map, enabling the network to simultaneously perceive the overall structural distribution and local salient edges, thereby improving the ability of HV branch intermediate features to recover textures, contours, and details.

[0035] Building upon the above embodiments, to avoid issues such as softened details, blurred boundaries, and texture discontinuities that may occur in the enhancement results if the network lacks explicit structural guidance, a method is adopted to further enhance the structural details of the intermediate features of the HV branch: a lightweight residual refinement branch is set. This lightweight residual refinement branch includes point convolution, nonlinear activation functions, and 3×3 depthwise convolution, which are structural compensation features constrained by structure-sensitive attention map weights. The formula used to generate the residual refinement features is as follows: In the formula, This indicates a lightweight residual refinement feature. This represents a point convolution operation. This represents a 3×3 depth convolution operation.

[0036] Based on the above, it should be noted that: Pointwise convolutions are used to fuse information between channels without significantly increasing computation, allowing chroma, texture, and edge responses in different channels to complement each other. Nonlinear activation functions are used to enhance the network's ability to express complex structural changes. 3×3 depthwise convolutions are used to capture local structural patterns in the spatial neighborhood, enhancing edge, texture, and contour information. Compared to ordinary convolutions, depthwise convolutions have fewer parameters and lower computational overhead, making them suitable for building lightweight pluggable modules.

[0037] Step 5: Modulate the residual thinning feature using a structure-sensitive attention map, and superimpose the modulated residual thinning feature onto the original HV branch intermediate feature to obtain the enhanced HV branch intermediate feature. Then, stitch the calibrated illumination feature with the enhanced HV branch intermediate feature, reconstruct the image through inverse HVI spatial transformation, and output the low-light enhanced image. The modulated residual refinement features are incorporated into the intermediate features of the original HV branch using a residual superposition method. The specific method is as follows: The structure-sensitive attention map modulates the residual refinement features. The modulated result is multiplied by a preset learnable scaling factor, and then added element-wise to the HV branch intermediate features in a residual manner to obtain the enhanced HV branch intermediate features. The formula used to calculate the structure-enhanced enhanced HV branch intermediate features is as follows: In the formula, This represents the intermediate features of the enhanced HV branch. This term indicates that the structure-sensitive attention map modulates the residual refinement features. This represents a learnable scaling factor that fills in and corrects missing or blurred structural details in the original HV branch intermediate features under low-light conditions, while preserving the inherent color tone and basic structural information of the original HV branch intermediate features without causing color shift or structural distortion. The result is an enhanced HV branch intermediate feature with clear texture, complete structure, and faithful color reproduction.

[0038] Based on the above, it should be noted that: Modulating the residual thinning features using a structure-sensitive attention map allows the residual thinning to primarily affect structure-sensitive regions, rather than indiscriminately altering the enhanced HV branch intermediate features across the entire image. Finally, the enhanced HV branch intermediate features are obtained through residual back-addition, which enhances structural details while maintaining the stability of the original HV features, reducing the risks of over-sharpening, chromatic aberration, and texture artifacts. It also fills in and corrects missing or blurred structural details in the original HV branch intermediate features under low-light conditions, while preserving the inherent color tone and basic structural information of the original HV branch intermediate features without causing color shifts or structural distortions. The final result is enhanced HV branch intermediate features with clear texture, complete structure, and faithful color reproduction.

[0039] Based on the above embodiments, the local illumination calibration operation is performed in the deep decoding stage of the I-branch decoder, and the structure-guided refinement operation is performed in the corresponding deep decoding stage of the HV-branch decoder. In an optional embodiment, all of the above operations are performed in the deep stage of the corresponding branch decoder. This configuration allows for the utilization of a larger receptive field and stronger semantic information for local illumination calibration and structure refinement, while avoiding excessive perturbation of shallow texture features, thus achieving a better balance between enhancement effect and feature stability.

[0040] The calibrated illumination features HV characteristics after structural enhancement Channel-dimensional stitching and fusion are performed in the HVI space to form a complete optimized HVI space feature map. Used to provide brightness, illumination, and low-frequency light information after partial exposure calibration. It is used to provide structure-guided refinement of chroma, texture, edge, and contour information. Together, they constitute the restored HVI features, enabling the enhancement result to have a more natural brightness distribution while maintaining good color stability and structural clarity.

[0041] The HVI space optimized feature map is subjected to inverse HVI space transformation, and the decoupled and optimized brightness, color and structural information is remapped back to the original RGB image space to complete the overall reconstruction of the image pixel level. The final output is a high-quality low-light enhancement image with uniform lighting, true colors, clear texture details, and no obvious distortion artifacts.

[0042] In the stage of processing the low-light image to be calibrated and outputting the low-light enhanced image, the present invention does not require input of a normal light image or additional manual annotation information. Only the RGB image of the low light to be calibrated needs to be input. The image will then undergo normalization preprocessing, HVI color space transformation, dual-branch enhancement, lightweight illumination calibration, structure-guided HV thinning, HVI feature fusion, and inverse HVI color space transformation in sequence, and finally output the low-light enhanced image.

[0043] Based on the above, it should be noted that: The final quality of a low-light enhanced image depends on the synergistic effect of illumination restoration and structure restoration. Restoring only brightness may result in a brighter image but with blurred edges and indistinct textures; conversely, emphasizing only structure may lead to enhanced details but overall underexposure or uneven brightness. This embodiment demonstrates... and The fusion of these elements allows for a unified expression of lighting enhancement and structural refinement within the HVI space. Then, an RGB-enhanced image is obtained through inverse HVI color space transformation, thereby achieving a balance between visual naturalness, local exposure consistency, and structural fidelity.

[0044] Based on the above embodiments, the dual-branch augmentation network employs a joint constraint of RGB space reconstruction loss and HVI space reconstruction loss during the training phase. The overall training loss function is as follows: in, This represents the overall training loss function. Indicates the RGB space reconstruction loss. Indicates HVI spatial reconstruction loss, The weighting coefficients represent the HVI space reconstruction loss.

[0045] Based on the above embodiments, the RGB space reconstruction loss and HVI space reconstruction loss are as follows: in, This represents the low-light enhanced image output by the network. This represents a normal light image. This represents the intermediate features of the enhanced HV branch recovered by the dual-branch augmentation network. This represents the enhanced I-branch intermediate features recovered by the dual-branch augmentation network. This represents the target HV color components obtained after transforming a normal light image into the HVI color space. This represents the target I luminance component obtained after transforming a normal light image into the HVI color space. This represents the L1 norm.

[0046] Based on the above, it should be noted that: The RGB space reconstruction loss is used to constrain the pixel-level differences between the network output image and the normal light image in the final visual space, making the enhancement result close to the normal light image in terms of overall color, brightness, and visual appearance. The HVI space reconstruction loss is used to constrain the restored HV color component and I luminance component respectively, making the HV branch output closer to the chromaticity structure component corresponding to the normal light image, and making the I branch output closer to the illumination intensity component corresponding to the normal light image. Through the joint constraint of RGB and HVI spaces, the visual consistency of the final image and the decoupled restoration effect of the branch features are ensured at the same time.

[0047] Based on the above embodiments, the training process is as follows: First, paired training samples of low-light and normal-light images are constructed, with the low-light image samples serving as network input and the normal-light image samples serving as supervised targets. The input low-light and normal-light image samples are then normalized, and the normal-light image samples undergo HVI color space transformation to obtain the target HV color components. and target I brightness component Then, the low-light image samples are transformed using the HVI color space to obtain the input HV color components and I luminance components, which are then fed into the HV branch and I branch respectively for feature recovery. In the I branch encoding and decoding stage, calibrated illumination features are obtained through illumination calibration. During the HV branch decoding stage, the structure-guided HV refinement module obtains the structurally enhanced HV features. Finally, the two are merged and subjected to inverse HVI color space transformation to obtain the network output enhanced image. .

[0048] Based on the above embodiments, in an optional training method, the Adam optimizer is used to update the network parameters, and the initial learning rate is set to... The training image cropping size was set to 256×256. During training, the overall training loss was used. As an optimization objective, the bi-branch augmentation network and its learnable parameters are updated using the backpropagation algorithm. Training is stopped and the model weights are saved when the preset number of training epochs is reached or the validation set metrics stabilize. The above training parameters are only one optional implementation method; those skilled in the art can adaptively adjust the learning rate, batch size, number of training epochs, pruning size, and loss weights according to the dataset size, hardware conditions, and actual deployment requirements.

[0049] Based on the above embodiments, it should be noted that: During the training phase, the normal light image serves only as a supervised target in the loss function calculation, constraining the network output of the enhanced image, the intermediate features of the enhanced HV branch, and the intermediate features of the enhanced I branch. During the inference phase, this invention does not require a normal light image or additional manual annotation; it only requires the input of the low-light RGB image to be enhanced. The image then sequentially undergoes HVI color space transformation, dual-branch enhancement, lightweight illumination calibration, structure-guided HV thinning, HVI feature fusion, and inverse HVI color space transformation to output the low-light enhanced image. Therefore, this invention is applicable to real-world low-light image enhancement scenarios.

[0050] Based on the above embodiments, it should be noted that the series of operations performed on the intermediate features of the I branch, including multi-scale average pooling, upsampling restoration, generating adaptive aggregation coefficients through projection operations, obtaining local illumination calibration basis through weighted aggregation, stitching to generate exposure-aware gating map, constructing residual preservation terms, and calculating calibrated illumination features based on the gating map and residual preservation terms, are considered as a lightweight illumination calibration module; the series of operations performed on the intermediate features of the HV branch, including channel-dimensional average pooling and channel-dimensional max pooling, stitching to obtain spatial description information, generating a structure-sensitive attention map through a projection layer, obtaining residual refinement features through point convolution, nonlinear activation, and depthwise separable convolution, modulating the residual refinement features based on the structure-sensitive attention map, and back-adding the residuals to obtain enhanced HV features, are considered as a structure-guided HV refinement module; Both the lightweight illumination calibration module and the structure-guided HV refinement module are lightweight pluggable modules. A lightweight pluggable module means that it can be embedded into an existing HVI dual-branch low-light image enhancement network without changing the basic HVI color space transformation method, the main structure of the dual-branch enhancement network, or introducing additional complex loss functions. Specifically, the lightweight illumination calibration module mainly consists of average pooling, upsampling, lightweight projection, sigmoid gating, and residual calculation, resulting in low computational overhead. The structure-guided HV refinement module mainly consists of channel pooling, lightweight spatial projection, point convolution, nonlinear activation, and depthwise convolution, with relatively few parameters. Therefore, this invention can improve the quality of low-light image enhancement while maintaining good model integration convenience and inference efficiency. Based on the above, it should be noted that: The core function of the lightweight illumination calibration module is to enhance the adaptability of the I-branch to spatially non-uniform illumination degradation. Through multi-scale low-frequency illumination response, adaptive weight aggregation, exposure-aware gating, and residual preservation, the I-branch can assign different calibration intensities to different regions, thereby alleviating problems such as insufficient enhancement in dark areas, overexposure in bright areas, and inconsistent local exposure. The core function of the structure-guided HV thinning module is to enhance the HV branch's ability to recover the structure of texture, edge, and contour regions. Through structure-sensitive attention map-guided residual thinning features, the HV branch can focus on strengthening areas with significant detail changes, thereby improving problems such as weakened structure, blurred boundaries, and softened textures in enhanced images.

[0051] Based on the above embodiments, the present invention can also be implemented through software, hardware, firmware, or any combination thereof. When implemented in software, the above methods can be encapsulated as an image enhancement program, a mobile image processing module, an edge vision enhancement module, or a server-side low-light image enhancement service. This program can be stored in a computer-readable storage medium and executed by a processor to perform steps such as low-light image input, HVI color space transformation, bi-branch feature recovery, illumination calibration, residual refinement, HVI feature fusion, and RGB image reconstruction. When implemented in hardware, the above network model is deployed in GPUs, NPUs, embedded AI chips, edge computing devices, or mobile terminals for nighttime image enhancement, low-light monitoring, mobile shooting enhancement, low-light visual perception, and downstream visual task preprocessing.

[0052] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0053] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0054] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0055] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A lightweight illumination and structure calibration low-light image enhancement method based on HVI space, characterized in that, The specific steps include: Step 1: Convert the low-light image to be calibrated to the HVI color space, separate the HV color component and the I luminance component, and input the two types of components into the trained dual-branch enhancement network. After encoding and decoding, extract the intermediate features of the I branch used for illumination correction and the intermediate features of the HV branch used for structural detail preservation. Step 2: Perform multi-scale average pooling and upsampling restoration on the intermediate features of branch I to generate multi-scale low-frequency illumination response. Generate adaptive aggregation coefficients for each scale through projection mapping. Weighted fuse the multi-scale low-frequency illumination response to obtain the illumination calibration basis. Step 3: The illumination calibration base and the intermediate feature of the I branch are spliced ​​together to generate an exposure-sensing gating map that can identify overexposed, underexposed, and normal areas. At the same time, the residual preservation terms of the two are constructed. The calibrated illumination features are calculated by combining the exposure-sensing gating map and the residual preservation terms. Step 4: Perform channel averaging and max pooling on the intermediate features of the HV branch, and concatenate them to obtain spatial structure information. Generate a structure-sensitive attention map through the projection layer. At the same time, perform pointwise convolution, nonlinear activation and depthwise separable convolution lightweight processing on the intermediate features of the HV branch to generate residual refined features. Step 5: Modulate the residual thinning features using a structure-sensitive attention map, then superimpose the modulated residual thinning features onto the original HV branch intermediate features to obtain the enhanced HV branch intermediate features. Combine the calibrated illumination features with the enhanced HV branch intermediate features, and reconstruct the image using inverse HVI spatial transformation to output a low-light enhanced image. For the intermediate features of the I branch that characterize the brightness distribution of the image, downsampling is performed using average pooling operations of different scales to extract global and local low-frequency information of illumination under different receptive fields; The pooled features at each scale are upsampled and restored one by one to the original feature size, thus obtaining a low-frequency illumination response that adapts to different illumination ranges and corresponds to multiple scales. Adaptive aggregation coefficients at various scales are generated through projection mapping, and the multi-scale low-frequency illumination responses are weighted and fused to obtain the illumination calibration basis. The specific logic is as follows: Multi-scale low-frequency illumination responses are input into a linear projection layer for feature mapping learning. Based on the differences in the contribution of illumination responses at different scales to the overall illumination calibration of the image, aggregate weight coefficients for the corresponding scales are adaptively generated. The corresponding low-frequency illumination response is weighted by adaptive aggregation weight coefficients at each scale, and the low-frequency illumination response after weighting at all scales is fused to generate an illumination calibration base that adapts to the brightness distribution of the entire image and has adaptive correction capabilities. The two refer to the illumination calibration base and the intermediate feature of the I branch. Based on the global standard brightness distribution represented by the illumination calibration base, the difference operation is performed with the original pixel brightness information represented by the intermediate feature of the I branch to construct the residual preservation term, which effectively captures the original brightness details of the image and the deviation of the real illumination, and preserves the inherent local brightness texture and light and shadow layers of the image. Pixel-level differential illumination adjustment is achieved by relying on the exposure-sensing gating map, which adaptively corrects the brightness deviation of overexposed and underexposed areas of the image. At the same time, the brightness and darkness details of the illumination adjustment result are constrained and compensated by the residual preservation term. Finally, the calibrated illumination features with uniform illumination and complete brightness and darkness details are obtained by fusion calculation.

2. The lightweight illumination and structure calibration low-light image enhancement method based on HVI space according to claim 1, characterized in that: The low-light image to be calibrated is converted to the HVI color space, and the HV color component and I luminance component are separated. The specific logic is as follows: The conventional RGB low-light image to be calibrated is mapped to the HVI color space, decoupling the image color information from the brightness information, and independently outputting the HV joint color component that represents the image hue and saturation, as well as the I brightness component that represents the image brightness. The HV branch intermediate features are structural color features obtained by encoding and decoding HV color components through a dual-branch enhancement network, which are used to characterize image texture, edge and color information. The intermediate features of the I branch are illumination features obtained by encoding and decoding the I brightness component through a dual-branch enhancement network, which are used to characterize the brightness distribution information of the image.

3. The lightweight illumination and structure calibration low-light image enhancement method based on HVI space according to claim 1, characterized in that: The average pooling operation using different scales requires at least two average pooling processes of different scales, and includes at least 5×5 average pooling and 9×9 average pooling.

4. The lightweight illumination and structure calibration low-light image enhancement method based on HVI space according to claim 1, characterized in that: By concatenating the illumination calibration base with the intermediate features of the I branch, an exposure-sensing gating map that can identify overexposed, underexposed, and normal areas is generated. The specific logic is as follows: The illumination calibration base and the intermediate features of the I branch, which characterize the brightness distribution information of the original image, are concatenated by channel dimension features to fuse global illumination prior information and local original brightness information. By comparing complementary dual feature information, the system automatically identifies overexposed, underexposed, and normally lit areas in an image, generating an exposure perception gating map with pixel-level exposure perception capabilities. The dual feature information consists of the illumination calibration base and the intermediate feature of the I branch.

5. The lightweight illumination and structure calibration low-light image enhancement method based on HVI space according to claim 1, characterized in that: The refined features of the modulated residual are superimposed onto the original HV branch intermediate features to obtain the enhanced HV branch intermediate features. The specific logic is as follows: The modulated residual refinement feature is a structural compensation feature after being constrained by the weights of the structure-sensitive attention map. The modulated residual refinement feature is incorporated into the original HV branch intermediate feature by residual superposition. This completes and corrects the missing and blurred structural details in the original HV branch intermediate feature under low light conditions, while preserving the inherent color tone and basic structural information of the original HV branch intermediate feature without color shift or structural distortion. The result is an enhanced HV branch intermediate feature with clear texture, complete structure, and faithful color.

6. The lightweight illumination and structure calibration low-light image enhancement method based on HVI space according to claim 1, characterized in that: The image is reconstructed by stitching together the illumination features after calibration and the intermediate features of the enhanced HV branch, and then undergoing inverse HVI spatial transformation to output a low-light enhanced image. The specific logic is as follows: The calibrated illumination features with optimized brightness are combined with the intermediate features of the HV branch with enhanced structural color by channel dimension splicing and fusion to form a complete HVI space-optimized feature map. The HVI space optimized feature map is subjected to inverse HVI space transformation, and the decoupled and optimized brightness, color and structural information is remapped back to the original RGB image space to complete the overall reconstruction of the image pixel level. The final output is a high-quality low-light enhancement image with uniform lighting, true colors, clear texture details, and no obvious distortion artifacts.

Images