Image enhancement method based on dual-domain light illumination prior and electronic device

Abstract

The application provides an image enhancement method based on a dual-domain light illumination prior and an electronic device, and relates to the field of image processing. The method comprises the following steps: acquiring a to-be-processed light illumination image; performing multi-scale enhancement on the to-be-processed light illumination image subjected to global light illumination enhancement according to the local light illumination distribution characteristics of the to-be-processed light illumination image to obtain a first light illumination image subjected to spatial domain enhancement, wherein the local light illumination distribution characteristics comprise a brightness mean value and a brightness standard deviation; enhancing the amplitude information of the to-be-processed light illumination image in the frequency domain to obtain a second light illumination image subjected to frequency domain enhancement; and fusing the to-be-processed light illumination image, the first light illumination image and the second light illumination image to obtain a target light illumination image subjected to enhancement.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more specifically, to an image enhancement method and electronic device based on dual-domain illumination prior. Background Technology

[0002] In real-world scenarios, low-light environments, nighttime shooting, or backlighting conditions result in insufficient light intensity entering the imaging device, leading to problems such as overall low image brightness, loss of detail in dark areas, increased noise, and color distortion. The signal-to-noise ratio of images drops sharply in low light, and the contrast imbalance between bright and dark areas significantly degrades image visual quality. While related image enhancement methods have made some progress, they still suffer from three main shortcomings: inaccurate illumination estimation, resulting in overly bright or dark enhancement areas; reliance on single-domain features in the spatial or frequency domains, neglecting their complementarity; and simplistic feature fusion mechanisms that cannot adaptively select effective information. Therefore, an efficient enhancement method capable of jointly modeling the illumination distribution in both the frequency and spatial domains is needed. Summary of the Invention

[0003] In view of this, the present invention provides an image enhancement method and electronic device based on dual-domain illumination prior.

[0004] One aspect of the present invention provides an image enhancement method based on dual-domain illumination prior, comprising: acquiring an illumination image to be processed; performing multi-scale enhancement on the illumination image to be processed after global illumination enhancement according to the local illumination distribution characteristics of the illumination image to be processed, to obtain a first illumination image after spatial enhancement, wherein the local illumination distribution characteristics include the mean brightness and the standard deviation of brightness; enhancing the amplitude information of the illumination image to be processed in the frequency domain, to obtain a second illumination image after frequency enhancement; and fusing the illumination image to be processed, the first illumination image, and the second illumination image to obtain an enhanced target illumination image.

[0005] According to an embodiment of the present invention, based on the local illumination distribution characteristics of the illumination image to be processed, multi-scale enhancement is performed on the illumination image to be processed after global illumination enhancement to obtain a first illumination image after spatial enhancement, comprising: enhancing the illumination image to be processed after global illumination enhancement based on the local illumination distribution characteristics of the illumination image to be processed to obtain a first intermediate illumination image; enhancing the first intermediate illumination image after global illumination enhancement based on the local illumination distribution characteristics of the first intermediate illumination image to obtain a second intermediate illumination image; wherein, the scale of the local illumination distribution characteristics of the first illumination image is smaller than the scale of the local illumination distribution characteristics of the illumination image to be processed; and enhancing the second intermediate illumination image after global illumination enhancement based on the overall illumination distribution characteristics of the second intermediate illumination image to obtain the first illumination image.

[0006] According to an embodiment of the present invention, enhancing a globally enhanced illumination image to obtain a first intermediate illumination image is performed based on the local illumination distribution characteristics of the illumination image to be processed. This includes: dividing the globally enhanced illumination image to be processed into regions to obtain multiple first sub-regions of a first size; adjusting the illumination distribution of each first sub-region based on the local illumination distribution characteristics of the corresponding sub-region of the illumination image to be processed to obtain an adjusted first sub-region; and stitching the multiple adjusted first sub-regions together to form a first intermediate illumination image according to the spatial position of the first sub-region in the illumination image to be processed.

[0007] According to an embodiment of the present invention, a second intermediate illumination image is obtained by enhancing the first intermediate illumination image after global illumination enhancement based on the local illumination distribution characteristics of the first intermediate illumination image. This includes: dividing the first intermediate illumination image after global illumination enhancement into regions to obtain multiple second sub-regions of a second size, wherein the second size is smaller than the first size; adjusting the illumination distribution of each second sub-region according to the local illumination distribution characteristics of the sub-region corresponding to the first intermediate illumination image to obtain adjusted second sub-regions; and stitching the multiple adjusted second sub-regions together to form the second intermediate illumination image according to their spatial positions in the illumination image to be processed.

[0008] According to an embodiment of the present invention, the illumination distribution of the first sub-region is adjusted based on the local illumination distribution characteristics of each first sub-region and the corresponding sub-region of the illumination image to be processed, to obtain the adjusted first sub-region. This includes: acquiring the local illumination distribution characteristics of each first sub-region corresponding to the sub-region extracted from the illumination image to be processed; correcting the mean brightness and standard deviation of brightness using adaptively learned parameter adjustment coefficients to obtain the corrected mean brightness and standard deviation of brightness; and adjusting the illumination distribution of the first sub-region based on the corrected mean brightness, the corrected standard deviation of brightness, and the mean brightness and standard deviation of brightness, to obtain the adjusted first sub-region.

[0009] According to an embodiment of the present invention, fusing a lighting image to be processed, a first lighting image, and a second lighting image to obtain an enhanced target lighting image includes: summing the lighting image to be processed, the first lighting image, and the second lighting image pixel by pixel to obtain an aggregated feature map; generating a first weight, a second weight, and a third weight based on the aggregated feature map using an attention mechanism; fusing the superposition result of the first weight and the first lighting image, the superposition result of the second weight and the lighting image to be processed, and the superposition result of the third weight and the second lighting image to obtain a fused feature map; and concatenating the feature-aligned fused feature map with the residual of the lighting image to be processed to obtain the enhanced target lighting image.

[0010] According to an embodiment of the present invention, based on an attention mechanism, generating a first weight, a second weight, and a third weight from an aggregated feature map includes: performing global pooling and average pooling on the aggregated feature map respectively, and concatenating the results of global pooling and average pooling to obtain a concatenated feature; inputting the concatenated feature into three feature extraction branches respectively, and outputting a first feature value, a second feature value, and a third feature value that enhance the features of the first illumination image, the illumination image to be processed, and the second illumination image respectively; and processing the first feature value, the second feature value, and the third feature value using a normalization function to obtain the first weight, the second weight, and the third weight.

[0011] According to an embodiment of the present invention, enhancing the amplitude information of the illumination image to be processed in the frequency domain to obtain a second illumination image after frequency domain enhancement includes: performing a Fourier transform on the illumination image to be processed for each color channel to obtain frequency domain information; performing feature enhancement on the amplitude information separated from the frequency domain information to obtain enhanced amplitude information; performing an inverse Fourier transform on the enhanced amplitude information and the phase information in the frequency domain information to obtain a third illumination image for each color channel after frequency domain enhancement; and concatenating the third illumination images of each color channel to obtain a second illumination image.

[0012] According to an embodiment of the present invention, the illumination enhancement model is configured to: generate a target illumination image based on the illumination image to be processed; the illumination enhancement model is trained in the following manner: inputting a sample illumination image into the illumination enhancement model to be trained to generate a target sample illumination image; using a loss function, adjusting the illumination enhancement model to be trained based on the target sample illumination image and a label illumination image of the real illumination corresponding to the sample illumination image until the training conditions are met, thereby obtaining a trained illumination enhancement model; the loss function includes: a pixel reconstruction loss characterizing the pixel difference between the target sample illumination image and the label illumination image, and an amplitude consistency loss characterizing the amplitude difference between the target sample illumination image and the label illumination image.

[0013] Another aspect of the present invention provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method described above.

[0014] Another aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the above-described method.

[0015] Another aspect of the present invention provides a computer program product comprising computer-executable instructions which, when executed, are used to implement the method described above.

[0016] This invention provides an image enhancement method based on dual-domain illumination priors, aiming to achieve adaptive restoration of illumination distribution in low-light images while preserving image structural details and color information. This invention establishes illumination distribution priors in both the frequency and spatial domains, achieving illumination consistency enhancement through the interaction and feedback mechanism of dual-domain features. Furthermore, it dynamically weights and fuses the frequency domain enhancement result (second illumination image), the spatial domain correction result (first illumination image), and the original image to be processed based on a self-attention mechanism, thereby improving overall brightness and color balance while maintaining detailed structural features. Attached Figure Description

[0017] The above and other objects, features and advantages of the present invention will become more apparent from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0018] Figure 1 A flowchart of an image enhancement method based on dual-domain illumination priors according to an embodiment of the present invention is shown.

[0019] Figure 2 A schematic diagram illustrating the spatial domain image enhancement principle of the image enhancement method based on dual-domain illumination prior according to an embodiment of the present invention is shown.

[0020] Figure 3 A schematic diagram illustrating the frequency domain image enhancement principle of the image enhancement method based on dual-domain illumination prior according to an embodiment of the present invention is shown.

[0021] Figure 4 A schematic diagram illustrating the principle of acquiring a second illumination image using an image enhancement method based on dual-domain illumination prior according to an embodiment of the present invention.

[0022] Figure 5 A schematic diagram illustrating the selective core feature fusion principle of the image enhancement method based on dual-domain illumination prior according to an embodiment of the present invention is shown.

[0023] Figure 6 The image restoration result is shown by combining the present model with multiple models according to an embodiment of the present invention.

[0024] Figure 7 A block diagram of an electronic device suitable for implementing an image enhancement method based on dual-domain illumination priors according to an embodiment of the present invention is shown. Detailed Implementation

[0025] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0026] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0028] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0029] In low-light conditions, the signal-to-noise ratio drops sharply, and the contrast imbalance between bright and dark areas significantly degrades image visual quality. Furthermore, inconsistent illumination distribution across different channels easily leads to color casts; simultaneously, camera sensors amplify noise in high-gain mode, making it difficult for traditional enhancement algorithms to balance brightness improvement and detail preservation. While image enhancement methods based on Retinex theory or deep learning have made some progress, they mostly model illumination within a single domain (spatial or frequency domain), failing to consider both global illumination patterns and local structural details. The lack of bidirectional feature interaction and prior constraints between the frequency and spatial domains prevents models from achieving a balance between brightness restoration and texture preservation. Additionally, feature fusion methods are often static convolutions or simple concatenations, unable to adaptively adjust branch weights based on image illumination distribution.

[0030] Therefore, the present invention provides an image enhancement method and electronic device based on dual-domain illumination prior, which at least partially solves the above-mentioned technical problems.

[0031] According to an embodiment of the present invention, an image enhancement method based on dual-domain illumination prior includes: acquiring an illumination image to be processed; performing multi-scale enhancement on the illumination image to be processed after global illumination enhancement based on the local illumination distribution characteristics of the illumination image to be processed, to obtain a first illumination image after spatial enhancement, wherein the local illumination distribution characteristics include the mean brightness and the standard deviation of brightness; enhancing the amplitude information of the illumination image to be processed in the frequency domain, to obtain a second illumination image after frequency enhancement; and fusing the illumination image to be processed, the first illumination image, and the second illumination image to obtain an enhanced target illumination image.

[0032] In real-world shooting scenarios, low-light, nighttime, or backlit environments result in insufficient light entering the imaging device, causing problems such as low image brightness, loss of detail in dark areas, color distortion, and reduced signal-to-noise ratio, leading to a deterioration in the visual quality of the image. The image to be processed can be a low-light image acquired in the aforementioned scenarios.

[0033] Figure 1 A flowchart of an image enhancement method based on dual-domain illumination prior, according to an embodiment of the present invention, is shown. Figure 1 As shown, the illumination image to be processed is enhanced by both spatial illumination distribution prior and frequency illumination distribution prior. The enhanced first illumination image, the second illumination image and the illumination image to be processed are then selectively fused with core features to obtain the enhanced target illumination image.

[0034] The spatial domain directly reflects the pixel spatial distribution of an image. Brightness restoration based on correlation features in the spatial domain allows for supplementary lighting while preserving local structure, resulting in clearer image content. Therefore, this invention constructs a spatial domain illumination modeling method based on local statistical features. First, it performs global illumination enhancement on the image to be processed. This is achieved through multi-stage convolution-activation cascade techniques, adaptively optimizing the global pixel illumination distribution. While balancing dark area brightening, bright area detail preservation, and noise and color distortion suppression, it achieves global brightness balance and hidden information restoration in low-light images, resulting in a natural and complete visual effect. To improve the stability and consistency of brightness enhancement, for each local area of ​​the globally enhanced image to be processed, its brightness is enhanced according to the corresponding local illumination distribution characteristics, thus obtaining the first illumination image. The local illumination distribution characteristics include the mean brightness and the standard deviation of brightness.

[0035] The frequency domain can intuitively present the essential characteristics of an image, such as brightness distribution, detail richness, and noise interference. Targeted optimization of overall brightness in the frequency domain can balance brightness without interfering with texture, edge, and other details, achieving a natural increase in global image brightness while preserving core information. Since image brightness is mainly reflected in amplitude information, while structure and texture are mainly maintained by phase information, this invention only enhances amplitude information in the frequency domain, thereby improving brightness while maintaining image structural stability. Specifically, firstly, the features of the red, green, and blue color channels of the image to be processed are transformed from the spatial domain to the frequency domain, and the amplitude and phase information of each color channel are obtained. These three color channels can be simply referred to as R, G, and B channels. Convolution and activation function processing are applied to the amplitude information to achieve lightweight amplitude enhancement. Subsequently, based on the enhanced amplitude and the original phase, the image is restored from the frequency domain to the spatial domain. The results of the three color channels are then concatenated to obtain the second illumination image enhanced in the frequency domain.

[0036] Finally, a selective feature fusion strategy based on a self-attention mechanism is adopted to adaptively fuse the illumination image to be processed, the first illumination image, and the second illumination image, so as to realize the information fusion of the illumination image to be processed after the brightness enhancement in the spatial and frequency domains, and finally obtain the enhanced target illumination image.

[0037] This invention provides an image enhancement method based on dual-domain illumination priors, aiming to achieve adaptive restoration of illumination distribution in low-light images while preserving structural details and color information. This invention establishes illumination distribution priors in both the frequency and spatial domains. Through a bidirectional prior collaboration mechanism between the frequency and spatial domains, it achieves a unification of global illumination compensation and local detail optimization, significantly improving the brightness and visual quality of low-light images while maintaining structural fidelity and natural color. Simultaneously, based on a self-attention mechanism, it dynamically weights and fuses the frequency domain enhancement results, spatial domain correction results, and original features, improving overall brightness and color balance while preserving structural details.

[0038] According to an embodiment of the present invention, based on the local illumination distribution characteristics of the illumination image to be processed, multi-scale enhancement is performed on the illumination image to be processed after global illumination enhancement to obtain a first illumination image after spatial enhancement, comprising: enhancing the illumination image to be processed after global illumination enhancement based on the local illumination distribution characteristics of the illumination image to be processed to obtain a first intermediate illumination image; enhancing the first intermediate illumination image after global illumination enhancement based on the local illumination distribution characteristics of the first intermediate illumination image to obtain a second intermediate illumination image; wherein, the scale of the local illumination distribution characteristics of the first illumination image is smaller than the scale of the local illumination distribution characteristics of the illumination image to be processed; and enhancing the second intermediate illumination image after global illumination enhancement based on the overall illumination distribution characteristics of the second intermediate illumination image to obtain the first illumination image.

[0039] According to an embodiment of the present invention, the first illumination image is obtained by performing three-level image brightness enhancement on the illumination image to be processed, and the first intermediate illumination image and the second intermediate illumination image are intermediate values ​​in the three-level image enhancement process.

[0040] It is understandable that the image to be processed, or the image to be processed after global illumination enhancement, can be divided into multiple sub-regions, each of which can be called a "local area". For each local area of ​​the image to be processed after global illumination enhancement, the enhanced mean and standard deviation of brightness are obtained based on the mean and standard deviation of brightness of each local area of ​​the image to be processed (i.e., local illumination distribution characteristics). Based on this, the brightness of the image to be processed after global illumination enhancement is enhanced to obtain the first intermediate illumination image.

[0041] Subsequently, global brightness enhancement is performed on the first intermediate illumination image using convolution. For example, a two-stage "convolution and activation function" approach is employed for global brightness enhancement. In the first stage, convolution captures local illumination distribution features of the image through the local receptive field, while the activation function filters effective illumination information and suppresses noise, initially distinguishing the areas requiring enhancement from normal areas. In the second stage, convolution further extracts global illumination correlations, and the activation function further enhances the nonlinear mapping capability. Through these cascaded operations, global illumination enhancement for the illumination image to be processed is finally achieved.

[0042] For each local area of ​​the first intermediate illumination image after global brightness enhancement, brightness enhancement is performed based on the local illumination distribution characteristics of the corresponding positions in the first intermediate illumination image to obtain the second intermediate illumination image. Similarly, the global brightness enhancement method and the method of brightness enhancement using local illumination distribution characteristics for the second intermediate illumination image are similar to those for the first illumination image, and will not be described again here. For the second intermediate illumination image after global brightness enhancement, brightness enhancement can be directly performed on the second intermediate illumination image based on the overall illumination distribution characteristics of the second intermediate illumination image to obtain the first illumination image. The subsequent two levels of global brightness enhancement methods and the methods of brightness enhancement using local / global illumination distribution characteristics are similar to those for the first illumination image, and will not be described again here.

[0043] The scale of the local illumination distribution features in the first illumination image is smaller than the scale of the local illumination distribution features in the illumination image to be processed. Here, the scale of the local illumination distribution features can refer to the size of the sub-region targeted by the local illumination distribution features; for example, the size of the sub-region targeted by the local illumination distribution features used in the first enhancement is smaller than the size of the sub-region targeted by the second enhancement.

[0044] In embodiments of the present invention, a multi-scale enhancement approach is employed to enhance the image under processing. This involves first analyzing local illumination distribution features at a medium scale, then at a small scale, and finally at the original scale to ensure the accuracy of low-light recovery in each region. Finally, the original scale enhancement ensures the reasonableness and accuracy of the restored real-world illumination distribution. In summary, by enhancing the image under processing with illumination distribution features at different scales and the overall illumination distribution, the embodiments of the present invention achieve multi-scale progressive enhancement, balancing illumination realism and accuracy.

[0045] Figure 2 A schematic diagram illustrating the spatial domain image enhancement principle of the image enhancement method based on dual-domain illumination prior according to an embodiment of the present invention is shown. Based on the local illumination distribution characteristics of the illumination image to be processed, the illumination image to be processed after global illumination enhancement is enhanced to obtain a first intermediate illumination image. This includes: dividing the illumination image to be processed after global illumination enhancement into regions to obtain multiple first sub-regions of a first size; adjusting the illumination distribution of each first sub-region according to the local illumination distribution characteristics of the corresponding sub-region of the illumination image to be processed to obtain adjusted first sub-regions; and stitching the multiple adjusted first sub-regions together to form the first intermediate illumination image according to their spatial positions in the illumination image to be processed.

[0046] like Figure 2As shown, firstly, the global illumination-enhanced image 201 of size height × width (H × W) is divided into regions, resulting in 16 first sub-regions of a first size. The specific division method is shown in Figure 202. Based on the mean brightness and standard deviation of the brightness of each first sub-region of the illumination image to be processed, the illumination distribution of the first sub-region corresponding to the first intermediate illumination image after global illumination enhancement is adjusted to obtain multiple adjusted first sub-regions. The adjusted first sub-regions are then stitched together according to their positions in the image to be processed to reconstruct the first intermediate illumination image of size H×W. In the above stitching process, the differences between the mean brightness and standard deviation of the image to be processed before global enhancement and the actual illumination distribution can be used to selectively adjust the illumination distribution of the image to be processed after global enhancement. The differences between the mean brightness and standard deviation of the image to be processed before global enhancement and the actual illumination distribution can be obtained through parameter adjustment coefficients learned during training.

[0047] Based on the local illumination distribution characteristics of the first intermediate illumination image, the first intermediate illumination image after global illumination enhancement is enhanced to obtain a second intermediate illumination image. This includes: dividing the first intermediate illumination image after global illumination enhancement into regions to obtain multiple second sub-regions of a second size, wherein the second size is smaller than the first size; adjusting the illumination distribution of each second sub-region according to the local illumination distribution characteristics of the corresponding sub-region of the first intermediate illumination image to obtain an adjusted second sub-region; and stitching the multiple adjusted second sub-regions into a second intermediate illumination image according to the spatial position of the second sub-region in the illumination image to be processed.

[0048] First, the obtained first intermediate lighting image is subjected to global brightness enhancement. For example... Figure 2 As shown, the first intermediate illumination image with height × width (H × W) after global illumination enhancement is divided into regions, resulting in 256 second sub-regions with a second size. The specific division method is shown in 203. Based on the first intermediate illumination image, the illumination distribution of the corresponding sub-regions of the first intermediate illumination image after global illumination enhancement is adjusted to obtain multiple adjusted second sub-regions. After stitching them together, a second intermediate illumination image with a size of H×W is obtained.

[0049] To facilitate understanding of the specific operations of adjusting the illumination distribution, we will take the adjustment of the illumination distribution in the first sub-region as an example.

[0050] Based on the local illumination distribution characteristics of each first sub-region and the corresponding sub-region of the illumination image to be processed, the illumination distribution of the first sub-region is adjusted to obtain the adjusted first sub-region. This includes: acquiring the local illumination distribution characteristics of each first sub-region corresponding to the sub-region extracted from the illumination image to be processed; correcting the mean luminance and standard deviation luminance using adaptively learned parameter adjustment coefficients to obtain the corrected mean luminance and standard deviation luminance; and adjusting the illumination distribution of the first sub-region based on the corrected mean luminance, the corrected standard deviation luminance, and the mean luminance and standard deviation luminance to obtain the adjusted first sub-region.

[0051] The illumination image to be processed is also divided into sizes of [size]. After dividing the image into 16 sub-regions, the average brightness of the corresponding sub-region in the illumination image to be processed is obtained for each first sub-region. With the standard deviation of brightness and This is used to accurately characterize the current illumination level and contrast distribution characteristics. Based on adaptive learning of the differences between the real illumination distribution and the low illumination distribution during training, parameter adjustment coefficients can be determined, such as parameter factors and bias terms. After learning the parameter adjustment coefficients, the mean brightness and standard deviation of the current illumination image to be processed can be corrected using formulas (1) and (2) formed by the parameter adjustment coefficients, respectively:

[0052] (1)

[0053] (2)

[0054] In the formula, , These are the parameter factors for the mean and standard deviation of brightness, adaptively learned during training. , This refers to the bias terms for the mean and standard deviation of brightness learned adaptively during training. The average brightness value after correction. This represents the corrected standard deviation of brightness.

[0055] according to , as well as , For the first sub-region The illumination distribution is adjusted according to formula (3) to obtain the adjusted first sub-region. :

[0056] (3)

[0057] It should be noted that the adjustment of the illumination distribution in the second sub-region and the adjustment of the illumination distribution in the entire second intermediate illumination image can be referenced from the first sub-region. The three are for different scales but the operations are the same or similar.

[0058] In embodiments of the present invention, the overall distribution of local illumination can be measured in the dimension of each sub-region using the local illumination distribution characteristics (mean brightness and standard deviation of brightness); similarly, the overall distribution of local illumination can be measured in the dimension of the final whole image using the overall illumination distribution characteristics (e.g., mean brightness and standard deviation of brightness) of the entire second intermediate illumination image. Throughout the illumination enhancement process, by correcting the mean brightness and standard deviation of brightness, which measure illumination distribution, and using the corrected mean brightness and standard deviation of brightness to revert the image, the realism and accuracy of illumination distribution adjustment can be improved.

[0059] Figure 3 A schematic diagram illustrating the frequency domain image enhancement principle of the image enhancement method based on dual-domain illumination prior according to an embodiment of the present invention is shown. The method involves enhancing the amplitude information of the illumination image to be processed in the frequency domain to obtain a second illumination image after frequency domain enhancement. This includes: performing a Fourier transform on the illumination image to be processed for each color channel to obtain frequency domain information; performing feature enhancement on the amplitude information separated from the frequency domain information to obtain enhanced amplitude information; performing an inverse Fourier transform on the enhanced amplitude information and the phase information in the frequency domain information to obtain a third illumination image for each color channel after frequency domain enhancement; and stitching the third illumination images of each color channel together to obtain the second illumination image.

[0060] like Figure 3 As shown, Fourier transforms can be performed on the features of each single color channel in the R, G, and B regions of the illumination image to be processed to obtain the frequency domain information of each color channel. The amplitude information of each color channel is then determined based on the parameters of its frequency domain information, thus achieving amplitude acquisition. For the amplitude information of each color channel, amplitude enhancement can be performed through convolution, and the enhanced results are fused. Finally, an inverse Fourier transform is used to restore the spatial domain, yielding the second illumination image. .

[0061] Figure 4 A schematic diagram illustrating the principle of acquiring a second illumination image using an image enhancement method based on dual-domain illumination prior according to an embodiment of the present invention.

[0062] like Figure 4 As shown, the specific process of obtaining the second illumination image includes: firstly, performing a fast Fourier transform on the illumination image to be processed using formula (4) to obtain its frequency domain information. :

[0063] (4)

[0064] In the formula, h and w represent spatial coordinates, u and v represent frequency domain coordinates, and j is the imaginary unit. This represents the image to be processed in a certain color channel. The coordinates in pixels.

[0065] The real part of the image to be processed is obtained after transformation. With the imaginary part Therefore, the amplitude information can be calculated using formulas (5) and (6) respectively. and phase information :

[0066] (5)

[0067] (6)

[0068] In the formula, u and v represent frequency domain coordinates. This is represented by the arctangent function.

[0069] In the experiment, the brightness of the image is mainly reflected in the amplitude information, while the structure and texture are mainly maintained by the phase information. Therefore, this invention only enhances the amplitude information, thereby improving brightness while maintaining image structure stability. Preferably, to avoid color shift caused by differences in color channels, this invention performs independent frequency domain enhancement processing on the R, G, and B color channels of the image to be processed. For each color channel, amplitude and phase information are obtained through Fast Fourier Transform, and then successively processed through 1×1 convolution and activation function. Lightweight amplitude enhancement is performed using 1×1 convolution to obtain the enhanced amplitude information. The calculation process is shown in formula (7):

[0070] (7)

[0071] In the formula, This represents a 1×1 convolution. It is the amplitude information of any color channel before enhancement.

[0072] Subsequently, the enhanced amplitude information is merged with the original unenhanced phase information, and an inverse Fourier transform is performed to restore it to the spatial domain, resulting in the frequency-domain enhanced third illumination image as shown in formula (8). :

[0073]

[0074] In the formula, j is the imaginary unit. This represents the inverse Fourier transform, sin is the function representation, and P represents the phase information.

[0075] After performing the above operations on the R, G, and B color channels respectively, we obtain the third illumination image for each color channel after frequency domain enhancement. The third illumination images of the three color channels are stitched together to obtain the second illumination image as shown in formula (9). :

[0076]

[0077] Cat() represents stitching together the third illumination image of the three color channels.

[0078] According to an embodiment of the present invention, fusing a lighting image to be processed, a first lighting image, and a second lighting image to obtain an enhanced target lighting image includes: summing the lighting image to be processed, the first lighting image, and the second lighting image pixel by pixel to obtain an aggregated feature map; generating a first weight, a second weight, and a third weight based on the aggregated feature map using an attention mechanism; fusing the superposition result of the first weight and the first lighting image, the superposition result of the second weight and the lighting image to be processed, and the superposition result of the third weight and the second lighting image to obtain a fused feature map; and concatenating the feature-aligned fused feature map with the residual of the lighting image to be processed to obtain the enhanced target lighting image.

[0079] In the fusion phase, firstly, the first illumination image is... Original image to be processed Second lighting image By adding elements one by one, we obtain the aggregated feature map L as shown in formula (10):

[0080]

[0081] Based on the attention mechanism, a first weight s1, a second weight s2, and a third weight s3 are generated according to the aggregated feature map L to dynamically adjust the contribution of each input branch. Then, the superposition results of the first weight s1 and the first illumination image, the superposition results of the second weight s2 and the image to be processed, and the superposition results of the third weight s3 and the second illumination image are fused to obtain a fused feature map U, thereby strengthening effective features, suppressing redundant information, and improving the image quality of the fused feature map. The expression for obtaining the fused feature map U is shown in formula (11):

[0082]

[0083] Furthermore, residual connections are introduced after the fused feature map to directly fuse the features of the illumination image to be processed with the enhanced fused feature map. Specifically, firstly, matching, mapping, and calibration operations are used to ensure that the illumination image to be processed and the fused feature map maintain consistency under a unified distribution, achieving feature alignment. The illumination image to be processed is then collaboratively fused with the fused feature map generated after multi-stage processing such as convolution and activation functions. This allows the core information such as the basic image structure and inherent brightness distribution contained in the illumination image to complement the illumination optimization features in the fused feature map, ultimately outputting features that combine the integrity of basic information with illumination enhancement effects. Through the above operations, the system can dynamically allocate weights according to the illumination conditions of the input image, focusing on the frequency domain enhancement branch in areas with insufficient illumination and strengthening the spatial domain enhancement branch and the original feature branch in areas with rich texture, thereby achieving optimal information fusion and complementarity. This preserves the basic structure and effective information of the illumination image to be processed, avoiding feature loss or gradient vanishing problems that occur in multi-stage processing, while also superimposing the illumination optimization features learned during the enhancement process, achieving the effect of no loss of basic information and accurate superposition of enhanced features. At the same time, the enhancement amplitude can be dynamically adjusted to suppress side effects such as overexposure and noise amplification, so that the final global illumination enhancement target illumination image is more in line with the true illumination distribution properties of the image, taking into account both brightness balance and detail integrity, and improving the model's adaptability and output stability in complex low-light scenes.

[0084] According to an embodiment of the present invention, based on an attention mechanism, generating a first weight, a second weight, and a third weight from an aggregated feature map includes: performing global pooling and average pooling on the aggregated feature map respectively, and concatenating the results of global pooling and average pooling to obtain a concatenated feature; inputting the concatenated feature into three feature extraction branches respectively, and outputting a first feature value, a second feature value, and a third feature value that enhance the features of the first illumination image, the illumination image to be processed, and the second illumination image respectively; and processing the first feature value, the second feature value, and the third feature value using a normalization function to obtain the first weight, the second weight, and the third weight.

[0085] First, global average pooling (GAP) and global max pooling (GMP) operations are performed on the aggregated feature map L, and the results of GAP and GMP are concatenated to obtain the concatenated feature s. Global average pooling uses the arithmetic mean of pixel values ​​at all spatial locations in L as the global representation, which smoothly integrates overall information, weakens local noise interference, and highlights the global distribution trend of features. Global max pooling selects the maximum pixel value in the spatial dimension of L as the global representation of that color channel, focusing on capturing the most significant and discriminative local features within the color channel, enhancing the sparsity of features and the recognizability of key information. The concatenated feature s is then input into the feature extraction branches of the first illumination image, the illumination image to be processed, and the second illumination image, respectively. After 1×1 convolution and activation by the PReLU activation function, the first feature value v1, the second feature value v2, and the third feature value v3 are obtained. Subsequently, the first weight s1, the second weight s2, and the third weight s3 are generated by normalization using the normalization function softmax, so as to adaptively adjust the importance of each input branch (L1, L2, L3) using the above three weights respectively.

[0086] Figure 5 A schematic diagram illustrating the selective core feature fusion principle of the image enhancement method based on dual-domain illumination prior according to an embodiment of the present invention is shown.

[0087] like Figure 5 As shown, firstly, L1, L2, and L3 are summed element-wise to obtain the aggregated feature map L. Global average pooling and global max pooling operations are then performed on L to generate concatenated features s. These concatenated features s are input into the feature extraction branches of the first illumination image, the illumination image to be processed, and the second illumination image, respectively. After each branch undergoes a 1×1 convolution and PReLU activation, the first feature value v1, the second feature value v2, and the third feature value v3 are obtained. Subsequently, the first weight s1, the second weight s2, and the third weight s3 are generated using the softmax normalization function to adaptively adjust the importance of each input branch. L1, L2, and L3 are concatenated with the outputs of their respective branches, and then the concatenated results are summed element-wise to obtain the fused feature map U after feature alignment.

[0088] According to an embodiment of the present invention, the illumination enhancement model is configured to: generate a target illumination image based on the illumination image to be processed; the illumination enhancement model is trained in the following manner: inputting a sample illumination image into the illumination enhancement model to be trained to generate a target sample illumination image; using a loss function, adjusting the illumination enhancement model to be trained based on the target sample illumination image and a label illumination image of the real illumination corresponding to the sample illumination image until the training conditions are met, thereby obtaining a trained illumination enhancement model; the loss function includes: a pixel reconstruction loss characterizing the pixel difference between the target sample illumination image and the label illumination image, and an amplitude consistency loss characterizing the amplitude difference between the target sample illumination image and the label illumination image.

[0089] For example, the illumination enhancement model includes a frequency domain enhancement branch, a spatial domain enhancement branch, and a Selective Core Feature Fusion Module (SCFF). The spatial domain enhancement branch is used to perform multi-scale enhancement on the illumination image to be processed, which has undergone global illumination enhancement, based on the local illumination distribution characteristics of the illumination image to be processed, to obtain a first illumination image after spatial enhancement. The frequency domain enhancement branch is used to enhance the amplitude information of the illumination image to be processed in the frequency domain, to obtain a second illumination image after frequency enhancement. The Selective Core Feature Fusion Module is used to fuse the illumination image to be processed, the first illumination image, and the second illumination image to obtain an enhanced target illumination image.

[0090] During the model training phase, this invention employs a loss function jointly constrained by the pixel domain and the frequency domain to ensure that the enhanced target sample illumination image closely approximates the real illumination label image in both brightness and structure. Specifically, the pixel reconstruction loss... The expression is shown in formula (12):

[0091] (12)

[0092] Where GT represents the label illumination image under real lighting conditions, This represents the illumination image of the target sample after the illumination has been recovered using the model. This represents the L1 paradigm. To constrain the amplitude uniformity of the frequency domain illumination distribution, an amplitude uniformity loss as shown in equation (13) is introduced. :

[0093] (13)

[0094] in, This indicates the amplitude information of the label's illumination image. This indicates the amplitude information of the sample illumination image.

[0095] Combining the two losses mentioned above, we obtain the loss function shown in formula (14). :

[0096] (14)

[0097] Where λ is the weighting coefficient, preferably 0.01. Through joint optimization, the model can align with the true brightness at the pixel level while accurately maintaining the illumination distribution characteristics at the frequency domain level, thereby achieving an enhancement effect of consistent global brightness and realistic local details.

[0098] Furthermore, the overall network framework of this invention can be combined with various existing structures (such as U-Net, Restormer, Retinexformer, etc.) to form an end-to-end low-light enhancement system. Frequency and spatial prior modules can be flexibly embedded into different layers of the backbone network, while the fusion module serves as a unified output. Through this architecture, this invention achieves dual optimization in illumination distribution modeling and structural information preservation, significantly outperforming related methods in terms of visual quality, color reproduction, and noise suppression.

[0099] Table 1 shows the quantitative comparison results of embedding the DIP model of the present invention into multiple models on the LOL-v1, LOL-v2-Real, and LOL-v2-Syn datasets:

[0100] Table 1

[0101]

[0102] Quantitative comparisons on the LSRW-Huawei and LSRW-Nikon image datasets are shown in Table 2:

[0103] Table 2

[0104]

[0105] Figure 6 The image restoration result is shown by combining this model with multiple models according to an embodiment of the present invention. Figure 6 As shown, (a) is the illumination image to be processed acquired under low-light conditions; (b), (c), (e), (g), and (i) are the image processing results obtained by brightness restoration using FECNet, MIRNet-v2, Retinexformer, EvLight, and GT models respectively; (d), (f), and (h) are the target illumination images obtained by combining the model (DIP) of this invention with MIRNet-v2, Retinexformer, and EvLight respectively. Figure 6It is understood that by using the DIP model of the present invention, the overall brightness and color balance of the image to be processed can be improved while maintaining the detailed structure.

[0106] Figure 7 A block diagram of an electronic device suitable for implementing an image enhancement method based on dual-domain illumination priors according to an embodiment of the present invention is shown. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present invention.

[0107] like Figure 7 As shown, an electronic device 700 according to an embodiment of the present invention includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0108] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 702 and / or RAM 703. It should be noted that the programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in said one or more memories.

[0109] According to an embodiment of the present invention, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the input / output (I / O) interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0110] According to embodiments of the present invention, the method flow according to embodiments of the present invention can be implemented as a computer software program. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by processor 701, it performs the functions defined in the system of the embodiments of the present invention. According to embodiments of the present invention, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0111] The embodiments of the present invention have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of the invention. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the invention, and all such substitutions and modifications should fall within the scope of the invention.

Claims

1. An image enhancement method based on dual-domain illumination prior, characterized in that, The method includes: Acquire the lighting image to be processed; Based on the local illumination distribution characteristics of the illumination image to be processed, the illumination image to be processed after global illumination enhancement is enhanced at multiple scales to obtain the first illumination image after spatial enhancement. The local illumination distribution characteristics include the mean brightness and the standard deviation of brightness. The amplitude information of the illumination image to be processed in the frequency domain is enhanced to obtain a second illumination image after frequency domain enhancement; and The illumination image to be processed, the first illumination image, and the second illumination image are fused to obtain an enhanced target illumination image; The step of performing multi-scale enhancement on the illumination image to be processed, which has undergone global illumination enhancement, based on the local illumination distribution characteristics of the illumination image to be processed, to obtain a first illumination image after spatial enhancement, includes: Based on the local illumination distribution characteristics of the illumination image to be processed, the illumination image to be processed after global illumination enhancement is enhanced to obtain a first intermediate illumination image. Based on the local illumination distribution characteristics of the first intermediate illumination image, the first intermediate illumination image after global illumination enhancement is enhanced to obtain a second intermediate illumination image; wherein, the scale of the local illumination distribution characteristics of the first illumination image is smaller than the scale of the local illumination distribution characteristics of the illumination image to be processed. Based on the overall illumination distribution characteristics of the second intermediate illumination image, the second intermediate illumination image, which has undergone global illumination enhancement, is enhanced to obtain the first illumination image.

2. The method according to claim 1, characterized in that, The step of enhancing the global illumination enhancement image of the illumination image to be processed based on the local illumination distribution characteristics of the illumination image to be processed, to obtain a first intermediate illumination image, includes: The global illumination-enhanced image to be processed is divided into regions to obtain multiple first sub-regions of a first size; Based on the local illumination distribution characteristics of each of the first sub-regions and the corresponding sub-regions of the illumination image to be processed, the illumination distribution of the first sub-regions is adjusted to obtain the adjusted first sub-regions; and According to the spatial position of the first sub-region in the illumination image to be processed, multiple adjusted first sub-regions are stitched together to form the first intermediate illumination image.

3. The method according to claim 2, characterized in that, The step of enhancing the first intermediate illumination image (after global illumination enhancement) based on the local illumination distribution characteristics of the first intermediate illumination image to obtain the second intermediate illumination image includes: The first intermediate illumination image after global illumination enhancement is divided into regions to obtain multiple second sub-regions of a second size, wherein the second size is smaller than the first size; Based on the local illumination distribution characteristics of each second sub-region and its corresponding sub-region in the first intermediate illumination image, the illumination distribution of the second sub-region is adjusted to obtain the adjusted second sub-region; and According to the spatial position of the second sub-region in the illumination image to be processed, multiple adjusted second sub-regions are stitched together to form a second intermediate illumination image.

4. The method according to claim 2, characterized in that, The step of adjusting the illumination distribution of the first sub-region based on the local illumination distribution characteristics of each first sub-region and the corresponding sub-region of the illumination image to be processed, to obtain the adjusted first sub-region, includes: Obtain the local illumination distribution features of each sub-region corresponding to the first sub-region extracted from the illumination image to be processed; The mean luminance and standard deviation luminance are corrected by adjusting the parameters learned adaptively, resulting in the corrected mean luminance and standard deviation luminance. Based on the corrected mean luminance, the corrected standard deviation luminance, the mean luminance, and the standard deviation luminance, the illumination distribution of the first sub-region is adjusted to obtain the adjusted first sub-region.

5. The method according to claim 1, characterized in that, The process of fusing the illumination image to be processed, the first illumination image, and the second illumination image to obtain the enhanced target illumination image includes: The pixel-by-pixel summation of the illumination image to be processed, the first illumination image, and the second illumination image is performed to obtain an aggregated feature map. Based on the attention mechanism, a first weight, a second weight, and a third weight are generated according to the aggregated feature map; The superposition results of the first weight and the first illumination image, the superposition results of the second weight and the illumination image to be processed, and the superposition results of the third weight and the second illumination image are fused to obtain a fused feature map; and The fused feature map after feature alignment is concatenated with the residual of the illumination image to be processed to obtain the enhanced target illumination image.

6. The method according to claim 5, characterized in that, The generation of first weights, second weights, and third weights based on the attention mechanism and the aggregated feature map includes: The aggregated feature map is subjected to global pooling and average pooling respectively, and the results of global pooling and average pooling are concatenated to obtain the concatenated feature map. The stitched features are input into three feature extraction branches respectively, and the outputs are the first feature value, the second feature value and the third feature value that enhance the features of the first illumination image, the illumination image to be processed and the second illumination image respectively; The first eigenvalue, the second eigenvalue, and the third eigenvalue are processed using a normalization function to obtain the first weight, the second weight, and the third weight.

7. The method according to claim 1, characterized in that, The step of enhancing the amplitude information of the illumination image to be processed in the frequency domain to obtain a second illumination image after frequency domain enhancement includes: Perform a Fourier transform on the illumination image to be processed for each color channel to obtain frequency domain information; The amplitude information separated from the frequency domain information is enhanced to obtain the enhanced amplitude information; Perform an inverse Fourier transform on the enhanced amplitude information and the phase information in the frequency domain information to obtain the third illumination image for each color channel after frequency domain enhancement; and The third illumination image of each color channel is stitched together to obtain the second illumination image.

8. The method according to any one of claims 1 to 7, characterized in that, The illumination enhancement model is configured to generate the target illumination image based on the illumination image to be processed; the illumination enhancement model is trained in the following manner: Input the sample lighting image into the lighting enhancement model to be trained to generate the target sample lighting image; Using a loss function, the illumination enhancement model to be trained is adjusted based on the target sample illumination image and the label illumination image of the real illumination corresponding to the sample illumination image until the training conditions are met, thus obtaining the trained illumination enhancement model. The loss function includes: pixel reconstruction loss, which characterizes the pixel differences between the target sample illumination image and the label illumination image, and amplitude consistency loss, which characterizes the amplitude differences between the target sample illumination image and the label illumination image.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more programs. The characteristic is that, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 8.

Images