Low-light image enhancement method and system based on optical prior and spectral constraint

Abstract

The application discloses a low-illumination image enhancement method and system based on optical prior and spectrum constraint, and the method comprises the following steps: acquiring a low-light image to be processed; and processing the low-light image by using a pre-trained image enhancement network, specifically: performing preset optical prior extraction on the low-light image to obtain optical prior images corresponding to three illumination-independent quantities; inputting the first optical prior image, the second optical prior image and the brightness prior image as auxiliary information into a preset first branch network together with the low-light image to extract an illumination map; inputting the third optical prior image as auxiliary information into a preset second branch network together with the low-light image to extract a structure map; multiplying the illumination map and the structure map and then adding the structure map to obtain a preliminary enhanced image; and performing denoising, color enhancement and brightness enhancement on the preliminary enhanced image to obtain a final enhanced image. The application can improve the feature expression capability of the network and the image enhancement effect.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a method and system for enhancing low-light images based on optical priors and spectral constraints. Background Technology

[0002] In daily life, many factors often prevent images from being captured under ideal lighting conditions, such as weak illumination and insufficient exposure time. These physical factors result in low-light images where most areas are dark, leading to degradation such as loss of detail, reduced contrast, amplified sensor noise, and color distortion. This degradation not only worsens the visual quality of the image but also affects downstream computer vision tasks, such as object detection, image matching, and semantic segmentation. Low-light image enhancement aims to restore low-light input images to well-exposed images, which is not only a visual requirement but also a crucial preprocessing step before downstream applications. Currently, although some neural network methods exist for low-light image enhancement, existing low-light image enhancement algorithms still suffer from the following problems:

[0003] (1) When using neural networks to extract low-light image features, there is a lack of extraction and use of prior information on imaging optics. This makes the neural network lack the support and guidance of a real physical model when modeling, which restricts the space for further improvement of image enhancement effect.

[0004] (2) When using neural networks to enhance low-light images, learning is usually done only in the spatial domain, lacking the use of frequency domain information. This makes it impossible for the neural network to fully mine useful information in the image, and also makes the overall enhancement effect not good enough.

[0005] Therefore, how to enhance the extraction of prior information on image illumination and utilize frequency domain information to aid network learning are key technical issues that urgently need to be addressed. Summary of the Invention

[0006] The main objective of this invention is to provide a low-light image enhancement method and system based on optical priors and spectral constraints that can simultaneously solve the above-mentioned problems.

[0007] The technical solution adopted in this invention is:

[0008] A low-light image enhancement method based on optical priors and spectral constraints is provided, comprising the following steps:

[0009] Acquire the low-light image to be processed;

[0010] The low-light image is processed using a pre-trained image enhancement network, specifically as follows:

[0011] The low-light image is subjected to a preset optical prior extraction to obtain optical prior maps corresponding to three illumination-independent quantities. The first optical prior map is the image H, which is illumination-independent under the assumption of constant spectral wavelength. The second optical prior map is the image S, which is illumination-independent under the further assumption of a matte object surface. The third optical prior map is the image W, which is illumination-independent under the further assumption of uniform illumination.

[0012] The first optical prior map, the second optical prior map, and the luminance prior map are used as auxiliary information and input together with the low-light image into the preset first branch network to extract the illuminance map; wherein, the luminance prior map is obtained by averaging the RGB three channels of the low-light image;

[0013] The third optical prior image is used as auxiliary information and input together with the low-light image into the preset second branch network to extract the structure map;

[0014] The illuminance map and the structure map are multiplied together and then added back to the structure map to obtain the preliminary enhanced image;

[0015] The initially enhanced image is denoised and its color and brightness are enhanced to obtain the final enhanced image.

[0016] During training, the image enhancement network introduces a Fourier loss function to perform a two-dimensional Fourier transform on the input image to obtain a spectrum, and learns the texture details and brightness of the image. The output image is then decomposed into high-frequency and low-frequency images using a fast Fourier transform, and then pixel-by-pixel constraints are applied to each image.

[0017] Following the above technical solution, during optical prior extraction, the spectral equation of the low-light image and its first and second partial derivatives with respect to wavelength are associated with the RGB spatial pixel values ​​of the low-light image to generate a mapping matrix. The spectrum of the low-light image is processed by a Gaussian derivative filter to obtain two spatial derivative maps. Then, combined with the expressions of three illumination-independent quantities, the first optical prior map, the second optical prior map, and the third optical prior map are calculated.

[0018] Following the above technical solution, the IGT module in the Retinexformer network is used to denoise the initially enhanced image. During each downsampling and upsampling process, the IGAB attention module guided by illumination information is inserted, and the intermediate features of the first branch network are introduced to guide the network to learn the contextual information of the image.

[0019] Following the above technical solution, during training, the IGT module introduces a Fourier loss function, performs a two-dimensional Fourier transform on the input image to obtain a spectrum, and uses a fast Fourier transform to decompose the output image into high-frequency and low-frequency images, and applies pixel-wise L1 regularization constraints to the high-frequency and low-frequency images respectively.

[0020] Following the above technical solution, during training, the IGT module performs the same preprocessing operations on each pair of images in the training set to expand the dataset, including flipping, rotating and translating.

[0021] Following the above technical solution, the IGT module is evaluated using peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) during testing.

[0022] Following the above technical solution, the U-net network is specifically used to denoise the initially enhanced image.

[0023] The present invention also provides a low-light image enhancement system based on optical priors and spectral constraints, comprising:

[0024] The image acquisition module is used to acquire the low-light image to be processed;

[0025] The image enhancement network module is used to process the low-light image through a pre-trained image enhancement network, specifically including:

[0026] The optical prior extraction submodule is used to perform preset optical prior extraction on the low-light image to obtain optical prior maps corresponding to three illumination-independent quantities. The first optical prior map is the image H, which is illumination-independent under the assumption of constant spectral wavelength; the second optical prior map is the image S, which is illumination-independent under the further assumption of a matte object surface; and the third optical prior map is the image W, which is illumination-independent under the further assumption of uniform illumination.

[0027] The image decomposition submodule is used to input the first optical prior image H, the second optical prior image S, and the brightness prior image as auxiliary information, together with the low-light image, into a preset first branch network to extract the illuminance map; wherein, the brightness prior image is obtained by averaging the RGB three channels of the low-light image; and the third optical prior image W is used as auxiliary information, together with the low-light image, into a preset second branch network to extract the structure map;

[0028] The denoising and enhancement submodule is used to multiply the illumination map and the structure map and then add them to the structure map to obtain a preliminary enhanced image; and to denoise the preliminary enhanced image and enhance its color and brightness to obtain the final enhanced image;

[0029] During training, the image enhancement network introduces a Fourier loss function to perform a two-dimensional Fourier transform on the input image to obtain a spectrum, and learns the texture details and brightness of the image. The output image is then decomposed into high-frequency and low-frequency images using a fast Fourier transform, and then pixel-by-pixel constraints are applied to each image.

[0030] Following the above technical solution, the optical prior extraction submodule is specifically used to associate the spectral equation of the low-light image and its first and second partial derivatives with respect to wavelength with the RGB spatial pixel values ​​of the low-light image to generate a mapping matrix; the spectrum of the low-light image is processed by a Gaussian derivative filter to obtain two spatial derivative maps, and then combined with the expressions of three illumination-independent quantities to calculate the first optical prior map H, the second optical prior map S, and the third optical prior map W.

[0031] The present invention also provides a computer storage medium storing a computer program executable by a processor, the computer program performing the low-light image enhancement method based on optical priors and spectral constraints described in the above technical solution.

[0032] The beneficial effects of this invention are as follows: By extracting optical prior maps and inputting them into the illuminance map branch network and the structure map branch network according to their characteristics to generate illuminance maps and structure maps, this invention achieves the extraction and utilization of optical prior information, providing physical model support and guidance for the network to mine image features, and improving the network's enhancement effect. Furthermore, this invention introduces Fourier loss into the low-light image enhancement network, allowing the network to better focus on high-frequency and low-frequency information in the image. By combining spatial and frequency domain information, it further enhances the network's feature representation ability and image enhancement effect.

[0033] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram illustrating the improved approach of the present invention based on the traditional Retinex theoretical method;

[0036] Figure 2 This is a flowchart of the low-light image enhancement method based on optical priors and spectral constraints according to an embodiment of the present invention;

[0037] Figure 3 This is a schematic diagram of the method framework of an embodiment of the present invention. Figure 1 ;

[0038] Figure 4 This is a schematic diagram of the method framework of an embodiment of the present invention. Figure 2 ;

[0039] Figure 5 This is a schematic diagram of the optical prior image extraction process according to an embodiment of the present invention;

[0040] Figure 6 This is a schematic diagram of the IGT module structure according to an embodiment of the present invention;

[0041] Figure 7 This is a schematic diagram of low-light image enhancement according to an embodiment of the present invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0043] It should be noted that the illustrations provided in the embodiments of the present invention are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0044] In this invention, it should also be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this application. Furthermore, the terms "first" and "second" are used only for descriptive and distinguishing purposes, and should not be construed as indicating or implying relative importance. In addition, it should be noted that the features of the various embodiments of this invention can be combined or integrated in part or in whole, and as those skilled in the art will understand, can interact and operate in different ways. Each embodiment can be implemented independently of each other, or implemented in a related relationship.

[0045] According to Retinex theory, a low-light image I can be decomposed into the product of a reflectance map R and an illuminance map L:

[0046]

[0047] The classic Retinex theory simulates the color perception mechanism of the human eye, restoring the true color and contrast of an image by separating illuminance and reflection.

[0048] However, the classical Retinex theory lacks the guidance of a physical model, which limits the potential for further improvement in image enhancement. Furthermore, the lack of utilization of frequency domain information prevents neural networks from fully extracting useful information from images. Figure 1 As shown, in order to solve the above two technical problems, the present invention mainly achieves this by introducing optical prior information and Fourier loss.

[0049] like Figure 2-4 As shown, the low-light image enhancement method based on optical priors and spectral constraints in this embodiment of the invention includes the following steps:

[0050] S1. Obtain the low-light image to be processed;

[0051] S2. Process the low-light image using a pre-trained image enhancement network, specifically including the following steps:

[0052] S21. Perform a preset optical prior extraction on the low-light image to obtain optical prior maps corresponding to three illumination-independent quantities. The first optical prior map is the image H that is illumination-independent under the assumption of constant spectral wavelength. The second optical prior map is the image S that is illumination-independent when the object surface is further assumed to be matte. The third optical prior map is the image W that is illumination-independent under the further assumption of uniform illumination.

[0053] S22. The first optical prior map, the second optical prior map, and the luminance prior map are used as auxiliary information and input together with the low-light image into the preset first branch network to extract the illuminance map; wherein, the luminance prior map is obtained by averaging the RGB three channels of the low-light image;

[0054] S23. The third optical prior image is used as auxiliary information and input together with the low-light image into the preset second branch network to extract the structure map.

[0055] S24. Multiply the illuminance map and the structure map, and then add them to the structure map to obtain the preliminary enhanced image;

[0056] S25. Denoise the initially enhanced image and enhance its color and brightness to obtain the final enhanced image.

[0057] In this embodiment of the invention, during training, the image enhancement network introduces a Fourier loss function, performs a two-dimensional Fourier transform on the input image to obtain a spectrum, enhances the learning of image texture details and brightness through image gradient constraints, and decomposes the output image into high-frequency and low-frequency images using fast Fourier transform, and performs pixel-by-pixel constraints on each.

[0058] In step S21, the optical prior extraction process is as follows:

[0059] According to the Kubelka-Munk theory describing color image imaging, given a wavelength λ, the incident spectrum E at spatial position x on the imaging plane can be expressed as:

[0060]

[0061] in, Indicates the spectrum of the light source, Indicates specular reflectivity. This represents the reflectivity of a material that is independent of illumination. For this theory, three assumptions are proposed, resulting in illumination-independent quantities under three different conditions.

[0062] Assumption 1: When the light is of equal energy, i.e. the spectral wavelength is constant, the light source spectrum... This can be simplified to wavelength-independent. :

[0063]

[0064] From this, we can deduce that:

[0065]

[0066] in, and Let E represent the first and second partial derivatives with respect to λ. and express The first and second partial derivatives with respect to λ. The result of formula x depends only on the material reflectivity and is independent of illumination; therefore, we can obtain the first illumination-independent quantity. :

[0067]

[0068] Assumption 2: Further assume that the object's surface is matte, i.e., specular reflectance. This allows us to derive a second light-independent quantity. :

[0069]

[0070] Assumption 3: Based on the above, if we continue to assume uniform illumination, then the spectrum of the light source can be simplified to a parameter independent of spatial location. A third light-independent quantity can be derived. :

[0071]

[0072] After deriving the three illumination-independent quantities from the spectral equation, it is necessary to correlate the variables of the spectral equation with the RGB color space. Based on the Gaussian color model, this involves the spectrum E and its partial derivative with respect to wavelength. and Linear mapping can be achieved using RGB pixel values:

[0073]

[0074] To achieve better mapping results, such as Figure 5 As shown, this invention can optionally use a convolutional layer as the extraction module for the illumination prior map to learn and generate the aforementioned 3×3 mapping matrix. Through the mapping matrix, the spectral map E and its partial derivative with respect to wavelength can be obtained. and The image is then processed using a Gaussian derivative filter to obtain the spatial derivative plot of E. and E, , , , Substituting these into the derived formulas for the three illumination-independent quantities, we can obtain three optical prior maps H, S, and W.

[0075] After extracting the optical prior maps H, S, and W, the structure maps are extracted from the low-light image I in the first branch network (structure map branch) and the first branch network (illuminance map branch) respectively, based on the previously extracted optical prior maps. Illuminance diagram The structure diagram Illuminance diagram Multiplication and AND structure diagram Adding them together yields a preliminary enhanced image. Then, the preliminary enhanced image Intermediate features of the illuminance map branch The images are input together into the denoising and enhancement module to obtain the final enhanced image. .

[0076] Specifically, according to Retinex theory, an image can be decomposed into an illuminance map and a reflectance map, the product of which is the original image. Based on this theory, this patent extracts a structure map with good image detail and an illuminance map reflecting illumination information from a low-light image, guided by optical and luminance prior maps. To this end, this invention designs two sets of identical network structures: a structure map branch and an illuminance map branch. These two sets of branches extract the structure map from the low-light image I, respectively. Illuminance diagram In addition to the low-light image I, the inputs to both branches also include the extracted optical prior maps H, S, W, and the brightness prior map. As supplementary information, the brightness prior map This is obtained by averaging the RGB three channels of the low-light image I. The optical prior map is derived from illumination-independent properties. Observation shows that different components of the optical prior map convey different information: optical prior maps H and S reflect luminance information more, while optical prior map W reflects structural information more. Therefore, H and S are input into the illuminance map branch as auxiliary information, and W is input into the structure map branch as auxiliary information. At the same time, to help the network better learn luminance information, an additional luminance prior map is used. As input to the illuminance map branch.

[0077] The structures of the illuminance map branch and the structure map branch can use ResNet blocks and depthwise separable convolutions. This patent uses depthwise separable convolutions to extract the illuminance map and structure map from the low-light image.

[0078] After obtaining the structure map and illumination map through image decomposition, in order to better preserve structural information, the structure map is... Illuminance diagram Multiplication and AND structure diagram Adding them together yields a preliminary enhanced image. Preliminary image enhancement While image details will be improved, enhancements are still needed in color, brightness, and noise suppression. To address these issues, preliminary image enhancement will be performed. The input denoising enhancement module further enhances the process, while also incorporating intermediate features from the illumination map branch. The input is processed together as auxiliary information, and the final enhanced image is output. .like Figure 7 As shown, the left column is the input low-light image, which is then processed by the method of this invention to obtain the enhanced right column of images.

[0079] Preferably, the denoising enhancement module can be implemented using an IGT (Illumination-Guided Multi-head Self-Attention) module to enhance the color, brightness, and noise suppression of the initially enhanced image. The IGT module adopts a U-net-like structure (e.g., Figure 6 As shown in the diagram, the overall module can be divided into a downsampling branch and an upsampling branch. The input image is first fed into the downsampling branch. In the downsampling branch, the input image is first processed through a 3×3 convolution to obtain features. Then, downsampled features at scales of 1 / 2 and 1 / 4 are generated by sequentially performing one 4×4 stride convolution with IGAB and stride of 2, and two 4×4 stride convolutions with IGAB and stride of 2. and Then let After two IGAB processes, the result was obtained. The upsampling branch is symmetrical with respect to the downsampling branch, which will... Upsampling is performed by repeating two sets of 2×2 deconvolution + 1×1 convolution + two IGAB operations with a stride of 2, resulting in upsampled features. and Skip connections are set between upsampled and downsampled features of the same scale to prevent information loss. Finally, The residual image is obtained through 3×3 convolution. The residual image is added to the input image to obtain the output image, which is the final enhanced image. The Illumination-Guided Attention Block (IGAB) introduces intermediate features from the illumination map branch to better guide the network in learning the image's contextual information. IGAB first reconstructs the 3D input features, then uses three fully connected layers to linearly project the reconstructed features to obtain m sets of... , , Matrix. Simultaneously, the illuminance features are reconstructed and linearly projected to obtain m... Matrix, and with the corresponding Matrix multiplication yields This introduces illumination information. For each set of matrices, the transpose of matrix K is multiplied by matrix Q, and the result is then multiplied by matrix V' to achieve self-attention. The results of m sets of self-attention are concatenated, processed through a fully connected layer, and then coupled with position encoding to obtain the output features.

[0080] To enable image augmentation networks to better capture low-frequency information representing brightness and color, and high-frequency information representing texture and detail, Fourier loss is introduced. The frequency of an image is an indicator of the drastic changes in grayscale levels; it represents the gradient of grayscale levels in a two-dimensional space. Performing a two-dimensional Fourier transform on the image yields a spectrum, which is the distribution map of the image gradient. Constraining the image gradient can enhance the network's learning of texture details and brightness. This results in an enhanced image output by the network. and the corresponding normal light image Using Fast Fourier Transform will enhance the image. Decomposed into high-frequency images and low-frequency images Similarly, normal light image Decomposed into high-frequency images and low-frequency images Then, pixel-wise L1 constraints are applied to both the high-frequency and low-frequency images to make the high-frequency and low-frequency images of the enhanced and normal light image decomposed as similar as possible, helping the network to better learn frequency domain information.

[0081]

[0082] In addition, conventional reconstruction loss is used to enhance the image. and the corresponding normal light image Apply pixel-by-pixel L1 constraints to ensure that the pixel values ​​at corresponding positions in the enhanced image and the normal light image are as similar as possible:

[0083]

[0084] The total loss function is as follows:

[0085]

[0086] To enhance low-light images using the image enhancement network designed in this invention, a dataset needs to be constructed to train the model. This patent trains the network on an Nvidia 3060 GPU using PyTorch with a batch size of 8 and the Adam optimizer. The LOL_v2_synthetic dataset is used as the initial dataset. LOL_v2_synthetic synthesizes low-light images from RAW images by analyzing the illumination distribution of low-light images. This dataset contains 1,000 pairs of low-light / normal images, with 900 pairs used for training and 100 pairs used for testing.

[0087] The specific training steps are as follows:

[0088] (1) Perform the same preprocessing operations such as flipping, rotating, and translating on each pair of images in the training set to expand the dataset and enhance the generalization of the model to low-light image enhancement.

[0089] (2) The model was trained using the training set, for a total of 150,000 iterations;

[0090] (3) Adjust the hyperparameters of the model based on the changes in the model’s accuracy on the validation set, and finally save the network model that performs best on the test set.

[0091] Model Testing: The saved model is used to enhance images in the test set, outputting the enhancement results for low-light images. The enhanced images are then evaluated against the corresponding normal-light images in the test set, using PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index Measure) as evaluation metrics.

[0092] PSNR: Peak Signal-to-Noise Ratio (PSNR) is commonly used to measure the level of image distortion or noise. A higher PSNR value between two images indicates greater similarity. Given a clean image x and a noisy image y of size m×n, the PSNR is calculated as follows:

[0093]

[0094]

[0095] in, The maximum possible pixel value of image I is 255 in this invention.

[0096] SSIM: Structural Similarity (SSIM) measures the similarity between two images based on brightness, contrast, and structure. A higher SSIM value indicates greater similarity. Given a clean image x and a noisy image y of size m×n, SSIM is calculated as follows:

[0097]

[0098] in, and Let x and y be the means of x and y, respectively. and Let x and y be the variances, respectively. Let xy be the variance. , L is set to 255. Set it to 0.01. Set it to 0.03.

[0099] The final test results are shown in Table 1 below:

[0100] Table 1. Comparison of PSNR and SSIM metrics for various models

[0101]

[0102] This invention extracts optical prior images and, based on their characteristics, inputs them into the illuminance map branch and the structure map branch respectively to generate illuminance and structure maps. This enables the extraction and utilization of optical prior information, providing physical model support and guidance for network-based image feature mining and improving the network's enhancement effect. Furthermore, the introduction of Fourier loss into the low-light image enhancement network allows the network to better focus on high-frequency and low-frequency information in the image. By combining spatial and frequency domain information, the network's feature representation ability and image enhancement effect are further improved.

[0103] To implement the above method embodiments, the present invention also provides a low-light image enhancement system based on optical priors and spectral constraints, comprising:

[0104] The image acquisition module is used to acquire the low-light image to be processed;

[0105] The image enhancement network module is used to process the low-light image through a pre-trained image enhancement network, specifically including:

[0106] The optical prior extraction submodule is used to perform preset optical prior extraction on the low-light image to obtain optical prior maps corresponding to three illumination-independent quantities. The first optical prior map is the image H, which is illumination-independent under the assumption of constant spectral wavelength; the second optical prior map is the image S, which is illumination-independent under the further assumption of a matte object surface; and the third optical prior map is the image W, which is illumination-independent under the further assumption of uniform illumination.

[0107] The image decomposition submodule is used to input the first optical prior image H, the second optical prior image S, and the brightness prior image as auxiliary information, together with the low-light image, into a preset first branch network to extract the illuminance map; wherein, the brightness prior image is obtained by averaging the RGB three channels of the low-light image; and the third optical prior image W is used as auxiliary information, together with the low-light image, into a preset second branch network to extract the structure map;

[0108] The denoising and enhancement submodule is used to multiply the illumination map and the structure map and then add them to the structure map to obtain a preliminary enhanced image; and to denoise the preliminary enhanced image and enhance its color and brightness to obtain the final enhanced image.

[0109] During training, the image enhancement network introduces a Fourier loss function to perform a two-dimensional Fourier transform on the input image to obtain a spectrogram. The network learns the texture details and brightness of the image through image gradient constraints. The output image is decomposed into high-frequency and low-frequency images using a fast Fourier transform, and pixel-by-pixel constraints are applied to each.

[0110] Specifically, the optical prior extraction submodule is used to associate the spectral equation of the low-light image and its first and second partial derivatives with respect to wavelength with the RGB spatial pixel values ​​of the low-light image to generate a mapping matrix; the spectrum of the low-light image is processed by a Gaussian derivative filter to obtain two spatial derivative maps, and then combined with the expressions of three illumination-independent quantities to calculate the first optical prior map H, the second optical prior map S, and the third optical prior map W.

[0111] Each module is mainly used to implement the various steps of the method embodiment, which will not be elaborated here.

[0112] This application also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, app store, etc., which stores a computer program. When the program is executed by a processor, it implements the corresponding function. This embodiment describes a low-light image enhancement method based on optical priors and spectral constraints, implemented by a computer-readable storage medium executed by a processor.

[0113] It should be noted that, depending on the implementation needs, the various steps / components described in this application can be broken down into more steps / components, or two or more steps / components or parts of the operation of steps / components can be combined into new steps / components to achieve the purpose of this invention.

[0114] The order of the steps in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0115] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A low-light image enhancement method based on optical priors and spectral constraints, characterized in that, Includes the following steps: Acquire the low-light image to be processed; The low-light image is processed using a pre-trained image enhancement network, specifically as follows: The low-light image is subjected to a preset optical prior extraction to obtain optical prior maps corresponding to three illumination-independent quantities. The first optical prior map is the image H, which is illumination-independent under the assumption of constant spectral wavelength. The second optical prior map is the image S, which is illumination-independent under the further assumption of a matte object surface. The third optical prior map is the image W, which is illumination-independent under the further assumption of uniform illumination. The first optical prior map, the second optical prior map, and the luminance prior map are used as auxiliary information and input together with the low-light image into the preset first branch network to extract the illuminance map; wherein, the luminance prior map is obtained by averaging the RGB three channels of the low-light image; The third optical prior image is used as auxiliary information and input together with the low-light image into the preset second branch network to extract the structure map; The illuminance map and the structure map are multiplied together and then added back to the structure map to obtain the preliminary enhanced image; The initial enhanced image is then denoised and its color and brightness are enhanced to obtain the final enhanced image. During training, the image enhancement network introduces a Fourier loss function to perform a two-dimensional Fourier transform on the input image to obtain a spectrum, and learns the texture details and brightness of the image. The output image is then decomposed into high-frequency and low-frequency images using a fast Fourier transform, and then pixel-by-pixel constraints are applied to each image.

2. The low-light image enhancement method based on optical priors and spectral constraints according to claim 1, characterized in that, During optical prior extraction, the spectral equation of the low-light image and its first and second partial derivatives with respect to wavelength are associated with the RGB spatial pixel values ​​of the low-light image to generate a mapping matrix. The spectrum of the low-light image is processed by a Gaussian derivative filter to obtain two spatial derivative maps. Then, combined with the expressions of three illumination-independent quantities, the first, second, and third optical prior maps are calculated.

3. The low-light image enhancement method based on optical priors and spectral constraints according to claim 1, characterized in that, Specifically, the IGT module in the Retinex Former network is used to denoise the initially enhanced image. During each downsampling and upsampling process, the IGAB attention module guided by illumination information is inserted, and the intermediate features of the first branch network are introduced to guide the network to learn the contextual information of the image.

4. The low-light image enhancement method based on optical priors and spectral constraints according to claim 3, characterized in that, During training, the IGT module also applies pixel-wise L1 regularization constraints to the decomposed high-frequency and low-frequency images to ensure that the high-frequency and low-frequency images of the enhanced image and the normal light image decomposed are consistent.

5. The low-light image enhancement method based on optical priors and spectral constraints according to claim 3, characterized in that, During training, the IGT module performs the same preprocessing operations on each pair of images in the training set to augment the dataset, including flipping, rotating, and translating.

6. The low-light image enhancement method based on optical priors and spectral constraints according to claim 3, characterized in that, During testing, the IGT module was evaluated using peak signal-to-noise ratio (PSNR) and structural similarity (SSIM).

7. The low-light image enhancement method based on optical priors and spectral constraints according to claim 1, characterized in that, Specifically, the U-net network is used to denoise the initially enhanced image.

8. A low-light image enhancement system based on optical priors and spectral constraints, characterized in that, include: The image acquisition module is used to acquire the low-light image to be processed; The image enhancement network module is used to process the low-light image through a pre-trained image enhancement network, specifically including: The optical prior extraction submodule is used to perform preset optical prior extraction on the low-light image to obtain optical prior maps corresponding to three illumination-independent quantities. The first optical prior map is the image H, which is illumination-independent under the assumption of constant spectral wavelength; the second optical prior map is the image S, which is illumination-independent under the further assumption of a matte object surface; and the third optical prior map is the image W, which is illumination-independent under the further assumption of uniform illumination. The image decomposition submodule is used to input the first optical prior image H, the second optical prior image S, and the brightness prior image as auxiliary information, together with the low-light image, into a preset first branch network to extract the illuminance map; wherein, the brightness prior image is obtained by averaging the RGB three channels of the low-light image; and the third optical prior image W is used as auxiliary information, together with the low-light image, into a preset second branch network to extract the structure map; The denoising and enhancement submodule is used to multiply the illumination map and the structure map and then add them to the structure map to obtain a preliminary enhanced image; and to denoise the preliminary enhanced image and enhance its color and brightness to obtain the final enhanced image; During training, the image enhancement network introduces a Fourier loss function to perform a two-dimensional Fourier transform on the input image to obtain a spectrum, and learns the texture details and brightness of the image. The output image is then decomposed into high-frequency and low-frequency images using a fast Fourier transform, and then pixel-by-pixel constraints are applied to each image.

9. The low-light image enhancement system based on optical priors and spectral constraints according to claim 8, characterized in that, The optical prior extraction submodule is specifically used to associate the spectral equation of the low-light image and its first and second partial derivatives with respect to wavelength with the RGB spatial pixel values ​​of the low-light image to generate a mapping matrix; the spectrum of the low-light image is processed by a Gaussian derivative filter to obtain two spatial derivative maps, and then combined with the expressions of three illumination-independent quantities to calculate the first optical prior map H, the second optical prior map S, and the third optical prior map W.

10. A computer storage medium, characterized in that, It contains a computer program that can be executed by a processor, which is used to implement the low-light image enhancement method based on optical priors and spectral constraints as described in any one of claims 1-7.

Images