A low-light image enhancement method and system

Abstract

A low-light image enhancement method and system, relating to the field of computer vision, solves the problems of uneven illumination distribution, loss of details and color distortion in the image restoration process in the illumination mapping technology in the prior art. The method includes obtaining a training set and a test set, and the low-light image is subjected to wavelet transform to obtain a low-frequency image subband of the low-light image and high-frequency image subbands in three different directions: horizontal, vertical and diagonal. The low-light image is enhanced based on a feature extraction network model, and the enhanced image is enhanced based on a low-frequency enhancement network model, and the final enhanced image is obtained by processing multiple hierarchical feature extraction structural units; the overall hierarchical network model is trained separately using a loss function; the low-light image in the test set and the corresponding normal illumination image form a sample pair, and the above operation is repeated to obtain the final enhanced image. The present invention is applicable to fields such as security monitoring.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a low-light image enhancement method and system. Background Art

[0002] In the visual system, nighttime and low-light conditions often lead to a significant decline in the quality of captured images, which is manifested in blurred details, more noise in dark areas, and overall grayness, which seriously affects the execution of various visual tasks, such as autonomous driving, target detection, etc.

[0003] With the rapid development of deep learning, a large number of deep learning-based algorithms have emerged in the field of low-light enhancement. However, existing low-light image enhancement methods are usually explored in the spatial domain and are mainly divided into two categories. The first method uses convolution to learn the distribution and characteristics of normal light images, but this method has poor generalization and incomplete basic theory due to limitations such as data sets. The second method is mainly based on Retinex, a combination of the retina and the cerebral cortex, which mines brightness information by decomposing the image. However, this method is difficult to effectively deal with the noise introduced in the decomposition step.

[0004] Existing low-light image enhancement methods mainly improve image quality by refining image details or uniformly improving global brightness, but these methods often have the following problems. They fail to fully utilize image information, resulting in uneven brightness and blurred details; they ignore the use of frequency domain features and instance-level semantic information of the image, and cannot effectively utilize high-frequency information in the frequency domain; the computational overhead is high, such as the transformer that captures long-range dependencies. Frequency domain features play an important role in restoring the structure and texture of images, while instance-level semantic information helps maintain the consistency and authenticity of images, especially in complex and diverse scenes.

[0005] In general, in order to effectively extract and enhance the details and semantic information of images under low-light conditions, while reducing computational overhead and achieving natural-perceived lighting effects for images, a stable and efficient low-light image enhancement method is urgently needed to overcome problems such as insufficient illumination and excessive noise. Summary of the invention

[0006] The present invention aims to solve the problems of uneven illumination distribution, detail loss and color distortion in the image restoration process in the illumination mapping technology in the prior art.

[0007] To solve the above technical problems, the present invention is achieved through the following technical solutions:

[0008] Solution 1: The present invention proposes a low-light image enhancement method, which includes the following steps:

[0009] Step 1: respectively obtain a training set and a test set, wherein both the training set and the test set are low-light images and their corresponding normal-light images;

[0010] Step 2: The low-light image is transformed by wavelet transform to obtain low-frequency image subbands of the low-light image and high-frequency image subbands in three different directions: horizontal, vertical, and diagonal.

[0011] Step 3, the high-frequency image subband is enhanced based on the feature extraction network model, that is, an enhanced low-frequency image and a high-frequency image are obtained;

[0012] Step 4, the enhanced high-frequency image is obtained by collaborative enhancement based on the high-frequency Mamba enhancement network model, and the enhanced stage image is obtained based on inverse wavelet transform;

[0013] Step 5, the enhanced stage image is enhanced based on the low-frequency enhancement network model, and processed by multiple hierarchical feature extraction structural units to obtain a processed enhanced image;

[0014] Step 6: Based on the processed enhanced image obtained in step 5 and the normal-light image corresponding to the low-light image in the training set, a multi-dimensional loss function is designed, and the feature extraction network model, the high-frequency Mamba enhancement network model, and the low-frequency network enhancement model are trained respectively using the loss function;

[0015] Step 7: The low-light image in the test set and its corresponding normal-light image form a sample pair, and the operations of steps 2 to 5 are performed to obtain the final enhanced image.

[0016] Further, a preferred implementation manner is provided, in which the low-frequency image subband and the three high-frequency image subbands of horizontal, vertical and diagonal of the low-light image in step 3 are enhanced based on the feature extraction network model by:

[0017] The low-frequency image features are extracted based on wavelet transform convolution, and the horizontal, vertical and diagonal high-frequency image sub-bands are extracted based on depthwise separable convolution.

[0018] Further, a preferred implementation manner is provided, wherein the enhanced high-frequency image in step 4 is obtained by collaboratively enhancing the high-frequency Mamba enhancement network model as follows:

[0019] The three high-frequency image information in the horizontal, vertical and diagonal directions are processed respectively based on the direction-aware Mamba structure. The direction-aware Mamba structure contains a 2D selective scanning module, that is, the enhanced high-frequency information in different directions is obtained. The low-frequency image and the high-frequency image are reconstructed based on the inverse wavelet transform to obtain the enhanced image, which is used as the input of the low-frequency enhancement network model.

[0020] Furthermore, a preferred implementation is provided, in which low-frequency information and high-frequency information are reconstructed based on inverse wavelet transform, and all sequences are combined to reconstruct the original 2D framework, specifically:

[0021]

[0022] In the formula, is the stage enhanced image obtained after processing, is the inverse wavelet transform algorithm, is the low-frequency image subband based on wavelet transform, is the horizontal high-frequency image subband based on wavelet transform, is the vertical high-frequency image subband based on wavelet transform, is the diagonal high-frequency image subband based on wavelet transform.

[0023] Further, a preferred implementation is provided, in which the low-frequency enhancement network model described in step 5 is specifically:

[0024] The low-frequency enhancement network model consists of two sub-block structures connected in series.

[0025] Among them, the first sub-block structure is composed of five layers connected in series, which is the fast Fourier convolution layer FFConv, followed by the normalization layer LN, the wavelet transform convolution layer WTConv, followed by the SimpleGate activation layer SG, and the last layer is the wavelet transform convolution layer;

[0026] The second sub-block structure consists of five sub-layers, namely the fast Fourier convolution layer, the layer normalization layer, the channel attention layer CA, the SimpleGate activation layer, and finally another wavelet transform-based convolution layer.

[0027] Furthermore, a preferred implementation manner is provided, wherein the two serially connected sub-blocks perform data circulation and transmission via a residual connection.

[0028] Further, a preferred implementation is provided, in which the multi-dimensional loss function in step 6 is specifically:

[0029]

[0030] In the formula, is the overall loss function, is the mean square error loss, is the structural similarity loss, is the perceived loss, For causal analysis of losses, is the instance-level semantic loss, ,in, are the weights of different loss functions respectively.

[0031] Solution 2: A low-light image enhancement system, the system comprising:

[0032] A training module, used to obtain a training set and a test set respectively, wherein the training set and the test set are both low-light images and their corresponding normal-light images;

[0033] A processing module, for obtaining low-light image low-frequency image subbands and high-frequency image subbands in different horizontal, vertical and diagonal directions through wavelet transformation of low-light image;

[0034] A feature extraction network model module, used for enhancing the high-frequency image subband based on the feature extraction network model, that is, obtaining enhanced low-frequency images and high-frequency images;

[0035] A collaborative enhancement module, for collaboratively enhancing the enhanced high-frequency image based on the high-frequency Mamba enhancement network model, and obtaining an enhanced stage image based on an inverse wavelet transform;

[0036] A hierarchical feature extraction module is used for obtaining the enhanced stage image by enhancing it based on a low-frequency enhancement network model, and obtaining the processed enhanced image by processing multiple hierarchical feature extraction structural units;

[0037] A training module is used to design a multi-dimensional loss function based on the enhanced image processed by the feature extraction module and the normal-light image corresponding to the low-light image in the training set, and use the loss function to train the feature extraction network model, the high-frequency Mamba enhancement network model, and the low-frequency network enhancement model respectively;

[0038] The enhancement module is used to form a sample pair of low-light images in the test set and their corresponding normal-light images, and execute operations from the processing module to the feature extraction module to obtain the final enhanced image.

[0039] Solution three: A computer device includes a processor and a storage medium, wherein the memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes any one of the methods described in Solution one.

[0040] Solution 4: A computer storage medium is used to store a computer program. When the computer program is read by a computer, the computer executes any one of the methods described in Solution 1.

[0041] The present invention is beneficial in that:

[0042] The present invention uses fast Fourier convolution and wavelet convolution, combined with the processing of high-frequency information and low-frequency information, to restore the details of the image in a refined manner, while avoiding the drastic increase in computational overhead caused by relying on large convolution kernels in traditional methods. This method not only retains the importance of frequency domain features, but also improves computational efficiency.

[0043] The present invention introduces causal analysis and contrastive learning to enhance image details while avoiding the interference of irrelevant factors in traditional methods.

[0044] The present invention significantly improves the semantic understanding of images in complex scenes, making the generated images more semantically reasonable and consistent.

[0045] The present invention demonstrates its powerful ability in improving image quality, effectively solving problems such as uneven brightness and detail loss in low-light image enhancement, and improving the overall image quality.

[0046] The low-light image enhancement method described in the present invention performs a large number of actual experiments on image data sets under low-light conditions and compares and analyzes them with the existing technology. The experimental results show that the method can not only effectively improve the image brightness, but also significantly reduce the noise level, maintain the clarity of image details, and reduce the demand for computing resources.

[0047] The present invention is applied to security monitoring, intelligent transportation, medical imaging, unmanned driving, satellite remote sensing and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of a low-light image enhancement method described in Implementation Example 1. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the implementation methods of the present application clearer, the technical solutions in the implementation methods of the present application will be clearly and completely described below in conjunction with the drawings in the implementation methods of the present application. Obviously, the described implementation methods are only part of the implementation methods of the present application, not all of the implementation methods.

[0050] Embodiment 1: This embodiment provides a low-light image enhancement method, and the low-light image enhancement method includes the following steps:

[0051] Step 1: respectively obtain a training set and a test set, wherein both the training set and the test set are low-light images and their corresponding normal-light images;

[0052] Step 2: The low-light image is transformed by wavelet transform to obtain low-frequency image subbands of the low-light image and high-frequency image subbands in three different directions: horizontal, vertical, and diagonal.

[0053] Step 3, the high-frequency image subband is enhanced based on the feature extraction network model, that is, an enhanced low-frequency image and a high-frequency image are obtained;

[0054] Step 4, the enhanced high-frequency image is obtained by collaborative enhancement based on the high-frequency Mamba enhancement network model, and the enhanced stage image is obtained based on inverse wavelet transform;

[0055] Step 5, the enhanced stage image is enhanced based on the low-frequency enhancement network model, and processed by multiple hierarchical feature extraction structural units to obtain a processed enhanced image;

[0056] Step 6: Based on the processed enhanced image obtained in step 5 and the normal-light image corresponding to the low-light image in the training set, a multi-dimensional loss function is designed, and the feature extraction network model, the high-frequency Mamba enhancement network model, and the low-frequency network enhancement model are trained respectively using the loss function;

[0057] Step 7: The low-light image in the test set and its corresponding normal-light image form a sample pair, and the operations of steps 2 to 5 are performed to obtain the final enhanced image.

[0058] Implementation method 2: This implementation method further limits the low-light image enhancement method described in implementation method 1. In step 3, the low-frequency image subband and the three high-frequency image subbands of horizontal, vertical and diagonal low-light images are enhanced based on the feature extraction network model. The method is as follows:

[0059] The low-frequency image features are extracted based on wavelet transform convolution, and the horizontal, vertical and diagonal high-frequency image sub-bands are extracted based on depthwise separable convolution.

[0060] Implementation method 3: This implementation method further limits the low-light image enhancement method described in implementation method 2. The enhanced high-frequency image in step 4 is obtained by collaborative enhancement based on the high-frequency Mamba enhancement network model:

[0061] The three high-frequency image information in the horizontal, vertical and diagonal directions are processed respectively based on the direction-aware Mamba structure. The direction-aware Mamba structure contains a 2D selective scanning module, that is, the enhanced high-frequency information in different directions is obtained. The low-frequency image and the high-frequency image are reconstructed based on the inverse wavelet transform to obtain the enhanced image, which is used as the input of the low-frequency enhancement network model.

[0062] Embodiment 4: This embodiment further limits the low-light image enhancement method described in Embodiment 3. Low-frequency information and high-frequency information are reconstructed based on inverse wavelet transform, and all sequences are merged to reconstruct the original 2D framework. Specifically:

[0063]

[0064] In the formula, is the stage enhanced image obtained after processing, is the inverse wavelet transform algorithm, is the low-frequency image subband based on wavelet transform, is the horizontal high-frequency image subband based on wavelet transform, is the vertical high-frequency image subband based on wavelet transform, is the diagonal high-frequency image subband based on wavelet transform.

[0065] Implementation 5: This implementation further limits the low-light image enhancement method described in Implementation 1. The low-frequency enhancement network model described in step 5 is specifically:

[0066] The low-frequency enhancement network model consists of two sub-block structures connected in series.

[0067] Among them, the first sub-block structure is composed of five layers connected in series, which is the fast Fourier convolution layer FFConv, followed by the normalization layer LN, the wavelet transform convolution layer WTConv, followed by the SimpleGate activation layer SG, and the last layer is the wavelet transform convolution layer;

[0068] The second sub-block structure consists of five sub-layers, namely the fast Fourier convolution layer, the layer normalization layer, the channel attention layer CA, the SimpleGate activation layer, and finally another wavelet transform-based convolution layer.

[0069] Embodiment 6: This embodiment further limits the low-light image enhancement method described in Embodiment 5, and the two serially connected sub-blocks perform data circulation and transmission through a residual connection.

[0070] Embodiment 7: This embodiment further limits the low-light image enhancement method described in Embodiment 1. The multi-dimensional loss function in step 6 is specifically:

[0071]

[0072] In the formula, is the overall loss function, is the mean square error loss, is the structural similarity loss, is the perceived loss, For causal analysis of losses, is the instance-level semantic loss, ,in, are the weights of different loss functions respectively.

[0073] Embodiment 8: This embodiment provides a low-light image enhancement system, the system comprising:

[0074] A training module, used to obtain a training set and a test set respectively, wherein the training set and the test set are both low-light images and their corresponding normal-light images;

[0075] A processing module, for obtaining low-light image low-frequency image subbands and high-frequency image subbands in different horizontal, vertical and diagonal directions through wavelet transformation of low-light image;

[0076] A feature extraction network model module, used for enhancing the high-frequency image subband based on the feature extraction network model, that is, obtaining enhanced low-frequency images and high-frequency images;

[0077] A collaborative enhancement module, for collaboratively enhancing the enhanced high-frequency image based on the high-frequency Mamba enhancement network model, and obtaining an enhanced stage image based on an inverse wavelet transform;

[0078] A hierarchical feature extraction module is used for obtaining the enhanced stage image by enhancing it based on a low-frequency enhancement network model, and obtaining the processed enhanced image by processing multiple hierarchical feature extraction structural units;

[0079] A training module is used to design a multi-dimensional loss function based on the enhanced image processed by the feature extraction module and the normal-light image corresponding to the low-light image in the training set, and use the loss function to train the feature extraction network model, the high-frequency Mamba enhancement network model, and the low-frequency network enhancement model respectively;

[0080] The enhancement module is used to form a sample pair of low-light images in the test set and their corresponding normal-light images, and execute operations from the processing module to the feature extraction module to obtain the final enhanced image.

[0081] Embodiment 9. This embodiment proposes a computer device, including a processor and a storage medium, wherein the memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes the method described in any one of embodiments 1 to 7.

[0082] Embodiment 10: This embodiment proposes a computer storage medium for storing a computer program. When the computer program is read by a computer, the computer executes the method described in any one of Embodiments 1 to 7.

[0083] Implementation 11. This implementation includes an example, which is used to explain the above implementation. Figure 1 To explain this embodiment,

[0084] Embodiment 1: A low-light image enhancement method described in this embodiment includes the following steps:

[0085] Step 1, respectively obtain a training set and a test set, where both the training set and the test set are low-light images and their corresponding normal-light images;

[0086] Step 2: Based on wavelet transform, the low-light image is transformed to obtain low-frequency image subbands and high-frequency image subbands in different horizontal, vertical and diagonal directions;

[0087] Step 3: The low-frequency image subband and the horizontal, vertical, and diagonal high-frequency image subbands of the low-light image are enhanced based on the feature extraction network model to obtain enhanced low-frequency images and high-frequency images;

[0088] Step 4: The high-frequency image of the low-light image is collaboratively enhanced based on the high-frequency Mamba enhancement network model, assisted by scanning with state space models in different directions, and the enhanced image is obtained based on inverse wavelet transform;

[0089] Step 5: Enhance the enhanced image based on the low-frequency enhancement network model, and obtain the final enhanced image through multiple levels of feature extraction structure;

[0090] Step 6: Design a multi-dimensional loss function based on the enhanced image and the normal-light image corresponding to the low-light image in the training set, and use the loss function to train the overall hierarchical network model;

[0091] Step 7: The low-light image in the test set and its corresponding normal-light image form a sample pair, and the operations of steps 2 to 5 are performed to obtain the final enhanced image.

[0092] Furthermore, the feature extraction network model is specifically as follows: first, the low-frequency image is based on wavelet transform convolution to extract features, while the high-frequency image subbands in three different directions, horizontal, vertical, and diagonal, are based on depth-separable convolution to extract features, and the low-frequency features are based on horizontal convolution, vertical convolution, and diagonal convolution, and the high-frequency details are further processed and supplemented through data connection.

[0093] Furthermore, the high-frequency Mamba network enhancement model, i.e., HFEB, is specifically as follows: first, high-frequency image information extracted in the horizontal, vertical, and diagonal directions is obtained based on the feature extraction network model, and the high-frequency Mamba network enhancement model mainly processes high-frequency image information in three different directions. The three types of high-frequency image information in the horizontal, vertical, and diagonal directions are processed based on the direction-aware Mamba structure respectively, and the direction-aware Mamba structure includes a 2D selective scanning module customized for the direction, i.e., 2D-SSM, thereby obtaining enhanced high-frequency information in different directions, and reconstructing the low-frequency information and the high-frequency information based on the inverse wavelet transform to obtain an enhanced stage image, which will be used as the input of the low-frequency enhancement network model in the subsequent processing.

[0094] Furthermore, the low-frequency enhancement network model, i.e., LFEB, is specifically as follows: the low-frequency enhancement network model is mainly composed of two sub-block structures connected in series. Among them, the first sub-block structure is composed of five layers connected in series, which is a fast Fourier convolution layer FFConv, followed by a layer normalization layer LN, a wavelet transform convolution layer WTConv, followed by a SimpleGate activation layer SG in series, and the last layer is a wavelet transform convolution layer. The second sub-block structure consists of five sub-layers, namely a fast Fourier convolution layer, a layer normalization layer, a channel attention layer CA, a SimpleGate activation layer, and finally another wavelet transform convolution layer. The two sub-block structures are connected through residual connections for data circulation and transmission.

[0095] Furthermore, in step S6, the network model loss function is specifically:

[0096]

[0097] In the formula, is the overall loss function, is the mean square error loss, is the structural similarity loss, is the perceived loss, For causal analysis of losses, is the instance-level semantic loss, ,in, are the weights of different loss functions respectively.

[0098] In the prior art, although there are a large number of algorithms based on deep learning, these methods generally have the problem of not being able to fully utilize image information and regional semantic information. Fourier methods are widely used to process frequency domain information, but they often perform poorly in terms of spatial locality under low light conditions. This shortcoming often leads to loss of details and excessive smoothing of images. Structural and contour details often cannot be well reconstructed, hindering effective subsequent processing and detail recognition. Wavelet transform can process information without losing details.

[0099] To solve the above problems, this embodiment designs a low-light image enhancement method based on causal wavelet transform, including the following steps:

[0100] Step 1: Obtain the data set, obtain the training set and test set respectively, the training set and test set are low-light images and its corresponding normal illumination image , the test set is low-light images and its corresponding normal illumination image .

[0101] Step 2: Based on wavelet transform, the low-light image obtains the low-frequency image subband and three high-frequency image subbands of horizontal, vertical and diagonal. The specific process is as follows:

[0102]

[0103] In the formula, is the input low-light image, is the low-frequency image subband based on wavelet transform, is the horizontal high-frequency image subband based on wavelet transform, is the vertical high-frequency image subband based on wavelet transform, is the diagonal high-frequency image subband based on wavelet transform.

[0104] According to the directional anisotropy of the obtained high-frequency sub-bands, a specific directional Mamba network structure is used for different scanning directions in the subsequent processing, so as to make fuller use of the high-frequency characteristics. High-frequency information reflects the contour and structure of the image, which can effectively help the algorithm improve the visual effect of the image. Compared with the traditional frequency domain feature extraction method, the wavelet transform-based image high-frequency feature extraction can not only maintain the original details of the image, but also has an advantage in the spatial locality of the image;

[0105] Step 3. The low-frequency image subband and the three high-frequency image subbands of the horizontal, vertical and diagonal images of the low-light image are jointly enhanced based on the feature extraction network model. Specifically, the high-frequency image is feature extracted based on the deep separable convolution DWSconv. Compared with the traditional convolution operation, the deep separable convolution saves computational effort while improving the scalability of the model and more fully exploits the high-frequency intrinsic features. The low-frequency image focuses on the wavelet transform convolution feature extraction. The wavelet transform convolution provides a larger receptive field and higher computational efficiency, and is suitable for extracting global feature information from low-frequency images. Since most of the information exists in the low-frequency components of the image, the feature extraction network model constructs horizontal convolution, vertical convolution and diagonal convolution based on convolution kernels in three different directions to extract the low-frequency components to supplement the details of the high-frequency components, thereby obtaining enhanced low-frequency and high-frequency images for subsequent further enhancement processing;

[0106] Step 4: The high-frequency image of the low-light image is collaboratively enhanced based on the high-frequency Mamba enhancement network model, assisted by scanning with state space models in different directions, and the enhanced image is obtained based on the inverse wavelet transform. The specific process is as follows: First, the high-frequency image information extracted in the horizontal, vertical and diagonal directions is obtained based on the feature extraction network model. The high-frequency Mamba network enhancement model mainly processes the high-frequency image information in three different directions. The three types of high-frequency image information in the horizontal, vertical and diagonal directions are processed based on the direction-aware Mamba structure. The direction-aware Mamba structure contains a 2D selective scanning module customized for the direction, namely 2D-SSM, to further enhance the modeling capability of high-frequency information.

[0107] The state space model SSM is applied to visual tasks, and many image restoration tasks now adopt the standard 2D-SSM architecture. Most existing image restoration tasks directly apply 2D-SSM to their framework without adapting to the frequency domain characteristics. The present invention designs three specialized high-frequency wavelet scanning structures: H-2D-SSM, V-2D-SSM and D-2D-SSM. These structures correspond to the scanning order of high-frequency wavelet components. The method converts two-dimensional high-frequency image features into a linear sequence by scanning in different directions: H-2D-SSM horizontal scanning from the upper left corner to the lower right corner. V-2D-SSM vertical scanning from the upper left corner to the lower right corner. D-2D-SSM diagonal scanning from the upper left corner to the lower right corner. Through the multi-path scanning strategy, the wavelet high-frequency information is better processed. Therefore, H-Mamba, V-Mamba, and D-Mamba network architectures are designed for horizontal high-frequency subbands, vertical high-frequency subbands, and diagonal high-frequency subbands, respectively, to obtain enhanced high-frequency information in different directions. Each Mamba network module first applies layer normalization LN, then applies a direction-specific visual state space module VSSM with residual connections, and then uses a gated feedforward network to enhance the efficiency of channel information flow. Based on the inverse wavelet transform, the low-frequency information and high-frequency information are reconstructed, and all sequences are merged to reconstruct the original 2D frame, specifically:

[0108]

[0109] In the formula, is the stage enhanced image obtained after processing, is the inverse wavelet transform algorithm, is the low-frequency image subband based on wavelet transform, is the horizontal high-frequency image subband based on wavelet transform, is the vertical high-frequency image subband based on wavelet transform, is the diagonal high-frequency image subband based on wavelet transform.

[0110] Step 5, enhance the enhanced image based on the low-frequency enhancement network model, and obtain the final enhanced image after multiple levels of feature extraction structure. The low-frequency enhancement network model is specifically as follows: the low-frequency enhancement network model is mainly composed of two sub-block structures connected in series. Among them, the first sub-block structure is connected in series by five sub-layers. The first is the fast Fourier convolution layer FFConv. FFConv enables the network to consider the global information of the image at the shallow level of the model, which plays a key role in image recovery. The subsequent layer is normalized LN. After Fourier convolution, considering the significant spatial variability of brightness information, the computational requirements of transformers, and the loss problem of dilated convolution, we further capture long-distance spatial correlation based on the wavelet transform convolution layer WTConv. The SimpleGate activation layer SG is connected in series later. Unlike the activation functions such as ReLU and Sigmoid commonly used in traditional methods, SG can minimize the loss of information during the activation process and maximize the retention of the original details of the image. The last layer is based on the wavelet transform convolution layer.

[0111] The second sub-block structure consists of five sub-layers. Since the processing of low-frequency components usually requires larger convolution kernels or global information, the two residual blocks use fast Fourier convolution in the initial layer to enhance the global features. The second sub-block structure is a fast Fourier convolution layer, a layer normalization layer, followed by a channel attention layer CA to further explore the relationship between channels, a SimpleGate activation layer, and finally another wavelet transform-based convolution layer. The two sub-block structures are connected through residual connections for data flow and transmission. After passing through the low-frequency enhancement network model, the low-frequency information is selectively enhanced to improve the image content and brightness. A hierarchical image feature recovery unit HBRB is composed of a feature extraction network model, a high-frequency Mamba network enhancement model, and a low-frequency enhancement network model. In the network architecture, multiple hierarchical image feature recovery units are interspersed in the architecture based on the U-Net basic architecture to more effectively reconstruct the overall brightness of the image and local micro details;

[0112] Step 6: Based on the enhanced image And the normal light images corresponding to the low light images in the training set The loss function is designed and used to train the feature extraction network model, the high-frequency Mamba enhanced network model and the low-frequency network enhanced model through the back propagation algorithm to continuously optimize the parameters of the network model.

[0113] To calculate the difference between pixels, use Loss function, mean square error loss function, and structural similarity loss are introduced to maintain the structural consistency between the enhanced image and the reference image. In order to further enhance the semantic information of the brightness enhancement results, many methods have effectively utilized the visual language knowledge model CLIP. However, these methods usually only maintain the global semantic consistency of the image, fail to ensure the local consistency of the image, resulting in imperfect local details of the restored image. On this basis, the invention designs a semantically guided instance-level CLIP loss to better enhance semantic consistency. In addition, the invention models different anomalies, such as illumination attenuation and color anomalies separately and considers them in the progress loss. In this way, the model can not only handle a single abnormal situation, but also handle multiple types of anomalies, thereby enhancing the performance of the model in complex scenes and improving the generalization ability of the model. Finally, a perceptual loss based on the VGG network for feature representation is introduced to ensure high-level semantic details and visual fidelity. The specific loss function is defined as:

[0114]

[0115] In the formula, is the overall loss function, is the mean square error loss, is the structural similarity loss, is the perceptual loss based on the VGG network architecture, For causal analysis of losses, is the instance-level semantic loss. Weight Set to , to better balance the contributions between the various losses;

[0116] Step S7: Set a judgment condition in the algorithm to determine whether a predetermined number of training iterations have been completed. When all training iterations are completed, the overall network model that has been trained and optimized will be obtained. and its corresponding normal illumination image The feature extraction network model, high-frequency Mamba enhancement network model and low-frequency enhancement network model will input low-light images. and its corresponding normal illumination image Processing is performed to output the final enhanced image.

[0117] In summary, this implementation designs a low-light image enhancement network model based on causal wavelet transform, utilizes the unique properties of the frequency domain components of wavelet transform to enhance low-light images, and effectively and collaboratively processes low-frequency and high-frequency information. The feature extraction model uses low-frequency information to guide the analysis and processing of high-frequency information. The low-frequency enhancement network model captures the global and local contextual information required for image enhancement by combining Fourier convolution with wavelet transform convolution to enhance brightness and details. At the same time, the high-frequency Mamba enhancement network model adopts a state space model structure with different directions to enhance high-frequency modeling for wavelet characteristics and improve performance in noisy environments. It solves existing problems such as uneven brightness, loss of details, and color distortion in low-light image enhancement, providing a powerful solution for improving image clarity.

[0118] As far as the tenth embodiment of this specification is concerned, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use in an instruction execution system, device or equipment or in combination with these instruction execution systems, devices or equipment. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion (electronic device) with one or N wirings, a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting or processing in other suitable ways as necessary, and then stored in a computer memory. It should be understood that the various parts of the present invention can be implemented in hardware, software, firmware or a combination thereof. In the above embodiment, N steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0119] Those skilled in the art will appreciate that the above are only preferred embodiments of the present invention, and the various embodiments of the present disclosure and / or the features described in the claims may be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. It is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art may still modify the technical solutions described in the aforementioned embodiments, or perform equivalent substitutions on some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

[0120] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A low-light image enhancement method, characterized in that: The low-light image enhancement method comprises the following steps: Step 1: respectively obtain a training set and a test set, wherein both the training set and the test set are low-light images and their corresponding normal-light images; Step 2: The low-light image is transformed by wavelet transform to obtain low-frequency image subbands of the low-light image and high-frequency image subbands in three different directions: horizontal, vertical, and diagonal. Step 3, the high-frequency image subband and the low-frequency image subband are both enhanced based on the feature extraction network model, that is, a preliminarily enhanced low-frequency image and a preliminarily enhanced high-frequency image are obtained respectively; Step 4: the high-frequency image after preliminary enhancement is input into the high-frequency Mamba enhancement network model for collaborative enhancement to obtain the final enhanced high-frequency image, and is combined with the low-frequency image subband through inverse wavelet transform to obtain a collaboratively enhanced stage image; Step 5, the collaboratively enhanced stage image is input into a low-frequency enhancement network model for enhancement processing, and processed by multiple hierarchical feature extraction structural units to obtain a processed enhanced image; Step 6: Based on the processed enhanced image obtained in step 5 and the normal-light image corresponding to the low-light image in the training set, a multi-dimensional loss function is designed, and the feature extraction network model, the high-frequency Mamba enhancement network model, and the low-frequency network enhancement model are trained respectively using the loss function; Step 7: The low-light image in the test set and the corresponding normal-light image are combined into a sample pair, and the operations from step 2 to step 5 are performed to obtain the final enhanced image; The low-frequency enhancement network model described in step 5 is specifically: The low-frequency enhancement network model consists of two sub-block structures connected in series. Among them, the first sub-block structure is composed of five layers connected in series, which is the fast Fourier convolution layer FFConv, followed by the normalization layer LN, the wavelet transform convolution layer WTConv, followed by the SimpleGate activation layer SG, and the last layer is the wavelet transform convolution layer; The second sub-block structure consists of five sub-layers, namely the fast Fourier convolution layer, the layer normalization layer, the channel attention layer CA, the SimpleGate activation layer, and finally another wavelet transform-based convolution layer.

2. The low-light image enhancement method according to claim 1, characterized in that: The low-frequency image subband and the three high-frequency image subbands of horizontal, vertical and diagonal in the low-light image in step 3 are enhanced based on the feature extraction network model by: The low-frequency image features are extracted based on wavelet transform convolution, and the horizontal, vertical and diagonal high-frequency image sub-bands are extracted based on depthwise separable convolution.

3. The low-light image enhancement method according to claim 1, characterized in that: The method of inputting the high-frequency image after preliminary enhancement described in step 4 into the high-frequency Mamba enhancement network model for collaborative enhancement is: The three high-frequency image information in the horizontal, vertical and diagonal directions are processed respectively based on the direction-aware Mamba structure. The direction-aware Mamba structure contains a 2D selective scanning module, that is, the enhanced high-frequency information in different directions is obtained. The low-frequency image and the high-frequency image are reconstructed based on the inverse wavelet transform to obtain the enhanced image, which is used as the input of the low-frequency enhancement network model.

4. The low-light image enhancement method according to claim 3, characterized in that: Based on the inverse wavelet transform, the low-frequency information and high-frequency information are reconstructed, and all sequences are merged to reconstruct the original 2D framework, specifically: In the formula, is the stage enhanced image obtained after processing, is the inverse wavelet transform algorithm, is the low-frequency image subband based on wavelet transform, is the horizontal high-frequency image subband based on wavelet transform, is the vertical high-frequency image subband based on wavelet transform, is the diagonal high-frequency image subband based on wavelet transform.

5. The low-light image enhancement method according to claim 1, characterized in that: The two serially connected sub-blocks perform data circulation and transmission via a residual connection.

6. The low-light image enhancement method according to claim 1, characterized in that: The multi-dimensional loss function in step 6 is specifically: In the formula, is the overall loss function, is the mean square error loss, is the structural similarity loss, is the perceived loss, For causal analysis of losses, is the instance-level semantic loss.

7. A low-light image enhancement system, characterized in that: The system comprises: A training module, used to obtain a training set and a test set respectively, wherein the training set and the test set are both low-light images and their corresponding normal-light images; A processing module, for obtaining low-light image low-frequency image subbands and high-frequency image subbands in different horizontal, vertical and diagonal directions through wavelet transformation of low-light image; A feature extraction network model module, which is used for enhancing the high-frequency image subband and the low-frequency image subband based on the feature extraction network model, that is, obtaining a preliminarily enhanced low-frequency image and a preliminarily enhanced high-frequency image respectively; A collaborative enhancement module is used to input the high-frequency image after preliminary enhancement into the high-frequency Mamba enhancement network model for collaborative enhancement to obtain a final enhanced high-frequency image, and combine it with the low-frequency image subband through inverse wavelet transform to obtain a collaboratively enhanced stage image; A hierarchical feature extraction module, which is used to input the collaboratively enhanced stage image into a low-frequency enhancement network model for enhancement processing, and obtain a processed enhanced image through processing by multiple hierarchical feature extraction structural units; A training module is used to design a multi-dimensional loss function based on the enhanced image processed by the feature extraction module and the normal-light image corresponding to the low-light image in the training set, and use the loss function to train the feature extraction network model, the high-frequency Mamba enhancement network model, and the low-frequency network enhancement model respectively; An enhancement module is used to form a sample pair of low-light images in the test set and their corresponding normal-light images, and execute operations from the processing module to the feature extraction module to obtain the final enhanced image; The low-frequency enhancement network model described in the hierarchical feature extraction module is specifically: The low-frequency enhancement network model consists of two sub-block structures connected in series. Among them, the first sub-block structure is composed of five layers connected in series, which is the fast Fourier convolution layer FFConv, followed by the normalization layer LN, the wavelet transform convolution layer WTConv, followed by the SimpleGate activation layer SG, and the last layer is the wavelet transform convolution layer; The second sub-block structure consists of five sub-layers, namely the fast Fourier convolution layer, the layer normalization layer, the channel attention layer CA, the SimpleGate activation layer, and finally another wavelet transform-based convolution layer.

8. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images