A low-light image enhancement method, device and medium based on wavelet transform

Abstract

The application discloses a low-light image enhancement method and device based on wavelet transform and a medium, and relates to the technical field of image processing.The method comprises the following steps: acquiring a to-be-processed image; performing preliminary feature extraction on the to-be-processed image by using a first convolution module to obtain to-be-processed image features; performing high-order curve iterative adjustment on a brightness channel of the to-be-processed image by using an illumination adjustment module to obtain a multi-channel illumination map; performing multi-branch multi-scale feature extraction and fusion on the to-be-processed image features and the multi-channel illumination map by using a structure prediction module to obtain a structure prediction map; performing bidirectional wavelet transform decomposition on the structure prediction map by using a detail repair module, weighting a spatial and channel attention mechanism, and then performing inverse bidirectional wavelet transform reconstruction to obtain a detail repair map; and performing performance activation and dimension reduction on the detail repair map by using a dimension reduction activation module to obtain an enhanced image of the to-be-processed image.The application realizes effective enhancement of a low-light image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, device and medium for low-light image enhancement based on wavelet transform. Background Technology

[0002] Images are witnesses to time, carrying rich history, culture, and emotions, providing invaluable visual memories and intellectual exchange for the development and progress of human society. However, due to insufficient natural light and limitations of shooting equipment, low-light images are often produced. These images often suffer from poor visibility, low contrast, and noise interference, which not only affect visual experience but also pose challenges to advanced visual tasks such as nighttime facial recognition.

[0003] Currently, low-light image enhancement is a complex and delicate process requiring specialized techniques and systematic methods. Accurately adjusting the illumination of low-light images, predicting structural information, and restoring lost details are the three key steps in low-light image enhancement. Firstly, existing deep learning methods for adjusting the illumination of low-light images typically employ a strategy of simply decomposing the image into the RGB color space. However, because visual attributes such as brightness and chromaticity are intertwined in the RGB color space, consistency in brightness and color can only be ensured when the RGB channel values ​​of the enhanced image are sufficiently close to the corresponding channel values ​​of the reference image. This matching is challenging, thus limiting the effectiveness of illumination adjustment and often resulting in problems such as overexposure, underexposure, and color cast.

[0004] Secondly, for predicting structural information in images, the initial convolutional layers in deep convolutional neural networks can identify simple features such as edges and textures. However, as the network depth increases, subsequent convolutional layers can learn more complex features, such as shapes and object parts. Therefore, existing deep learning methods extract and capture structural information from images by stacking convolutional layers. However, as the number of network layers increases, gradients may encounter vanishing or exploding problems during backpropagation, which may affect the effective capture of structural information by deep convolutional neural networks. Furthermore, the multiple stacking of convolutional layers can lead to some features being extracted and learned multiple times, resulting in feature redundancy.

[0005] Finally, noise amplification is a common challenge for image detail restoration. To address image noise, existing deep learning methods add denoising modules to the network. The essence of denoising is smoothing the high-frequency components of the image, which are precisely where image details are contained. Therefore, this approach often leads to the loss of image details, resulting in blurred edges and distortion in the enhanced image. In conclusion, existing deep learning methods perform poorly in low-light image processing. Summary of the Invention

[0006] The purpose of this application is to provide a low-light image enhancement method, device, and medium based on wavelet transform, which achieves effective enhancement of low-light images.

[0007] To achieve the above objectives, the embodiments of this application provide the following solutions:

[0008] In a first aspect, this application provides a low-light image enhancement method based on wavelet transform, the low-light image enhancement method based on wavelet transform comprising:

[0009] Acquire the image to be processed; the image to be processed includes at least images with an average brightness of less than 80 candela per square meter;

[0010] A model for image enhancement is determined; the model includes: a first convolutional module, an illumination adjustment module, a structure prediction module, a detail restoration module, and a dimension reduction activation module.

[0011] The first convolution module is used to perform preliminary feature extraction on the image to be processed to obtain the image features.

[0012] The brightness channel of the image to be processed is iteratively adjusted using a high-order curve using the lighting adjustment module to obtain a multi-channel lighting map;

[0013] The structure prediction module is used to perform multi-branch, multi-scale feature extraction and fusion on the features of the image to be processed and the multi-channel illumination map to obtain a structure prediction map.

[0014] After performing bidirectional wavelet transform decomposition, spatial and channel attention mechanism weighting, and inverse bidirectional wavelet transform reconstruction on the structure prediction map using the detail restoration module, the detail restoration map is obtained.

[0015] The dimensionality reduction and activation module is used to perform performance activation and dimensionality reduction on the detail restoration image to obtain an enhanced image of the image to be processed.

[0016] Optionally, the illumination adjustment module includes: a multilayer perceptron module, a higher-order curve module, and a second convolution module; the illumination adjustment module is used to iteratively adjust the brightness channel of the image to be processed using higher-order curves to obtain a multi-channel illumination map, specifically including:

[0017] Extract a histogram representing global information of the image to be processed from the brightness channel of the image to be processed;

[0018] The histogram is input into the multilayer perceptron module to obtain a set of coefficients for iterative adjustment of higher-order curves;

[0019] Based on the coefficient set, the histogram is iteratively adjusted using the higher-order curve module to obtain a single-channel illumination map;

[0020] The single-channel illumination map is enhanced using the second convolution module to obtain a multi-channel illumination map.

[0021] Optionally, the structure prediction module includes multiple perceptual feature extraction branches at different scales and a third convolutional module; the structure prediction module is used to perform multi-branch, multi-scale feature extraction and fusion on the features of the image to be processed and the multi-channel illumination map to obtain a structure prediction map, specifically including:

[0022] The image features to be processed and the multi-channel illumination map are concatenated by channel and then input into multiple perceptual feature extraction branches of different scales in parallel to obtain the scale perceptual features on each perceptual feature extraction branch.

[0023] After the scale-aware features on all the perceptual feature extraction branches are concatenated by channel, they are input into the third convolution module for feature fusion to obtain the structure prediction map.

[0024] Optionally, the detail restoration module includes: a bidirectional wavelet transform module, an attention weighting module, and a scale reconstruction module; after performing bidirectional wavelet transform decomposition, spatial and channel attention mechanism weighting, and inverse bidirectional wavelet transform reconstruction on the structure prediction map using the detail restoration module, a detail restoration map is obtained, specifically including:

[0025] The structure prediction map is decomposed using the bidirectional wavelet transform module to obtain a wavelet feature map.

[0026] The attention weighting module is used to adjust the weights of the wavelet feature map using spatial attention and channel attention mechanisms to obtain spatial and channel fusion features;

[0027] The wavelet feature map is concatenated with the spatial channel fusion feature to obtain the attention fusion feature;

[0028] The attention fusion features are subjected to inverse bidirectional wavelet transform using the scale reconstruction module to obtain the reconstructed feature map;

[0029] The reconstructed feature map and the structure prediction map are stitched together to obtain the detail restoration map.

[0030] Optionally, the formula for calculating the coefficient set is:

[0031]

[0032] In the formula, {α0, α1,…, α t} is the coefficient set, α t Let G(·) be the t-th coefficient in the coefficient set, and let G(·) be a multilayer perceptron. This is the brightness channel, where μ is the desired brightness value. This indicates the extraction of histogram information.

[0033] Optionally, the iterative formula in the higher-order curve module is:

[0034]

[0035] In the formula, α k For the k-th coefficient in the coefficient set, For the brightness channel of the k-th iteration, Let t be the brightness channel in the (k+1)th iteration, and t be the maximum number of iterations.

[0036] Optionally, each perceptual feature extraction branch is equipped with a k×k convolutional kernel module and a branch weighting module; the calculation formula for the scale-aware features is:

[0037]

[0038] In the formula, SAFE(·,·,·) is the scale-aware feature extraction function. For multi-channel lighting diagrams, I f Let k be the image features to be processed, k be the scale of the convolution kernel, R(·) be the branch weighting module, and F be the weighting factor of the convolution kernel. k (·) denotes a convolution operation with a k×k convolution kernel. This indicates a layer-by-layer accumulation operation.

[0039] Optionally, the formula for calculating the structural prediction map is:

[0040]

[0041] In the formula, I spmThis is the structure prediction map. Fusion(·) is the third convolutional module, n is the number of perceptual feature extraction branches, ∑ represents the concatenation operation of perceptual features at different scales, and SAFE(·,·,·) is the scale-aware feature extraction function. For multi-channel lighting diagrams, I f denoted as the image features to be processed, and k represents the scale of the convolution kernel.

[0042] Secondly, this application also provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the wavelet transform-based low-light image enhancement method described above.

[0043] Thirdly, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned low-light image enhancement method based on wavelet transform.

[0044] Based on the specific embodiments provided in this application, the following technical effects are disclosed in this application:

[0045] Because the V channel (luminance channel) of the HSV color space can effectively represent the brightness information of an image, the illumination adjustment module of this application extracts the luminance channel from the image to be processed to improve the overall brightness of the image, and uses a high-order curve iteration method to adjust the luminance channel to more accurately capture the complex nonlinear relationships in the image to be processed. Secondly, the structure prediction module of this application adopts a multi-branch, multi-scale feature extraction and fusion method, and uses convolution kernels of different sizes to effectively capture the global context information and local detail information of the image to be processed, ensuring the structural integrity of the image to be processed. Finally, the detail restoration module of this application uses bidirectional wavelet transform to remove noise in the image to be processed and enhances image details and contours. At the same time, it also uses spatial and channel attention mechanisms to enhance specific high-frequency components of the image to be processed and suppresses specific frequency wavelet coefficients to reduce the impact of noise. Through the above three improvements, this application achieves effective enhancement of low-light images. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart illustrating the low-light image enhancement method based on wavelet transform provided in this application embodiment;

[0048] Figure 2 This is a schematic diagram of the structure of the image enhancement model provided in the embodiments of this application;

[0049] Figure 3 This is a schematic diagram of the structure of the lighting adjustment module provided in an embodiment of this application;

[0050] Figure 4 This is a schematic diagram of the structure prediction module provided in an embodiment of this application;

[0051] Figure 5 This is a schematic diagram of the structure of the detail repair module provided in the embodiments of this application;

[0052] Figure 6 This is a schematic diagram of the structure of the first convolution module provided in an embodiment of this application;

[0053] Figure 7 This is a schematic diagram of the structure of the dimension reduction activation module provided in the embodiments of this application;

[0054] Figure 8 This is a schematic diagram of the spatial attention mechanism module provided in an embodiment of this application;

[0055] Figure 9 This is a schematic diagram of the channel attention mechanism module provided in an embodiment of this application;

[0056] Figure 10 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0057] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0058] The purpose of this application is to provide a low-light image enhancement method, device, and medium based on wavelet transform, which achieves effective enhancement of low-light images.

[0059] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0060] Example 1

[0061] like Figure 1 As shown, this embodiment provides a low-light image enhancement method based on wavelet transform, specifically including:

[0062] Step S1: Obtain the image to be processed; the image to be processed must include images with an average brightness of less than 80 cd / m². 2 The image.

[0063] In this embodiment, the image to be processed mainly refers to a low-light image, which is generally defined as an image with an average brightness below 80 cd / m². 2 Furthermore, over 80% of the pixel brightness values ​​in the brightness histogram are concentrated between 0 and 160 cd / m². 2 The image within the specified range. In this embodiment, the image to be processed is represented as... H and W represent the height and width of the image to be processed, respectively, and 3 represents the number of channels in the image to be processed.

[0064] Step S2: Construct an image enhancement model.

[0065] In this embodiment, Figure 2 The image enhancement model shown includes: a first convolutional module, an illumination adjustment module, a structure prediction module, a detail restoration module, and a dimension reduction activation module.

[0066] like Figure 3 As shown, the lighting adjustment module includes: a multilayer perceptron module, a higher-order curve module, and a second convolutional module. The multilayer perceptron module consists of five 3×3 convolutional layers (Conv); the second convolutional module consists of two 3×3 Conv layers.

[0067] like Figure 4 As shown, the structure prediction module includes three perceptual feature extraction branches at different scales and a third convolutional module. Each perceptual feature extraction branch has a k×k convolutional kernel module and a branch weighting module; the Conv scales in the k×k convolutional kernel modules are 3×3, 5×5, 7×7, and 9×9, respectively; the branch weighting module consists of a sequentially connected activation layer (PReLU), a dropout layer, and two 1×1 Conv layers; the third convolutional module consists of two 3×3 Conv layers.

[0068] like Figure 5 As shown, the detail restoration module includes: a bidirectional wavelet transform module (BWT), an attention weighting module, and a scale reconstruction module. The attention weighting module consists of two sequentially connected 3×3 Conv layers, a spatial attention mechanism module (SAM), a channel attention mechanism module (CAM), and a 1×1 Conv layer; the scale reconstruction module consists of a sequentially connected inverse bidirectional wavelet transform module (IBWT), a 3×3 Conv layer, a PReLU layer, and a 1×1 Conv layer.

[0069] also, Figure 6 The first convolutional module shown consists of multiple 3×3 Convs; Figure 7The dimensionality reduction activation module shown consists of a 1×1 Conv, Dropout, PReLU, Dropout, two layers of 1×1 Conv and 3×3 Conv connected in sequence.

[0070] Step S3: Use the first convolution module to perform preliminary feature extraction on the image to be processed to obtain the image features.

[0071] In this embodiment, the image to be processed I low The image undergoes initial feature extraction via multiple 3×3 Convolutional layers in the first convolutional module, yielding the features of the image to be processed. In the first convolutional module, the multiple 3×3Convs are not connected sequentially. To prevent gradient vanishing, residual connections are used in the middle of the multiple 3×3Convs.

[0072] Step S4: Use the illumination adjustment module to perform high-order curve iterative adjustment on the brightness channel of the image to be processed to obtain a multi-channel illumination map.

[0073] In this embodiment, step S4 specifically includes:

[0074] Extract histograms representing global information of the image from the brightness channel of the image to be processed; input the histograms into the multilayer perceptron module to obtain a set of coefficients for iterative adjustment of higher-order curves; based on the set of coefficients, iteratively adjust the histograms using the higher-order curve module to obtain a single-channel illumination map; use the second convolution module to perform channel boosting on the single-channel illumination map to obtain a multi-channel illumination map.

[0075] Among them, for the acquired image to be processed Calculate the maximum value of the RGB three channels at each pixel location to obtain the brightness channel (V channel) of the image to be processed. Then, from Extract histograms that can represent global information of the image to be processed. Histogram obtains the coefficient set {α0, α1, ..., α...} for iterative adjustment of higher-order curves through a 5-layer multilayer perceptron. t}

[0076] Furthermore, the formula for calculating this coefficient set is as follows:

[0077]

[0078] In the formula, α t Let G(·) be the t-th coefficient in the coefficient set, G(·) be the multilayer perceptron, and μ be the desired luminance value. This indicates the extraction of histogram information.

[0079] Furthermore, the iterative formula in the higher-order curve module is:

[0080]

[0081] In the formula, For the brightness channel of the k-th iteration (when k=0, for ), Let be the luminance channel for the (k+1)th iteration, and t be the maximum number of iterations. After adjusting the above iterative formula, a single-channel illumination map will be obtained. In order for the image enhancement model to better capture the data features of the adjusted V channel, The number of channels is increased from 1 to 16 by using two 3×3 Conv layers in the second convolutional module to obtain a multi-channel illumination map.

[0082] Step S5: Use the structure prediction module to perform multi-branch, multi-scale feature extraction and fusion of the image features to be processed and the multi-channel illumination map to obtain the structure prediction map.

[0083] In this embodiment, step S5 specifically includes:

[0084] The image features to be processed and the multi-channel illumination map are concatenated by channel and then input into multiple perceptual feature extraction branches of different scales in parallel to obtain scale-aware features on each perceptual feature extraction branch. After concatenating the scale-aware features on all perceptual feature extraction branches by channel, the concatenation is input into the third convolution module for feature fusion to obtain the structure prediction map.

[0085] In order to extract and fuse structural information at different scales, the image features I to be processed f First, we will work with the multi-channel lighting diagram. The data is concatenated by channel, and then scale-aware features are extracted in parallel through branches containing convolutional kernels of different scales and branch weighting modules. The formula for calculating scale-aware features is as follows:

[0086]

[0087] In the formula, SAFE(·,·,·) is the scale-aware feature extraction function, k is the scale of the convolution kernel, R(·) is the branch weighting module, and F k (·) denotes a convolution operation with a k×k convolution kernel. This indicates a layer-by-layer accumulation operation.

[0088] The scale-aware features extracted from different branches are concatenated by channel, and then integrated by the two-layer 3×3 Conv of the third convolutional module to obtain the structure prediction map. The formula for calculating the structural prediction diagram is:

[0089]

[0090] In the formula, Fusion(·) is the third convolutional module, n is the number of branches for perceptual feature extraction, and ∑ represents the splicing operation of perceptual features at different scales.

[0091] Step S6: After performing bidirectional wavelet transform decomposition, spatial and channel attention mechanism weighting, and inverse bidirectional wavelet transform reconstruction on the structure prediction map using the detail restoration module, the detail restoration map is obtained.

[0092] In this embodiment, step S6 specifically includes:

[0093] The structure prediction map is decomposed using a bidirectional wavelet transform module to obtain wavelet feature maps. The wavelet feature maps are then weighted using spatial and channel attention mechanisms using an attention weighting module to obtain spatial and channel fusion features. The wavelet feature maps and spatial and channel fusion features are then concatenated to obtain attention fusion features. The attention fusion features are then subjected to inverse bidirectional wavelet transform using a scale reconstruction module to obtain reconstructed feature maps. Finally, the reconstructed feature maps and structure prediction maps are concatenated to obtain detail restoration maps.

[0094] In order to restore details in the image to be processed and suppress noise amplified during the enhancement process, the detail restoration module will modify the structure prediction image I. spm Further processing is then performed. First, the detail repair module uses a two-way wavelet transform module to refine the structure prediction map. Decomposition yields wavelet feature maps. The expression for the entire process is:

[0095] I bwt =B(I spm )

[0096] In the formula, B(·) represents the wavelet decomposition function of the bidirectional wavelet transform module. To enhance detail and suppress noise, wavelet feature map I... but First, preliminary feature extraction is performed using a 2-layer 3×3 Conv in the attention weighting module, and then the results are sent to the spatial attention mechanism module (see [link]). Figure 8 ) and the channel attention mechanism module (see Figure 9 The spatial attention mechanism module helps image enhancement models locate target objects, improves the network's perception of subtle features, and suppresses noise and interference in wavelet feature maps. The channel attention mechanism module helps image enhancement models select the most representative feature channels, thereby improving the network's ability to express input data.

[0097] To further enhance the representational power of the detail enhancement module, 1×1 Conv is used to efficiently fuse features from the two attention mechanism modules. To facilitate gradient propagation within the network and accelerate convergence, the detail enhancement module also introduces multiple residual connections. For example, the spatial channel fusion features obtained from 1×1 Conv fusion are combined with wavelet feature map I. bwt The residuals are concatenated to obtain the attention fusion feature I. am The entire process can be simplified as follows:

[0098]

[0099] In the formula, SA(·) and CA(·) represent the functions of the spatial and channel attention mechanisms, respectively. Subsequently, the attention fusion feature I is applied using an inverse bidirectional wavelet transform module. am Reconstructing back to the original size, the features reconstructed to the original size are sequentially processed through 3×3 Conv, PReLU, and 1×1 Conv, and then combined with the structure prediction diagram I. spm The residuals are stitched together to obtain the final output detail-restored image. The entire process can be simplified as follows:

[0100]

[0101] In the formula, IB(·) represents the feature reconstruction function of the inverse bidirectional wavelet transform module.

[0102] Step S7: Use the dimensionality reduction activation module to perform performance activation and dimensionality reduction on the detail restoration image to obtain the enhanced image of the image to be processed.

[0103] The combination of 1×1 Conv and PReLU introduces non-linear transformations into the network, thereby enhancing its expressive power. Dropout randomly "drops" a portion of neurons during training, resulting in a different network structure in each training iteration—essentially training multiple different sub-networks—which improves the generalization ability of the image enhancement model. Therefore, this embodiment introduces multiple 1×1 Conv, PReLU, and Dropout loss layers in the dimensionality reduction activation module to improve the performance of the image enhancement model. The 3×3 Conv layer in the dimensionality reduction activation module reduces the feature dimension to 3 channels, facilitating subsequent conversion to an RGB three-channel image.

[0104] After the above steps S1 to S7, the image enhancement model will finally output the enhanced image of the image to be processed.

[0105] Example 2

[0106] This embodiment provides a training process for an image enhancement model, specifically including:

[0107] The first step is to obtain a dataset containing 500 low / normal light image pairs (a total of 1000 images). 485 low / normal light image pairs in the dataset are used as the training set, and 15 low / normal light image pairs are used as the test set.

[0108] The second step involved training and evaluating the image augmentation model on a dataset with an image resolution of 400×600 and a batch size of 16. Using the Adam optimizer, with both the learning rate and weight decay set to 0.0001, the model was trained for 15,000 epochs. The weight file was saved every 10 epochs, and Wandb was used to visualize the various numerical values ​​during training.

[0109] The low-light images from 485 low / normal light image pairs in the training set are input into the image enhancement model to determine their corresponding enhanced images. Based on the normal light images from the 485 low / normal light image pairs, the image enhancement model is trained with the goal of minimizing the total loss function.

[0110] Specifically, the total loss function of the image enhancement model is:

[0111]

[0112] In the formula, This represents the total image loss value. The image color loss value. This represents the image brightness loss value. This represents the image structure loss value.

[0113] Among them, image color loss value The calculation formula is as follows:

[0114]

[0115] In the formula, I out An augmented image of the input image to the model, I high This is a normally lit image paired with the model input image, where m represents the image height, n represents the image width, and (i,j) represents the pixel coordinates in the image. <·,·> are used to calculate the cosine similarity between two vectors. This formula utilizes... Cumulative calculation I out and I high The cosine similarity is used to obtain the overall similarity value between the two images, and then 1 is subtracted from the overall similarity value to obtain the dissimilarity value between the two images as the image color loss value.

[0116] Furthermore, image brightness loss value The calculation formula is as follows:

[0117]

[0118] In the formula, c is the color channel, b(·) represents an image patch centered at a certain pixel, and b(·) - min b(·) is used to eliminate the influence of constants. This formula first utilizes the enhanced image I... out The pixel value matrix of the image block centered at pixel (i, j) minus the minimum value in that image block, I high Perform the same operation; then calculate the cosine similarity between the two image blocks, and sum the cosine similarities of each image block and each channel to obtain the overall similarity value; finally, subtract the overall similarity value from 1 to obtain the dissimilarity value between the two images as the image brightness loss function value.

[0119] Image structure loss function value The calculation formula is as follows:

[0120]

[0121] In the formula, This is used to calculate the structural differential matrix. The structure in an image is often reflected in the differences between pixels. If there is a significant difference between two pixels, meaning that their regions have certain structural properties, then the structural differential matrix of the image can be obtained by translation and subtraction. Specifically, shifting the entire image to the right by one pixel and then subtracting the original image from the shifted image yields the horizontal structural differential matrix. The same method applies to the calculation of the vertical structural differential matrix.

[0122] The third step is to effectively test the trained image enhancement model using 15 low / normal light images in the test set.

[0123] Fourthly, to improve the quality and usability of the final image, it is also necessary to further refine the I... out Further enhancements were made to four aspects: color, contrast, brightness, and sharpness, to obtain clearer and more detailed images.

[0124] All enhancement processes (color, contrast, brightness, and sharpness) can be described by the following formula:

[0125] EnhancedImage=α·OriginImage+(1-α)·BlendedImage

[0126] In the formula, OriginImage is the original image to be enhanced, and BlendedImage is the blended image used for blending. This blended image maintains the same size and dimensions as OriginImage. Specifically: for color enhancement, the blended image is a grayscale image of the original image; for contrast enhancement, the blended image is a grayscale image where all values ​​are the average brightness value of the original image; for brightness enhancement, the blended image is a completely black image; and for sharpness enhancement, the blended image is a blurred image processed by a smoothing filter. α is a blending factor. By adjusting the value of α, color, contrast, brightness, and sharpness enhancements can be performed on the original image using linear interpolation, thereby controlling the corresponding attributes of the enhanced image.

[0127] In summary, the low-light image enhancement method based on wavelet transform provided in this application includes the process of iteratively adjusting the brightness channel of the low-light image using higher-order curves, extracting the structural information of the low-light image using multi-branch parallel convolution, and restoring high-quality details of the low-light image using wavelet transform.

[0128] First, the V channel of the HSV color space has been proven to effectively represent the brightness information of low-light images. Therefore, this application extracts the V channel from low-light images specifically for enhancing their overall brightness. Employing a high-order curve iterative adjustment method more accurately captures the complex nonlinear relationships in low-light images, allowing the image enhancement model to finely adjust the brightness and contrast. Therefore, using a high-order curve iterative adjustment method for the V channel ensures that the enhanced low-light image is similar in brightness to the reference image, achieving a more natural effect.

[0129] Secondly, this application proposes a structure prediction module based on multi-branch parallel convolution technology. Multi-branch parallel convolution is a deep learning network structure that contains multiple branches, each with its own convolution operation. In the structure prediction module, different branches employ convolution kernels of different sizes: large convolution kernel branches have a wider receptive field, which helps capture the global contextual structure information of low-light images; small convolution kernel branches are beneficial for finely parsing the local detail information of low-light images. From fine-grained small convolution kernels to coarse-grained large convolution kernels, each branch gradually acquires structural information of low-light images at different scales, thereby enabling the image enhancement model to more comprehensively and deeply understand and maintain the structural integrity of low-light images.

[0130] Finally, this application also proposes a detail restoration module combining wavelet transform and attention mechanisms. Wavelet transform can convert low-light images into a series of multi-scale frequency domain representations, while noise usually manifests as outliers in high-frequency or low-frequency subbands. By implementing thresholding or other denoising methods in the wavelet domain, these outliers can be effectively removed, achieving image signal denoising. On the other hand, wavelet transform can also decompose low-light images into multiple frequency-level detail and contour structure layers. By enhancing or suppressing wavelet coefficients at specific frequencies, details and contours can be enhanced. Therefore, in the detail restoration module, this application first uses bidirectional wavelet transform to decompose the image into low-frequency components containing global structure and contours, and high-frequency components rich in detail texture and edge information. To specifically enhance specific high-frequency components of the image to improve the overall detail clarity, and to suppress specific frequency wavelet coefficients to mitigate the impact of noise, spatial attention and channel attention mechanisms are also introduced in the detail restoration module.

[0131] Regarding loss functions, previous deep learning methods tended to use absolute loss functions like L1 and L2, which require rigorous calculation of the difference between predicted and true values. However, ill-posed problems like low-light image enhancement, where there is no absolute mapping between low-light and normal-light images, limit the model's generalization ability. Therefore, this application uses relative loss functions. The difference between relative and absolute loss functions is that relative loss functions calculate the cosine similarity between predicted and true values. This means that even without a strict one-to-one mapping during training, the model can learn a wider range of features and patterns. Furthermore, relative loss functions do not focus on specific numerical values ​​but rather on the similarity between predicted and true values, making the model less susceptible to data distribution and enabling it to handle various types of input data more robustly.

[0132] Therefore, this application can effectively address issues such as overexposure or underexposure, color deviation, and loss of detail that occur during low-light image enhancement, generating enhanced images with clear textures and rich details. Experiments show that the image enhancement model proposed in this application outperforms existing low-light image enhancement methods and exhibits a more natural visual restoration effect on the test set, fully validating the effectiveness of this technology.

[0133] Example 3

[0134] This embodiment provides a computer device, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 10As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores video tag processing data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When executed by the processor, the computer program implements a low-light image enhancement method based on wavelet transform.

[0135] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0136] Example 4

[0137] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0138] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0139] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0140] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0141] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A low-light image enhancement method based on wavelet transform, characterized in that, The low-light image enhancement method based on wavelet transform includes: Acquire the image to be processed; the image to be processed includes at least images with an average brightness of less than 80 candela per square meter; A model for image enhancement is determined; the model includes: a first convolutional module, an illumination adjustment module, a structure prediction module, a detail restoration module, and a dimension reduction activation module. The first convolution module is used to perform preliminary feature extraction on the image to be processed to obtain the image features. The brightness channel of the image to be processed is iteratively adjusted using a high-order curve using the lighting adjustment module to obtain a multi-channel lighting map; The structure prediction module is used to perform multi-branch, multi-scale feature extraction and fusion on the features of the image to be processed and the multi-channel illumination map to obtain a structure prediction map. After performing bidirectional wavelet transform decomposition, spatial and channel attention mechanism weighting, and inverse bidirectional wavelet transform reconstruction on the structure prediction map using the detail restoration module, the detail restoration map is obtained. The dimensionality reduction and activation module is used to perform performance activation and dimensionality reduction on the detail restoration map to obtain an enhanced image of the image to be processed. The illumination adjustment module includes a multilayer perceptron module, a higher-order curve module, and a second convolution module. The illumination adjustment module performs iterative adjustment of the brightness channels of the image to be processed using higher-order curves to obtain a multi-channel illumination map. Specifically, this includes: extracting a histogram representing global information of the image from its brightness channels; inputting the histogram into the multilayer perceptron module to obtain a coefficient set for iterative adjustment of the higher-order curves; based on the coefficient set, iteratively adjusting the histogram using the higher-order curve module to obtain a single-channel illumination map; and using the second convolution module to perform channel boosting on the single-channel illumination map to obtain a multi-channel illumination map. The structure prediction module includes multiple perceptual feature extraction branches at different scales and a third convolution module. The structure prediction module performs multi-branch, multi-scale feature extraction and fusion on the image features to be processed and the multi-channel illumination map to obtain a structure prediction map. Specifically, this includes: concatenating the image features to be processed and the multi-channel illumination map by channel, and then inputting them in parallel into multiple perceptual feature extraction branches at different scales to obtain scale-aware features on each perceptual feature extraction branch; concatenating the scale-aware features from all perceptual feature extraction branches by channel, and then inputting them into the third convolution module for feature fusion to obtain the structure prediction map. The detail restoration module includes a bidirectional wavelet transform module, an attention weighting module, and a scale reconstruction module. The detail restoration module performs bidirectional wavelet transform decomposition, spatial and channel attention mechanism weighting, and inverse bidirectional wavelet transform reconstruction on the structure prediction map to obtain a detail restoration map. Specifically, this includes: decomposing the structure prediction map using the bidirectional wavelet transform module to obtain a wavelet feature map; applying spatial and channel attention mechanisms to the wavelet feature map using the attention weighting module to obtain spatial and channel fusion features; concatenating the wavelet feature map with the spatial and channel fusion features to obtain attention fusion features; performing an inverse bidirectional wavelet transform on the attention fusion features using the scale reconstruction module to obtain a reconstructed feature map; and concatenating the reconstructed feature map with the structure prediction map to obtain the detail restoration map.

2. The low-light image enhancement method based on wavelet transform according to claim 1, characterized in that, The formula for calculating the coefficient set is: In the formula, For the coefficient set, For the first in the coefficient set t One coefficient, It is a multilayer perceptron. For the brightness channel, The desired brightness value, This indicates the extraction of histogram information.

3. The low-light image enhancement method based on wavelet transform according to claim 1, characterized in that, The iterative formula in the higher-order curve module is: In the formula, For the first in the coefficient set k One coefficient, For the first k The brightness channel of the next iteration For the first k +1 iteration of the brightness channel, t This represents the maximum number of iterations.

4. The low-light image enhancement method based on wavelet transform according to claim 1, characterized in that, Each perceptual feature extraction branch is equipped with... The convolutional kernel module and the branch weighting module; the calculation formula for the scale-aware features is: In the formula, For scale-aware feature extraction functions, For multi-channel lighting diagrams, For the features of the image to be processed, The scale of the convolution kernel. For the branch weighting module, express The convolution operation of the convolution kernel, This indicates a layer-by-layer accumulation operation.

5. The low-light image enhancement method based on wavelet transform according to claim 1, characterized in that, The formula for calculating the structural prediction map is: In the formula, For structural prediction diagrams, For the third convolutional module, To extract the number of branches for perceptual features, This represents the splicing operation of perceptual features at different scales. For scale-aware feature extraction functions, For multi-channel lighting diagrams, For the features of the image to be processed, The scale is the size of the convolution kernel.

6. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the low-light image enhancement method based on wavelet transform as described in any one of claims 1-5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the low-light image enhancement method based on wavelet transform as described in any one of claims 1-5.

Images