Zero-sample low-light image enhancement method based on dynamic feature aggregation and color correction
Through the zero-sample low-light image enhancement method of dynamic feature aggregation and color correction, the jump connection and attention mechanism are used to adaptively select features, and combined with color prior information to correct the enhancement curve, solving the insufficient image enhancement and color distortion in extremely dark environments. The problem is that brightness improvement and color contrast improvement have been achieved, overcoming the limitations of model generalization capabilities.
Patent Information
- Application Number
- CN202410911635.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-09
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-07-09
AI Technical Summary
The existing zero-sample low-light image enhancement methods lack flexibility in extremely dark environments and cannot fully enhance the image. They also have problems of color distortion and low color contrast. The generalization ability of the model is limited by the limitations of the training data.
Using the zero-sample low-light image enhancement method based on dynamic feature aggregation and color correction, the enhancement curve is modified by adaptively selecting and learning features through jump connection and attention mechanism, combining color prior information to correct the enhancement curve to avoid color distortion and improve the overall color Contrast.
Effectively enhance image brightness in extremely dark environments, prevent color distortion, improve color contrast, and solve the problems of difficulty in obtaining training samples and limited model generalization capabilities.
Smart Images

Figure CN118941483A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of computer vision and image processing, and in particular relates to a zero-sample low-light image enhancement method based on dynamic feature aggregation and color correction. Background Art
[0002] Low-light image enhancement technology can effectively remove defects in images such as low brightness, local overexposure or underexposure, and color distortion caused by suboptimal lighting conditions such as dim light, backlight, and uneven lighting, restore the information in the scene and improve the aesthetic quality of the image. It has always been an important research direction in image processing.
[0003] With the rapid development of deep learning technology in recent years, it has gradually become mainstream in the field of low-light image enhancement, and related methods have therefore become more accurate and more robust. Low-light image enhancement methods based on deep learning can be divided into supervised learning-based methods and unsupervised learning-based methods according to their learning strategies. Although supervised learning-based methods can achieve better visual effects and evaluation indicators on certain data sets, they require paired low-light-normal light images as training data, which is difficult to obtain in many suboptimal lighting scenes. The limitation of training data also makes this type of method have poor generalization ability and is difficult to cope with scenes with complex and changing lighting conditions. Among the unsupervised learning-based methods, although the unsupervised learning-based methods do not require paired images, they still need to select and classify images into low-light images and normal light images when constructing the data set, which to a certain extent still limits the generalization ability of the model.
[0004] Therefore, methods based on zero-shot learning have also emerged. This type of method goes a step further and completely eliminates the need for paired and unpaired images, and sets a series of no-reference loss functions that do not rely on labels or true values to guide training, avoiding the bias of manually selecting training data that limits the generalization ability of the model. However, existing zero-shot methods have some limitations. First, suboptimal lighting scenes are inherently diverse, and the lighting distribution between these scenes varies greatly, but the ordinary convolutional neural networks in existing zero-shot methods lack flexibility when facing images under different conditions, so they cannot fully enhance images in some scenes, especially in extremely dark environments. Then, since the existing zero-shot methods only use the color constancy assumption to construct the color loss function to prevent overall color cast, and lack constraints on the color retention of local areas or targets, the enhanced images have problems with color distortion and low color contrast. Summary of the invention
[0005] The purpose of the present invention is to provide a zero-shot low-light image enhancement method based on dynamic feature aggregation and color correction, apply dynamic feature aggregation to the enhancement curve estimation network, give full play to the ability of jump connection and attention mechanism to adaptively select and learn features, estimate suitable enhancement curves from a variety of suboptimal lighting scenes, especially low-light images in extremely dark environments, to enhance the brightness of the image, and use color prior information in the low-light image to correct the enhancement curve to prevent color distortion during the enhancement process and improve the overall color contrast. In addition, the zero-shot learning method can effectively solve the problems of difficulty in obtaining training samples and the limitation of model generalization ability by training data.
[0006] To solve the above technical problems, the technical solution of the present invention is: a zero-sample low-light image enhancement method based on dynamic feature aggregation and color correction, comprising the following steps:
[0007] S1. Scale the preset training image to a size of n×n, calculate the color correction factor for its three color channels of R, G, and B, input the scaled training image and its color correction factor into a preset enhancement curve estimation network, obtain the corresponding enhancement curve for each pixel at each iteration according to the preset curve expression, correct the enhancement curve by the color correction factor, iterate the scaled training image by the corrected enhancement curve k times, and obtain a preliminary enhancement result; wherein, 128≤n≤512, and n is an integer multiple of 16; 2≤k≤10;
[0008] S2. Calculate the reference-free loss for the parameter matrix of the modified enhancement curve and the preliminary enhancement result, and train the enhancement curve estimation network for a preset number of training rounds according to the back propagation method;
[0009] S3. Calculate the color correction factor of the preset test image, and input the test image and its color correction factor into the trained enhancement curve estimation network to obtain the enhancement curve corresponding to the test image, correct the enhancement curve by the color correction factor of the test image, iterate the corresponding enhancement curve of the test image after correction k times, and obtain the final enhanced image.
[0010] S1 is specifically:
[0011] S1a, for each pixel in the three color channels R, G, and B of the scaled training image, the corresponding color correction factor is calculated according to the maximum and minimum values of the pixel values of the channel in which it is located;
[0012] S1b, the scaled training image and its color correction factor are spliced and input into the enhancement curve estimation network to predict the enhancement curve generated after each iteration;
[0013] S1c, the enhancement curve is corrected by the color correction factor, and the scaled training image is iterated k times through the corrected enhancement curve to obtain a preliminary enhancement result.
[0014] S2 is specifically:
[0015] S2a, calculate the no-reference loss according to the parameter matrix of the modified enhancement curve and the preliminary enhancement result, and use the no-reference loss to weight the total loss function value. The no-reference loss specifically includes four types of loss functions, namely, neighborhood contrast loss, global color loss, brightness loss, and enhanced smoothness loss;
[0016] S2b. Train the enhanced curve estimation network for a preset number of training rounds using a back propagation method according to the total loss function value.
[0017] S1a is specifically:
[0018] The color correction factor is defined as follows: For a pixel I(i,j) with coordinates (i,j), the corresponding color correction factor is defined as:
[0019]
[0020] Among them, MAX and MIN are the normalized pixel value ranges, φ and Respectively represent the maximum and minimum values in the three color channels of R, G, and B;
[0021] The overall color correction factor ψ(i,j) is calculated and integrated from the three color channels of R, G, and B:
[0022]
[0023] Among them, ψ R (i,j),ψ G (i,j),ψ B (i, j) are the color correction factors corresponding to pixel I(i, j) in the three color channels of R, G, and B respectively, and the size of ψ is 3×n×n.
[0024] S1b is specifically:
[0025] The scaled training image and its color correction factor are concatenated and input into the enhancement curve estimation network, which includes at least three 3×3 shallow feature extraction convolution layers, two dynamic feature extraction modules with the same structure, a channel attention block and a pixel attention block; wherein the dynamic feature extraction block includes at least three 3×3 convolution layers, a ReLU activation function layer, a channel attention block and a pixel attention block; the channel attention block includes at least two 1×1 convolution layers, an average pooling layer, a ReLU activation function layer and a Sigmoid activation function layer; the pixel attention block includes at least two 3×3 convolution layers, a ReLU activation function layer and a Sigmoid activation function layer;
[0026] For the input of the scaled training image and its color correction factor spliced together, a 3×3 convolution is first used to extract the shallow feature F, and F is sequentially passed through two dynamic feature extraction modules with the same structure. The second dynamic feature extraction module takes the output of the first dynamic feature extraction module as input. In the first dynamic feature extraction module, F is first subjected to residual mapping including a 3×3 convolution layer and a ReLU activation function to obtain the feature F r :
[0027]
[0028] F r Through a 3×3 convolutional layer, the channel and pixel attention are calculated and feature weighted, and a skip connection is made to obtain the feature F selected by attention. att :
[0029]
[0030] Among them, PA and CA are pixel attention block and channel attention block respectively;
[0031] In the channel attention weighting, for the feature F of the cth channel c , perform global average pooling, and then pass through two 1×1 convolutional layers and Sigmoid and ReLU activation functions to obtain its weight It is expressed as:
[0032]
[0033] Among them, GAP is the global average pooling, σ is the Sigmoid activation function;
[0034] Multiply the features of each channel by its corresponding weight to complete the adaptive selection of features of different channels. For the feature F of the cth channel, c Multiplying it by its corresponding weight is expressed as:
[0035]
[0036] in, is the feature weighted by the cth channel;
[0037] The channel-weighted features Perform pixel attention weighting and selection: features weighted by channels Through two 3×3 convolutional layers and Sigmoid and ReLU activation functions, the weights of all pixels in the feature map are obtained. Make a skip connection to get the output feature F of the pixel attention block * , expressed as:
[0038]
[0039] In the dynamic feature extraction module, feature F * That is, the feature F selected by attention att , F att Through a 3×3 convolutional layer and skip connection, the output feature F of the nth dynamic feature extraction module is obtained n , expressed as:
[0040]
[0041] The features of different depths extracted by the two dynamic feature extraction modules are dynamically aggregated through attention operation to obtain the final extracted feature F agg , expressed as:
[0042]
[0043] F agg Through two 3×3 post-processing convolutional layers, the parameter matrix of the enhancement curve with a size of 3k×n×n is predicted. Enhancement curves corresponding to the k-iterative process for each pixel in each color channel of the low-light image.
[0044] S1c is specifically:
[0045] The enhancement curve is corrected by the color correction factor to obtain the parameter matrix of the final enhancement curve for:
[0046]
[0047] Where i represents the i-th iterative enhancement of the image;
[0048] The corresponding enhancement curve is obtained according to the preset curve expression, and the scaled training image is iterated through the modified enhancement curve. The modified enhancement curve is expressed as:
[0049]
[0050] Among them, Y i represents the image after the i-th iteration enhancement;
[0051] Each iterative enhancement takes the output of the previous iterative enhancement as input, and outputs the preliminary enhancement result after k iterations.
[0052] S2a is specifically:
[0053] Calculate the neighborhood contrast loss L between the initial enhancement result and the scaled training image nc :
[0054]
[0055] Where A is the number of local regions of size 4×4 divided in the image, N4(i) represents the four adjacent regions above, below, left and right centered on region i, Y and I represent the preliminary enhancement result and the average intensity of the local region of the scaled training image, respectively;
[0056] Calculate the global color loss L for the preliminary enhancement results rgb And the brightness loss L br ; Among them, the global color loss L rgb The calculation method is expressed as:
[0057]
[0058] Among them, Y m and Y n Represents the average brightness of the m and n color channels respectively;
[0059] Brightness loss L br The calculation method is expressed as:
[0060]
[0061] Where M is the number of non-overlapping local regions of size 16×16, and Y k is the average brightness of each local area; E is a constant, representing a brightness level with good visibility;
[0062] Calculate the enhanced smoothing loss L for the parameter matrix of the modified enhancement curve TV , expressed as:
[0063]
[0064] Among them, N represents the batch size during training, and Represent the gradients in the horizontal and vertical directions respectively, is the parameter matrix of the modified enhancement curve.
[0065] Finally, the total loss function L total It is expressed as the weighted sum of the above four loss functions, expressed as:
[0066] L total =αL nc +L rgb +βL br +γL TV
[0067] Among them, α, β, and γ are all constants for controlling the weights of the loss terms.
[0068] S2b is specifically:
[0069] After obtaining the total loss function value, use the ADAM optimizer with 1e -4 The learning rate is used to train the enhanced curve estimation network for a preset number of training rounds through the back propagation method according to the total loss function value.
[0070] n is 256.
[0071] k is 8.
[0072] Compared with the prior art, the present invention has the following beneficial effects:
[0073] The present invention applies dynamic feature aggregation to the enhancement curve estimation network. The ability of jump connections and attention mechanisms to adaptively select and learn features is fully utilized to estimate appropriate enhancement curves from a variety of suboptimal lighting scenes, especially low-light images in extremely dark environments, to enhance the brightness of the image. At the same time, the enhancement curve is corrected using color prior information in low-light images to prevent color distortion during the enhancement process and improve the overall color contrast. In addition, the zero-sample learning method can effectively solve the problem of difficulty in obtaining training samples and the limitation of model generalization ability by training data. BRIEF DESCRIPTION OF THE DRAWINGS
[0074] Figure 1 It is a schematic diagram of a flow chart of an embodiment of the present invention;
[0075] Figure 2 It is a schematic diagram of the structure of the overall framework and the enhanced curve estimation network in an embodiment of the present invention;
[0076] Figure 3 It is a structural schematic diagram of a dynamic feature extraction module, a channel attention block and a pixel attention block in an embodiment of the present invention;
[0077] Figure 4 is a low-light input image tested in an embodiment of the present invention;
[0078] Figure 5 This is the enhanced result image tested in the embodiment of the present invention. DETAILED DESCRIPTION
[0079] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0080] refer to Figure 1-Figure 5 The technical solution of the present invention is: a zero-sample low-light image enhancement method based on dynamic feature aggregation and color correction, comprising the following steps:
[0081] S1. Scale the training image to 256×256 (in this embodiment, n is 256) as the input image, first calculate the color correction factor, then input the input image and its color correction factor into the enhancement curve estimation network to obtain the corresponding enhancement curve, then use the color correction factor to correct it, and finally iterate the training image eight times (in this embodiment, k is 8) according to the corrected curve to obtain a preliminary enhancement result.
[0082] S1a, first calculate its color correction factor. For the pixel I(i,j) with coordinates (i,j), its corresponding correction factor is defined as:
[0083]
[0084] Among them, MAX and MIN represent the normalized pixel value range, φ and Respectively represent the maximum and minimum values in the current color channel. The overall color correction factor is calculated and integrated from the three RGB color channels:
[0085]
[0086] Its size is 3 × 256 × 256. S1b, first concatenate the color correction factor with the low-light image in the channel dimension, and then input it into the enhancement curve estimation network to predict the enhancement curve required for iteration.
[0087] First, a 3×3 convolution layer is used to extract shallow features, and then they are fed into two identical dynamic feature extraction modules in sequence. The latter uses the output features of the former as input to extract features of different depths. Each dynamic feature extraction module combines residual mapping, attention mechanism, and jump connections of different lengths. In each feature dynamic, for the input feature F, a residual mapping including a 3×3 convolution and ReLU activation is first performed to obtain the feature F. r :
[0088]
[0089] Then to F r Use the attention mechanism for feature selection. r Through a 3×3 convolution, channel and pixel attention are calculated and feature weighted, and a skip connection is obtained
[0090]
[0091] Where PA and CA represent pixel attention blocks and channel attention blocks, respectively. Specifically, in the channel attention weighting, for the feature F of the cth channel c , first perform global average pooling, then pass two 1×1 convolutions and Sigmoid and ReLU activations to get its weight
[0092]
[0093] Where GAP stands for global average pooling and σ is Sigmoid activation. The feature of each channel is multiplied by its corresponding weight to complete the adaptive selection of features of different channels.
[0094]
[0095] After that, the channel-weighted features Perform pixel attention weighting and selection. This process includes two 3×3 convolutions and Sigmoid and ReLU activations to obtain the weights of all pixels in the feature map. Each pixel is then multiplied by its corresponding weight to achieve adaptive selection of features at different positions. Then another jump connection is performed to obtain the feature F extracted through dynamic selection. * .
[0096]
[0097] Finally, through another 3×3 convolution and skip connection, the output of the feature dynamic extraction block is obtained:
[0098]
[0099] Next, the features of different depths extracted by the two dynamic feature extraction modules are dynamically aggregated through additional attention operations to obtain the final extracted feature F agg :
[0100]
[0101] Finally, the feature is passed through two 3×3 post-processing convolutional layers to predict the parameter matrix of the enhancement curve with a size of 24×256×256 Enhancement curves corresponding to eight iterations of the process for each pixel in each color channel of the low-light image.
[0102] S1c, using the color correction factor to correct the enhancement curve, and then iterating the input training image eight times according to the corrected curve to obtain a preliminary enhancement result.
[0103] First, the enhancement curve is corrected with the color correction factor to obtain the parameter matrix of the final enhancement curve for:
[0104]
[0105] Here, i represents the i-th iterative enhancement of the image. Then, the brightness of the input image is adjusted using the following curve:
[0106]
[0107] Among them, Y i It represents the image after the i-th iteration enhancement. Each iteration enhancement takes the previous output as input, and after eight iterations, the initial enhancement result is output.
[0108] S2. Calculate the no-reference loss for the parameters and enhancement results of the enhanced curve corrected in step 1, and then guide the training of the enhanced curve estimation network through the back propagation method until the number of training rounds is completed.
[0109] S2a, calculate three loss functions for the obtained enhancement results, namely neighborhood contrast loss L nc , global color loss L rgb And the brightness loss L br The neighborhood contrast loss encourages maintaining the brightness relationship of adjacent local areas during the enhancement process by calculating the changes in the brightness relationship between adjacent areas before and after enhancement.
[0110]
[0111] Where A is the number of local regions of size 4×4 divided in the image, N4(i) represents the four adjacent regions above, below, left and right centered on region i. Y and I represent the preliminary enhancement result and the average intensity of the local region of the input image, respectively.
[0112] According to the gray world algorithm, for an image with a large number of color changes, the average values of the three RGB components tend to the same gray value. Therefore, a global color loss is constructed between the RGB color channel pairs of the enhanced results to prevent overall color deviation.
[0113]
[0114] where Y m and Y n Represents the average brightness of the m and n color channels respectively.
[0115] Brightness loss constrains the local average brightness of the enhanced result to a fixed level, which is used to make the low-light image obtain appropriate brightness after enhancement while preventing local overexposure or underexposure.
[0116]
[0117] Where M is the number of non-overlapping local regions of size 16×16, and Y k is the average brightness of each local area. E is a constant, representing a brightness level with good visibility.
[0118] In addition, the value of the enhancement curve parameter represents the amount of pixel enhancement. In order to prevent artifacts caused by excessive differences in the enhancement curves of adjacent pixels, the enhancement smoothing loss L is calculated for the parameter matrix of all modified enhancement curves. TV To guide enhanced smoothness.
[0119]
[0120] Where N represents the batch size during training. and Represent the horizontal and vertical gradients respectively. is the parameter matrix of the modified enhancement curve.
[0121] Finally, the total loss function L total It is expressed as the weighted sum of the above four loss functions.
[0122] L total =αL nc +L rgb +βL br +γL TV
[0123] Among them, α, β, and γ are all constants for controlling the weights of the loss terms.
[0124] S2b, after obtaining the total loss function value, use the ADAM optimizer with 1e -4 The learning rate is back-propagated and the enhanced curve estimation network is optimized until the specified number of training rounds is completed to obtain a trained model.
[0125] S3. Load the trained model parameters into the enhancement curve estimation network, input the low-light image for test, first calculate the color correction factor, then input the low-light image and the correction factor into the enhancement curve estimation network to obtain the enhancement curve, then correct it with the correction factor, and finally iterate the low-light image eight times according to the corrected enhancement curve to output the final enhanced image.
[0126] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. Zero-shot low-light image enhancement method based on dynamic feature aggregation and color correction, It is characterized in that The following steps are involved: S1. Scale the preset training image to a size of n×n, calculate the color correction factor for its three color channels of R, G, and B, input the scaled training image and its color correction factor into a preset enhancement curve estimation network, obtain the corresponding enhancement curve for each pixel at each iteration according to the preset curve expression, correct the enhancement curve by the color correction factor, iterate the scaled training image by the corrected enhancement curve k times, and obtain a preliminary enhancement result; wherein, 128≤n≤512, and n is an integer multiple of 16; 2≤k≤10; S2. Calculate the reference-free loss for the parameter matrix of the modified enhancement curve and the preliminary enhancement result, and train the enhancement curve estimation network for a preset number of training rounds according to the back propagation method; S3. Calculate the color correction factor of the preset test image, and input the test image and its color correction factor into the trained enhancement curve estimation network to obtain the enhancement curve corresponding to the test image, correct the enhancement curve by the color correction factor of the test image, iterate the corresponding enhancement curve of the test image after correction k times, and obtain the final enhanced image.
2. The zero-shot low-light image enhancement method based on dynamic feature aggregation and color correction according to claim 1, It is characterized in that S1 is specifically: S1a, for each pixel in the three color channels R, G, and B of the scaled training image, the corresponding color correction factor is calculated according to the maximum and minimum values of the pixel values of the channel in which it is located; S1b, the scaled training image and its color correction factor are spliced and input into the enhancement curve estimation network to predict the enhancement curve generated after each iteration; S1c, the enhancement curve is corrected by the color correction factor, and the scaled training image is iterated k times through the corrected enhancement curve to obtain a preliminary enhancement result.
3. The zero-shot low-light image enhancement method based on dynamic feature aggregation and color correction according to claim 2, It is characterized in that S2 is specifically: S2a, calculate the no-reference loss according to the parameter matrix of the modified enhancement curve and the preliminary enhancement result, and use the no-reference loss to weight the total loss function value. The no-reference loss specifically includes four types of loss functions, namely, neighborhood contrast loss, global color loss, brightness loss, and enhanced smoothness loss; S2b. Train the enhanced curve estimation network for a preset number of training rounds using a back propagation method according to the total loss function value.
4. The zero-shot low-light image enhancement method based on dynamic feature aggregation and color correction according to claim 3, It is characterized in that S1a is specifically: The color correction factor is defined as follows: For a pixel I(i, j) with coordinates (i, j), its corresponding color correction factor is defined as: Among them, MAX and MIN are the normalized pixel value ranges, φ and Respectively represent the maximum and minimum values in the three color channels of R, G, and B; The overall color correction factor ψ(i, j) is calculated and integrated from the three color channels of R, G, and B as follows: Among them, ψ R (i, j), ψ G (i, j), ψ B (i, j) are the color correction factors corresponding to the pixel I(i, j) in the three color channels of R, G, and B respectively, and the size of ψ is 3×n×n.
5. The zero-shot low-light image enhancement method based on dynamic feature aggregation and color correction according to claim 4, It is characterized in that S1b is specifically: The scaled training image and its color correction factor are concatenated and input into the enhancement curve estimation network, which includes at least three 3×3 shallow feature extraction convolution layers, two dynamic feature extraction modules with the same structure, a channel attention block and a pixel attention block; wherein the dynamic feature extraction block includes at least three 3×3 convolution layers, a ReLU activation function layer, a channel attention block and a pixel attention block; the channel attention block includes at least two 1×1 convolution layers, an average pooling layer, a ReLU activation function layer and a Sigmoid activation function layer; the pixel attention block includes at least two 3×3 convolution layers, a ReLU activation function layer and a Sigmoid activation function layer; For the input of the scaled training image and its color correction factor spliced together, a 3×3 convolution is first used to extract the shallow feature F, and F is sequentially passed through two dynamic feature extraction modules with the same structure. The second dynamic feature extraction module takes the output of the first dynamic feature extraction module as input. In the first dynamic feature extraction module, F is first subjected to residual mapping including a 3×3 convolution layer and a ReLU activation function to obtain the feature F r : F r Through a 3×3 convolutional layer, the channel and pixel attention are calculated and feature weighted, and a skip connection is made to obtain the feature F selected by attention. att : Among them, PA and CA are pixel attention block and channel attention block respectively; In the channel attention weighting, for the feature F of the cth channel c , perform global average pooling, and then pass through two 1×1 convolutional layers and Sigmoid and ReLU activation functions to obtain its weight It is expressed as: Among them, GAP is the global average pooling, sσ is the Sigmoid activation function; Multiply the features of each channel by its corresponding weight to complete the adaptive selection of features of different channels. For the feature F of the cth channel, c Multiplying it by its corresponding weight is expressed as: in, is the feature weighted by the cth channel; The channel-weighted features Perform pixel attention weighting and selection: features weighted by channels Through two 3×3 convolutional layers and Sigmoid and ReLU activation functions, the weights of all pixels in the feature map are obtained. Make a skip connection to get the output feature F of the pixel attention block * , expressed as: In the dynamic feature extraction module, feature F * That is, the feature F selected by attention att , F att Through a 3×3 convolutional layer and skip connection, the output feature F of the nth dynamic feature extraction module is obtained n , expressed as: The features of different depths extracted by the two dynamic feature extraction modules are dynamically aggregated through attention operation to obtain the final extracted feature F agg , expressed as: F agg Through two 3×3 post-processing convolutional layers, the parameter matrix of the enhancement curve with a size of 3k×n×n is predicted. Enhancement curves corresponding to the k-iterative process for each pixel in each color channel of the low-light image.
6. The zero-shot low-light image enhancement method based on dynamic feature aggregation and color correction according to claim 5, It is characterized in that S1c is specifically: The enhancement curve is corrected by the color correction factor to obtain the final parameter matrix of the enhancement curve for: Where i represents the i-th iterative enhancement of the image; The corresponding enhancement curve is obtained according to the preset curve expression, and the scaled training image is iterated through the modified enhancement curve. The modified enhancement curve is expressed as: Among them, Y i represents the image after the i-th iteration enhancement; Each iterative enhancement takes the output of the previous iterative enhancement as input, and outputs the preliminary enhancement result after k iterations.
7. The zero-shot low-light image enhancement method based on dynamic feature aggregation and color correction according to claim 6, It is characterized in that S2a is specifically: Calculate the neighborhood contrast loss L between the initial enhancement result and the scaled training image nc : Among them, A is the number of local areas of size 4×4 divided in the image, N 4 (i) represents the four adjacent regions above, below, left and right of region i, Y and I represent the preliminary enhancement result and the average intensity of the local region of the scaled training image, respectively; Calculate the global color loss L for the preliminary enhancement results rgb And the brightness loss L br ; Among them, the global color loss L rgb The calculation method is expressed as: Among them, Y m and Y n Represents the average brightness of the m and n color channels respectively; Brightness loss L br The calculation method is expressed as: Where M is the number of non-overlapping local regions of size 16×16, and Y k is the average brightness of each local area; E is a constant, representing a brightness level with good visibility; Calculate the enhanced smoothing loss L for the parameter matrix of the modified enhancement curve TV , expressed as: Among them, N represents the batch size during training, and Represent the gradients in the horizontal and vertical directions respectively, is the parameter matrix of the modified enhancement curve. Finally, the total loss function L total It is expressed as the weighted sum of the above four loss functions, expressed as: L total =αL nc +L rgb +βL br +γL TV Among them, α, β, and γ are all constants for controlling the weights of the loss terms.
8. The zero-shot low-light image enhancement method based on dynamic feature aggregation and color correction according to claim 3, It is characterized in that S2b is specifically: After obtaining the total loss function value, use the ADAM optimizer with 1e -4 The learning rate is used to train the enhanced curve estimation network for a preset number of training rounds through the back propagation method according to the total loss function value.
9. The zero-shot low-light image enhancement method based on dynamic feature aggregation and color correction according to claim 1, It is characterized in that n is 256.
10. The zero-shot low-light image enhancement method based on dynamic feature aggregation and color correction according to claim 1, It is characterized in that k is 8.
Citation Information
Patent Citations
No-reference low-illumination image enhancement method based on local scene perception
CN115205160A
No-reference low-illumination image enhancement method based on high-order curve iteration
CN116912114A
HSV and Transform-based unsupervised low-illumination image enhancement method
CN118247192A
Cited By
Image preprocessing neural network model, training method, near-eye display method, device, equipment and medium
CN120852207A
Photographing method, electronic device, readable storage medium and program product
CN122765333A
