Unmanned aerial vehicle imaging data enhancement method for low-illumination scene

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a low-illumination scene-oriented unmanned aerial vehicle imaging data enhancement method, which comprises the following steps of: S1, acquiring unmanned aerial vehicle imaging data and constructing a training data set; s2, data preprocessing based on illumination partition; s3, constructing and training a data enhancement model; and S4, enhancing the imaging data of the unmanned aerial vehicle. The data preprocessing based on the illumination partition in the step S2 comprises the following steps: generating a low-illumination area mask and a medium-illumination area mask based on the brightness information of the low-illumination image; a Gaussian smooth convolution operation is applied to the low-illuminance region identified by the low-illuminance region mask to suppress noise, and an original value is reserved for the medium-illuminance region identified by the medium-illuminance region mask to generate a pre-processed image. The problems of low-illumination image noise amplification and detail loss are solved by generating low-illumination region masks and medium-illumination region masks and respectively adopting noise reduction and retention strategies, and the balance of dark region noise reduction and bright region detail protection is realized.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method for enhancing drone imaging data in low-light scenarios. Background Technology

[0002] In practical applications, drones often need to perform tasks in various low-light environments, such as nighttime inspections, emergency rescues, and security monitoring under complex weather conditions. Images acquired in these low-light scenarios generally suffer from insufficient brightness, reduced contrast, high noise levels, and blurred texture details, leading to a significant decrease in the performance of subsequent tasks such as target detection, scene segmentation, and visual navigation. Traditional image enhancement methods, such as histogram equalization and multi-scale Retinex, can improve image brightness to some extent, but they often amplify noise, destroy the natural contrast of the image, and have unstable enhancement effects. Deep learning-based low-light enhancement methods (such as LLNet and SID) often use single-path network structures, which cannot simultaneously consider the global illumination context and the restoration of local texture details, easily leading to problems such as oversmoothing, color distortion, and insufficient detail in dark areas. Furthermore, existing methods generally employ a uniform loss function and a fixed learning rate, treating different illumination areas equally and ignoring the differences in visual sensitivity and training convergence speed between dark and bright areas. This results in insufficient dark area recovery and low training efficiency. Therefore, there is an urgent need for an enhancement method for low-light UAV imaging that can adaptively process different brightness areas, effectively suppress noise and preserve details while improving overall brightness and contrast, and improve the recovery quality and training efficiency of dark areas through optimization strategies to meet the application needs of UAVs in complex low-light environments.

[0003] Existing technologies still have many shortcomings in practical applications: conventional enhancement methods such as histogram equalization cannot adaptively process different brightness regions, easily amplifying noise in dark areas and destroying natural contrast in bright areas, resulting in unstable enhancement effects. Single-path deep learning models struggle to simultaneously consider global illumination context and local texture details, leading to insufficient contrast or blurred textures in enhanced images. Loss functions with uniform weights apply the same optimization objective to all regions, ignoring the more critical visual sensitivity of dark areas, resulting in insufficient recovery of dark area details and uneven enhancement quality. Fixed learning rates and uniform parameter update strategies ignore the differences in convergence speed and optimization requirements between dark and bright areas, easily leading to slow parameter learning in dark areas, getting stuck in local optima, and limiting overall training efficiency and enhancement effects. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method for enhancing UAV imaging data in low-light scenarios. It solves the problem of adaptive processing for different brightness areas, effectively suppressing noise and preserving details while improving overall brightness and contrast.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for enhancing UAV imaging data in low-light scenarios, comprising the following steps:

[0006] S1. UAV imaging data acquisition and training dataset construction: acquire raw low-light images in low-light scenes, and acquire high-quality images under normal illumination in the same scene as real label images to construct training data.

[0007] S2. Data preprocessing based on illuminance partitioning: The original low-illuminance image is partitioned by illuminance to generate illuminance masks that identify different brightness areas. Based on the illuminance masks, noise reduction is performed on the low-illuminance areas and detail preservation is performed on the medium-illuminance areas to generate a preprocessed image.

[0008] S3. Data augmentation model construction and training: Construct and train a data augmentation model, which includes a dual-branch feature extraction network, an attention enhancement module, and a reconstruction module. The training process adopts a partition weighted loss function and an adaptive update mechanism for illumination perception parameters.

[0009] S4. UAV imaging data enhancement: Load the trained data enhancement model, perform the data preprocessing based on illumination partitioning on the new low-light UAV image in sequence, and process it through the data enhancement model to output the enhanced image.

[0010] Preferably, the data preprocessing based on illuminance zoning in S2 includes:

[0011] Generate low-light region masks and medium-light region masks based on the brightness information of low-light images;

[0012] Gaussian smoothing convolution is applied to the low-light regions identified by the low-light region mask to suppress noise, and the original values ​​are preserved for the medium-light regions identified by the medium-light region mask to generate a preprocessed image.

[0013] Preferably, the construction of the dual-branch feature extraction network includes:

[0014] The global branch uses dilated convolution to expand the receptive field to capture the global illumination context information of the image and outputs a global feature map.

[0015] The local branch uses standard convolution to focus on local regions to extract high-frequency texture details of the image and outputs a local branch feature map.

[0016] Preferably, the attention enhancement module is used for:

[0017] The original low-light image is subjected to parallel average pooling and max pooling operations in the spatial dimension. The pooling results are then concatenated and processed by convolution and activation functions to generate an illumination attention map that reflects the relative brightness of each spatial location. Then, the illumination attention map and its complement are used to weight the global feature map and the local branch feature map respectively to achieve feature fusion and generate a fused feature map.

[0018] Preferably, the reconstruction module is used for:

[0019] The fused feature map is input into a cascaded residual set for stepwise feature optimization;

[0020] The output of the residual group is then added to the preprocessed image, and the final enhanced image is generated through global residual concatenation.

[0021] Preferably, the data augmentation model construction and training in S3 adopts a partition-weighted loss function, which assigns higher weights to pixel-level differences and structural information loss in low-light areas and lower weights to pixel-level differences and structural information loss in medium-light areas.

[0022] Preferably, the partition-weighted loss function includes a region-weighted loss, which is calculated as follows:

[0023] L L1 =λ low ||M low ⊙(I enh -I gt )||1+λ mid ||M mid ⊙(I enh -I gt )||1;

[0024] In the formula, L L1 M represents the region-weighted L1 loss, used to measure the pixel-level difference between the enhanced and real images; low M is the mask for the low-light region; mid For the medium illumination region mask; I enh The enhanced image is represented by ⊙; ⊙ represents the element-wise multiplication operator; λ represents the image itself. low This represents the weighting coefficient for low-light areas, with a preferred value of 0.7, used to increase the weight of dark area loss; I gt This represents the true labeled image, a high-quality image under normal illumination; ||·||1 represents the L1 norm; λ midThis represents the weighting coefficient for the medium illumination area, with a preferred value of 0.3, used to reduce the weighting of bright area loss.

[0025] Preferably, in S3, the data augmentation model construction and training adopts illumination-aware learning rate scheduling, and the effective learning rate is dynamically calculated based on the relative proportion of dark areas in the image. When the proportion of dark areas is high, the learning rate is increased.

[0026] Preferably, the effective learning rate is calculated as follows:

[0027] η eff =η base ·(0.5+0.5·r) a ;

[0028] In the formula, η eff η represents the effective learning rate, the actual step size used for updating network parameters; base represents the base learning rate, preferably 0.001, which serves as the baseline value for learning rate scheduling; 'a' represents the adjustment factor, preferably 0.5, used to control the sensitivity of the learning rate to the proportion of dark areas; and 'r' represents the proportion of dark areas.

[0029] Preferably, the data augmentation model construction and training in S3 adopts an illumination-aware adaptive parameter update mechanism, wherein the adaptive parameter update includes:

[0030] Based on the illumination attention map and the proportion of dark areas, the importance weights of global branch parameters, local branch parameters, and attention enhancement and reconstruction module parameters are calculated.

[0031] The corrected gradient vector is calculated based on the total loss gradient, parameter importance weights, and historical momentum, and the scaled gradient step vector is generated by combining the effective learning rate and gradient variance estimation.

[0032] The trainable parameters are updated using the scaling gradient step vector, and a weight decay regularization term is added to address overfitting.

[0033] This invention provides a method for enhancing UAV imaging data in low-light scenarios. It offers the following advantages:

[0034] 1. This invention generates masks for low-light and medium-light areas, and employs noise reduction and preservation strategies respectively to solve the problems of noise amplification and detail loss in low-light images, achieving a balance between noise reduction in dark areas and detail preservation in bright areas.

[0035] 2. This invention employs a global dilated convolution branch and a local compact convolution branch, combined with an illumination attention map for adaptive feature fusion, which preserves global contrast information while enhancing local texture details, thereby achieving illumination-aware feature modeling.

[0036] 3. This invention introduces illumination weights into the loss function, giving higher weights to pixels in dark areas. Combined with SSIM and gradient loss, the model focuses more on the quality of dark areas and the restoration of edge details, which is in line with the visual sensitivity characteristics of the human eye.

[0037] 4. This invention solves the problem that fixed learning rate and uniform parameter update strategies ignore the differences in convergence speed and optimization requirements between dark and bright areas, which can easily lead to slow parameter learning in dark areas, getting stuck in local optima, and limiting the overall training efficiency and enhancement effect. Attached Figure Description

[0038] Figure 1 This is a flowchart of the method of the present invention;

[0039] Figure 2 The original low-light image taken by the drone of this invention;

[0040] Figure 3 This is the enhanced image of the present invention;

[0041] Figure 4 This is a schematic diagram comparing the brightness distribution of the present invention;

[0042] Figure 5 This is a schematic diagram of the equalized brightness distribution of the present invention;

[0043] Figure 6 This is a schematic diagram of the color channels for low-light images according to the present invention;

[0044] Figure 7 This is a multi-directional Gabor difference diagram of the present invention;

[0045] Figure 8 This is a comparison chart of the peak signal-to-noise ratios of different methods of the present invention;

[0046] Figure 9 This is a comparison diagram showing the structural similarity of different methods of the present invention. Detailed Implementation

[0047] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Example:

[0049] Please see the appendix Figure 1 -Appendix Figure 9 This invention provides a method for enhancing UAV imaging data in low-light scenarios, the main steps of which are as follows:

[0050] S1. UAV Imaging Data Acquisition and Training Dataset Construction

[0051] The drone imaging data acquisition is carried out by drones in various low-light scenarios, including dim lighting environments such as dusk, night, cloudy days, and indoor shadows, to ensure coverage of different lighting conditions and scene types;

[0052] During data acquisition, the drone is equipped with a high-performance RGB camera and fixed shooting parameters such as resolution, focal length and aperture are set to capture raw low-light images.

[0053] Meanwhile, in order to obtain the corresponding real label images, high-quality images under normal illumination are captured in the same scene by adjusting the exposure time and ISO gain of the drone camera or using external supplementary lighting equipment to ensure that the image content is strictly aligned and to avoid scene movement or distortion.

[0054] The real label images undergo post-processing, such as color correction and noise removal, to provide clear, natural reference images.

[0055] The dataset is divided into a training set, a validation set, and a test set. The training set is used for model parameter learning, the validation set is used for hyperparameter tuning and early stopping detection, and the test set is used for final performance evaluation.

[0056] Optionally, data augmentation techniques such as random cropping, horizontal flipping, and brightness fine-tuning can be applied to increase data diversity and improve the model's generalization ability and robustness.

[0057] S2. Data preprocessing based on illuminance zoning

[0058] Low-light drone images often have problems such as extremely uneven brightness distribution and low signal-to-noise ratio in dark areas. Conventional global histogram equalization methods often amplify image noise and destroy the natural contrast of the image when processing such images.

[0059] This invention processes input images using an illumination-sensing zoning strategy to eliminate noise amplification and detail loss, preserving details in mid-bright areas while suppressing noise in dark areas. The specific steps are as follows:

[0060] 1. Generate illuminance mask

[0061] Based on the brightness image of the low-light image, an element-wise comparison is performed using low-light thresholds and high-light thresholds to generate low-light region masks and medium-light region masks. The low-light region mask identifies pixels with brightness values ​​lower than or equal to the low-light threshold, while the medium-light region mask identifies pixels with brightness values ​​between the low-light threshold and the high-light threshold, thus distinguishing different brightness regions. This is represented as follows:

[0062] Mlow =I(D(I) in )≤τ low );

[0063] M mid =I(τ) low <D(I in )≤τ high );

[0064] In the formula, M low This represents a low-light region mask, used to identify pixels with brightness values ​​lower than or equal to a low-light threshold. It is a binary matrix with elements of 0 or 1 and dimensions H×W; M mid This represents a mid-illuminance region mask, used to identify pixels with brightness values ​​between the low-illuminance threshold and the high-illuminance threshold. It is a binary matrix with elements taking values ​​of 0 or 1, and dimensions of H×W; in This represents a low-light image, with dimensions H×W×3, and is an RGB three-channel image from raw data captured by a drone; D(I in ) represents the brightness of a low-light image, such as the Y channel of YUV color or the L channel of Lab color; H is the image height; W is the image width; τ low This represents the low-light threshold, preferably set to 0.1, used to distinguish low-light areas; τ high This represents the high-illuminance threshold, preferably 0.5, used to distinguish medium-illuminance areas; I(·) is the element comparison indicator function, which performs element-by-element comparisons on the matrix, specifically for low-illuminance images. in For each pixel in the array, check if the condition is true. If it is true, output 1 at the corresponding pixel position; otherwise, output 0.

[0065] In the embodiment, based on the original low-light image taken by the drone, it can be seen that the overall brightness is low, the contrast is poor, and the details are not obvious.

[0066] 2. Application partitioning

[0067] Gaussian smoothing convolution is applied to low-light regions to suppress noise, while the original values ​​are preserved in mid-light regions. The smoothed low-light regions and the original mid-light regions are combined using element-wise multiplication to generate a preprocessed image. The low-light region mask and the mid-light region mask are then broadcast to the RGB channels to achieve noise reduction in low-light regions and detail preservation in mid-light regions, as shown below:

[0068]

[0069] In the formula, I part The image represents the preprocessed image, with dimensions H×W×3, indicating the results of noise reduction in low-light areas and preservation of the original image in mid-light areas; M lowThis represents a low-light region mask, used to select low-light pixels for smoothing; ⊙ represents the element-wise multiplication operator. Represents the convolution operator; K smo This represents a Gaussian filter kernel used for smoothing and noise reduction in low-light areas. The kernel size is 5×5×3×3, meaning it has 3 input channels and 3 output channels to ensure effective smoothing filtering of color images. In practical implementation... The term represents the low-light image I in For each spatial location, a cross-channel weighted sum is performed within a 5×5 neighborhood, with the weights determined by a Gaussian function; and... Item and M mid ⊙I in When performing element-wise multiplication, the low-light area mask and the medium-light area mask will be automatically applied to the three RGB channels through a broadcast mechanism.

[0070] It should be noted that, The term involves performing Gaussian smoothing convolution operations on low-light regions to suppress noise. The characterization only applies smoothing results to low-light areas, while M mid ⊙I in The feature retains the original values ​​in the mid-illuminance region, avoiding unnecessary smoothing in the mid-illuminance region, thus preserving details, while reducing noise amplification in the low-illuminance region.

[0071] S3, Data Augmentation Model Construction and Training

[0072] I. Construction of a Two-Branch Feature Extraction Network

[0073] Low-light images often face the problem of global contrast degradation and local texture blurring at the same time. Conventional single-path convolutional neural networks are difficult to effectively extract both global illumination context information and local high-frequency texture details.

[0074] This invention employs a dual-branch encoder structure, comprising a global branch and a local branch, to jointly model the correlation between illumination and texture in an image. The specific steps are as follows:

[0075] 1. Global branch feature extraction

[0076] A global branch network is constructed using dilated convolutional layers. By setting different dilation rates, the receptive field is expanded to capture a wide range of illumination context information from the input preprocessed image. The global branch network outputs a global branch feature map, which carries the global contrast information of the image, represented as follows:

[0077]

[0078] In the formula, F gbThis represents the global branch feature map, which carries the global contrast information of the image. Its dimensions are H / 4×W / 4×C; C is the number of feature channels. An empirical value of C=64 is set to achieve a balance between computational complexity and feature representation capability. The network function representing the global branch is defined by the parameter θ. g Definition; θ g The network parameters for the global branch are trainable parameters, including the weights and biases of the dilated convolutional layers; r represents the dilation rate, preferably [2, 4, 8], used to control the sampling interval of the convolutional kernels and expand the receptive field; d il This represents the hyperparameters of dilated convolution, used to set the spacing of the convolution kernels.

[0079] In its implementation, the global branch network is a network structure composed of dilated convolutional layers, specifically including three dilated convolutional layers. Each dilated convolutional layer is followed by BatchNorm and ReLU activation functions to capture global illumination context information.

[0080] 2. Local branch feature extraction

[0081] A local branch network is constructed using compact convolutional layers. Downsampling is performed by setting a specific stride to extract high-frequency texture details from the input preprocessed image. The local branch network outputs a local branch feature map, which carries the local texture detail information of the image, represented as:

[0082]

[0083] In the formula, F ld This represents a local branch feature map, used to carry local texture details of the image, with dimensions of H / 2×W / 2×C; The network function representing a local branch is defined by the parameter θ. d Definition; θ d The network parameters for the local branches are trainable parameters, including the weights and biases of the convolutional layers; s represents the convolution stride, preferably [2, 1, 1], used to control the downsampling rate of the feature maps; s tr The hyperparameter representing the convolution stride is used to control the step size of the sliding window.

[0084] In the specific implementation, the network function of the local branch is a network structure composed of compact convolutional layers, specifically including 3 convolutional layers. Each convolutional layer is followed by BatchNorm and ReLU activation functions, and downsampling is performed with a stride of 2 to capture high-frequency texture details.

[0085] II. Construction of Attention Enhancement Module

[0086] Conventional feature stitching or addition operations fail to consider the differences in importance of different image regions due to their illumination differences, and cannot adaptively adjust the fusion weights of global and local features according to the illumination distribution, which usually leads to insufficient recovery of details in dark areas or excessive smoothing in bright areas.

[0087] This invention constructs an attention enhancement module to achieve feature fusion for illumination perception, making dark areas more reliant on denoised global features, while bright areas enhance detailed features, achieving adaptive fusion. The specific steps are as follows:

[0088] 1. Illuminance Attention Map Generation

[0089] Based on the input low-light image, average and maximum brightness information are extracted through spatial average pooling and max pooling operations. After channel concatenation, convolution operations and a sigmoid activation function are applied to generate an illumination attention map, providing spatial weight allocation to guide feature fusion, as shown below:

[0090] A att =Sig(Conv 1×1 (AvgP(I in )⊕MaxP(I in )));

[0091] In the formula, A att This represents the illumination attention map, used to guide the spatial weight allocation for feature fusion, with values ​​ranging from [0,1]. H×W The dimension is H×W; Sig(·) is the Sigmoid activation function, used to compress the output to the range of 0-1; Conv 1×1 (·) represents a 1×1 convolution operation. 1×1 (AvgP(I in )⊕MaxP(I in The term )) is used for channel dimension transformation and feature fusion; AvgP(·) represents the spatial dimension average pooling operation, generating channel weights, AvgP(I in The term ) is used to extract the average brightness information of the image; ⊕ indicates the channel concatenation operation; MaxP(·) represents the spatial dimension max pooling operation, generating channel weights, MaxP(I in The item is used to extract the maximum brightness information of the image.

[0092] 2. Feature upsampling alignment

[0093] For dimension Global branch feature map F gb and dimension are Local branch feature map F ld Bilinear upsampling is performed separately, and the global branch feature map F gbUpsampling by 4 times, local branch feature map F ld Upsample by 2 times, unifying the spatial dimensions of both images to the original image size H×W×C, while keeping the number of channels C=64 unchanged.

[0094] 3. Attention-weighted feature fusion

[0095] Upsampling is performed on the global and local branch feature maps to align their spatial dimensions. Then, element-wise weighted fusion is performed using the illumination attention map and its complement. The illumination attention map is applied to the channels via a broadcast mechanism to achieve adaptive fusion of global contrast features and local detail features, generating a fused feature map, represented as follows:

[0096] F fuse =A att ⊙Up(F gb )+(1A att -A att )⊙Up(F ld );

[0097] In the formula, F fuse This represents a fused feature map with dimensions H×W×C. It adaptively fuses global contrast features and local detail features using an illumination attention map, enhancing noise reduction in dark areas while preserving more detail in bright areas; Up(F gb Up(F) represents the upsampled global feature map, which is the output feature map after upsampling the global branch feature map and carries the global contrast information of the image; ld ) represents upsampled local features, which is the output feature map after upsampling the local branch feature map, carrying the detailed texture information of the image; 1A att Representation of attentional diagram A based on illuminance att A matrix of all 1s with the same dimensions, having dimensions H×W, (1A) att -A att The term represents the complement of the attention map.

[0098] In specific implementation, A att ⊙Up(F gb ) item and (1A att -A att )⊙Up(F ld When performing element-wise multiplication of item A, the illuminance attention map A att It will be automatically applied to C channels via a broadcast mechanism.

[0099] III. Reconstruction Module

[0100] In the reconstruction stage of low-light image enhancement, conventional methods are prone to oversmoothing, which can easily lead to loss of image details and color distortion, making it difficult to maintain both the natural look and richness of detail of the image at the same time.

[0101] This invention employs a cascaded residual group structure for reconstruction, thereby generating an enhanced image. The specific steps are as follows:

[0102] 1. Cascaded residual calculation

[0103] By cascading multiple residual blocks, the feature representation is gradually refined, enhancing the model's expressive power and gradient propagation, as shown below:

[0104] R k =R k-1 +Conv 3×3 (LR(Conv 1×1 (R k-1 )));

[0105] In the formula, R k This represents the output feature of the k-th residual block, carrying a progressively refined feature representation; that is, the feature quality is gradually improved through residual blocks, enhancing details and structure. k-1 This represents the output feature of the (k-1)th residual block. When k=1, the output feature R0 of the 0th residual block represents the initial feature, which is derived from F. fuse The result is obtained through a 3×3 convolution, i.e., R0 = Conv 3×3 (F fuse Conv 3×3 (·) indicates a 3×3 convolution operation, specifying 3 output channels in the last convolutional layer to ensure image enhancement. enh The number of channels is 3; k is the residual block index, and k = 1, ..., K; LR(·) represents the LeakyReLU activation function, which implements a nonlinear transformation; Conv 1×1 represents a 1×1 convolution operation, used for cross-channel interaction and feature compression; K represents the number of residual blocks, preferably 4, which controls the depth of the network.

[0106] It should be noted that 1×1 convolutions are used for cross-channel interaction and feature compression, 3×3 convolutions are used for spatial feature extraction, LeakyReLU provides non-linearity, and the input and output are added through skip connections. Residual connections help gradient flow, avoid gradient vanishing and preserve details, while avoiding over-smoothing, thus achieving true color restoration and detail enhancement.

[0107] 2. Enhanced image generation

[0108] The fused feature map is mapped to the image space through a 3x3 convolution operation and combined with the preprocessed image to generate an enhanced image. This process preserves the detailed information of the preprocessed image while utilizing deep features for reconstruction, as shown below:

[0109] I enh =Conv 3×3 (R K)+I part ;

[0110] In the formula, I enh This represents an enhanced image with dimensions H×W×3;

[0111] R K This represents the output feature of the Kth residual block, which is the output feature of the last residual block. It carries a deep feature representation with dimensions H×W×C.

[0112] In the embodiment, the resulting image processed by the enhancement algorithm has improved brightness, enhanced contrast, and clearer details.

[0113] In this embodiment, the pixel brightness distribution of the original low-light image and the enhanced image is compared and shown. The low-light image shows a left-skewed distribution with dark pixels dominating, while the enhanced image has a more balanced distribution that extends to the right.

[0114] In this embodiment, the brightness distribution after equalization is further analyzed. According to the brightness distribution after histogram equalization, it can be found that the distribution is flatter and the contrast is enhanced.

[0115] Furthermore, the red, green, and blue color channels of the RGB color channel are displayed separately for the low-light image color channel. Under low-light conditions, the brightness of each channel is low. However, by enhancing the image color channel, the brightness of each color channel is improved, and the color information is richer and more accurate.

[0116] Furthermore, by extracting texture features through Gabor filters, and using Gabor filters in different directions (0° and 45°) to extract texture information, it can be seen that the texture features of the low-light image are weak and blurry, while the texture features of the enhanced image are clearer and richer. The multi-directional Gabor difference map shows the directional changes of the texture.

[0117] I. Loss Function Calculation

[0118] The conventional L1 loss function applies equal weights to all image regions, ignoring the fact that the human visual system is more sensitive to noise in dark areas. This can easily lead to the network failing to prioritize the quality of low-light areas, resulting in insufficient recovery of details in dark areas.

[0119] This invention employs a partitioned weighted loss function to optimize image quality, with particular emphasis on preserving edge details. The specific steps are as follows:

[0120] 1. Calculation of region-weighted L1 loss

[0121] Based on low-light and medium-light region masks, L1 loss is calculated by applying different weights to the pixel-level differences between the enhanced image and the real label image. Low-light regions are given higher weights to increase the penalty for dark area loss, while medium-light regions are given lower weights, as shown below:

[0122] L L1 =λ low ‖M low ⊙(I enh -I gt )‖1+λ mid ‖M mid ⊙(I enh -I gt )‖1;

[0123] In the formula, L L1 λ represents the region-weighted L1 loss, used to measure the pixel-level difference between the enhanced and real images; low This represents the weighting coefficient for low-light areas, with a preferred value of 0.7, used to increase the weight of dark area loss; I gt This represents the true labeled image, a high-quality image under normal illumination; ||·||1 represents the L1 norm; λ mid This represents the weighting coefficient for the medium illumination area, with a preferred value of 0.3, used to reduce the weighting of bright area loss.

[0124] 2. Calculation of structural similarity loss

[0125] Based on low-light and medium-light region masks, a structural similarity index is calculated for corresponding regions of the enhanced image and the ground truth labeled image. This index is then combined with region weights to generate a structural similarity loss, which measures the degree of preservation of image structural information. This loss is expressed as:

[0126] L ssim =1-[λ low ·SSIM(M low ⊙I enh M low ⊙I gt )+λ mid ·SSIM(M mid ⊙

[0127] I enh M mid ⊙I gt )];

[0128] In the formula, L ssim Represents structural similarity loss, used to measure the degree to which structural information of an image is preserved;

[0129] SSIM(·,·) represents the Structural Similarity Index function, which is used to evaluate the similarity between two images in terms of brightness, contrast, and structure.

[0130] 3. Calculation of total loss function

[0131] The region-weighted L1 loss and structural similarity loss are added together, and a gradient loss term is added to form the total loss function. The gradient loss is calculated using the Sobel operator to determine the gradient difference between the augmented image and the ground truth image, emphasizing edge detail preservation, and is expressed as:

[0132]

[0133] In the formula, Loss represents the total loss function, which is used for the overall optimization objective of network training; The gradient operator is represented, specifically computed using the Sobel operator; λ grad This is the gradient loss weight, and its preferred value is 0.1.

[0134] II. Illumination-aware learning rate scheduling

[0135] The convergence speed varies in different illumination areas, with dark areas typically converging more slowly. Conventional fixed learning rate scheduling methods cannot adaptively adjust the learning rate to adapt to this difference, resulting in low overall training efficiency.

[0136] This invention employs a region-sensitive learning rate scheduling mechanism to accelerate the optimization process for difficult samples in dark regions, thereby improving training efficiency and the quality of dark region enhancement. The specific steps are as follows:

[0137] 1. Pixel statistics for dark and medium-bright areas

[0138] Make the low-light area mask M low and medium illumination region mask M mid Calculate the number of non-zero elements in the mask of the low-light region, ‖M. low The number of non-zero elements in the 0 and medium illumination region masks, M mid ‖0,‖·‖0 represents the number of non-zero elements in the calculation mask, representing the pixel scale of each region.

[0139] 2. Calculation of Dark Area Ratio

[0140] Based on the number of non-zero elements in the low-light and medium-light region masks and the brightness variance of each region, the relative proportion of dark areas is calculated. This relative proportion quantifies the importance of low-light regions in the image and is used for learning rate adjustment, expressed as:

[0141]

[0142] In the formula, r represents the relative proportion of the dark area, used to quantify the importance of low-light areas in the image; Var(·) represents the calculation of the region's brightness variance, specifically by extracting the brightness values ​​corresponding to non-zero pixels and then calculating the variance of these brightness values, reflecting the degree of brightness variation in the region. A large variance indicates strong regional contrast, which may contain more details; ||M low ||0 represents the number of non-zero elements in the low-light region mask;|M mid ||0 represents the number of non-zero elements in the medium illumination region mask; ∈ represents a minimal constant, preferably 10. -5 This is used to prevent numerical instability when the denominator is zero.

[0143] 3. Calculation of effective learning rate

[0144] Based on the relative proportion of the dark area and the base learning rate, the effective learning rate is dynamically calculated using an illumination adjustment coefficient and a regulation factor. The effective learning rate is used for network parameter updates. When the proportion of the dark area is high, the learning rate is increased to accelerate convergence, as expressed below:

[0145] η eff =η base ·(0.5+0.5·r) α ;

[0146] In the formula, η eff η represents the effective learning rate, the actual step size used for updating network parameters; base α represents the base learning rate, preferably 0.001, which serves as the baseline value for learning rate scheduling; α represents the adjustment factor, preferably 0.5, used to control the sensitivity of the learning rate to the proportion of dark areas.

[0147] It should be noted that the (0.5+0.5·r) term represents the illumination adjustment coefficient, which is used to map the proportion of dark areas to the adjustment range of [0.5, 1.0] to ensure that the learning rate is not too low. That is, when r=0, the learning rate is 0.5 times the baseline, and when r=1, it is 1 times. When the proportion of dark areas is high, the learning rate increases to accelerate the convergence of difficult samples in dark areas, thereby achieving adaptive adjustment of the learning rate and improving training efficiency and dark area quality.

[0148] III. Trainable Parameter Update

[0149] In low-light image enhancement tasks, conventional optimization algorithms such as SGD or Adam use a uniform update strategy for all network parameters, ignoring the different effects of different parameters on dark area restoration. This can easily lead to parameters that contribute significantly to dark area restoration converging slowly or getting trapped in local optima.

[0150] This invention employs an illuminance-sensing adaptive parameter update mechanism to ensure that parameters crucial for dark area recovery receive a larger update magnitude and a better update direction. The specific steps are as follows:

[0151] 1. Parameter Importance Weight Calculation

[0152] Based on the illumination attention map and the proportion of dark areas, the importance weights of global branch parameters, local branch parameters, and attention enhancement and reconstruction module parameters are calculated. These importance weights integrate global statistics and local attention information to quantify the potential contribution of parameters to dark area recovery, expressed as:

[0153]

[0154] In the formula, The network parameter θ represents the global branch. g The importance weight is used to amplify the update priority of the dominant parameters of the dark region corresponding to the dilated convolutional layer; Network parameters θ representing local branches d Importance weights are used to enhance the optimization of parameters related to bright regions corresponding to compact convolutional layers; The network parameters θ represent the attention enhancement module and the reconstruction module. r Importance weights are combined with structural similarity loss gradients to maintain the naturalness of the reconstruction; A att (i,j) represents the element in the i-th row and i-th column of the illumination attention map; i is the row index; j is the column index; μ A The threshold for the mean of the attention map is calculated as follows: Used to distinguish between dark and bright areas, A att (i,j)≤μ A Characterizing the dark area, A att (i,j)>μ A Represents the bright area; I(·) is a conditional comparison indicator function, outputting 1 if the condition is true, and 0 otherwise; κ g κ is the global branch weight coefficient, preferably set to 0.5, used to enhance the influence of the dark area proportion r; d β is the local branch weight coefficient, preferably 0.3, used to balance the optimization of bright areas; β is the base value of attention enhancement and reconstruction weights, preferably 0.8, to ensure that the weights are not too small and to maintain the basic importance of the reconstruction module in the optimization process; γ is the gradient sensitivity factor, preferably 10, used to control the exponential decay of the gradient magnitude; exp(·) is the natural exponential function. For structural similarity loss L ssim Regarding the network parameters θ of the attention enhancement module and the reconstruction module r The gradient vector reflects the sensitivity of the parameters to structural recovery.

[0155] It should be noted that the network parameter θ of the global branch g Importance weight Item A is obtained through the average illuminance attention map.att Attention values ​​for pixels in the mid-dark area and superimposed with κ g The r term ensures that the global parameter weights increase when the proportion of dark areas is high, and the network parameters θ of local branches are also increased. d Importance weight The term focuses on the attention complement of pixels in the bright area, combined with κ. d The (1-r) term avoids ignoring bright areas; network parameters θ of the attention enhancement module and the reconstruction module. r Importance weight The term utilizes structural similarity loss L ssim The gradient decays exponentially, penalizing parameters with small gradient magnitudes to enhance edge structure recovery.

[0156] 2. Gradient direction correction and step size scaling

[0157] The corrected gradient vector is calculated based on the total loss gradient, parameter importance weights, and historical momentum. This is then combined with the effective learning rate and gradient variance estimation to generate a scaled gradient step vector. This scaled gradient step vector is used for parameter updates, achieving step size adjustment for illumination sensing, and is expressed as:

[0158]

[0159] In the formula, s t This represents the scaling gradient step size vector, used for parameter updates; The corrected gradient vector, which integrates importance weights and historical momentum, is calculated as follows: μ represents the momentum decay factor, preferably 0.9, which controls the proportion of the historical momentum vector in the current correction gradient; w represents the parameter importance weight vector, which is... and It is composed of parameter sets, with dimension |Θ|, and is used to scale the gradient parameter by parameter; g t Let represent the gradient vector of the total loss at the t-th iteration, with dimension |Θ|. It is the gradient vector of the total loss function with respect to the set of all trainable parameters at the t-th iteration, obtained through backpropagation; Θ represents the trainable parameters, which is the set of all trainable parameters, i.e., Θ = {θ}. g ,θ d ,θ r}, |Θ| represents the dimension of all trainable parameters; m t-1 Let m represent the momentum vector in the (t-1)th iteration. At t=1, the initial momentum vector m0=0, with dimension |Θ|, passing through m t =μm t-1 +(1-μ)g t Iterative update; m t Let represent the momentum vector of the t-th iteration; This represents the element-wise division operator; v tLet represent the gradient variance estimate vector for the t-th iteration, calculated as follows: At t=1, the initial gradient variance estimation vector v0=0, with dimension |Θ|; s VE This represents the variance decay factor, with a preferred value of 0.999.

[0160] It should be noted that g t The term ⊙w amplifies the gradient of the dark region-sensitive parameter through importance weights, μm t-1 The term introduces momentum to smooth noise and scales the gradient step size vector s. t Combined with the effective learning rate η eff Together, they achieve step size adjustment for illumination perception.

[0161] 3. Parameter update of regularization constraints

[0162] The trainable parameters are updated by applying a scaling gradient step vector, and a weight decay regularization term is added to prevent overfitting. This regularization term imposes a stronger penalty on parameters with large values, improving the model's generalization ability. This is expressed as:

[0163]

[0164] In the formula, Θ t Let Θ represent the trainable parameters in the t-th iteration. t-1 λ represents the trainable parameters for the (t-1)th iteration; reg This represents the regularization strength coefficient, with a preferred value of 10. -4 To prevent the model from overfitting the training data; sign(·) represents the sign function, which increments or decrements the sign of the output parameter by 1; ρ E This represents the regularization exponent, with a preferred value of 0.75, used to achieve non-uniform weight decay.

[0165] It should be noted that the gradient step size vector s is scaled. t Perform gradient descent. The term, as a regularization term, applies to |Θ t-1 When |>1, apply stronger penalties to large weights and reduce penalties to small weights to avoid excessive compression of dark area-related parameters, thereby improving the model's generalization ability.

[0166] IV. Model Iterative Training and Termination Condition Judgment

[0167] The model training adopts an iterative optimization approach, and the training is carried out in multiple iteration cycles based on the training dataset. In each iteration cycle, the training data is randomly shuffled and divided into mini-batches, which are then input into the network for forward propagation to calculate the enhanced image and loss function.

[0168] The loss function includes region-weighted L1 loss, structural similarity loss, and gradient loss. The gradient is calculated through backpropagation, and an illumination-aware adaptive parameter update mechanism is applied to adjust the network parameters, including the parameters of the global branch, local branch, and attention enhancement and reconstruction modules.

[0169] During training, the changes in training loss and validation loss are monitored in real time, and the model performance is evaluated periodically using the validation set.

[0170] The model stops iterating based on its performance on the validation set. Training is stopped early to prevent overfitting when the validation loss stops decreasing or starts to increase over several consecutive iterations (e.g., 10 iterations).

[0171] At the same time, a maximum number of iteration cycles (such as 100 iteration cycles) is set as a backup stopping condition to ensure that training is completed within a reasonable time.

[0172] The model checkpoint mechanism is enabled to save the model parameters with the least validation loss for final testing and deployment.

[0173] S4, UAV imaging data enhancement

[0174] After the model is trained, it is used to enhance low-light drone imaging data.

[0175] First, load the trained model parameters and initialize the network structure, including the bi-branch feature extraction, attention enhancement module, and reconstruction module.

[0176] For new low-light drone images, an illumination partition-based data preprocessing step is applied to generate an illumination mask and perform Gaussian smoothing noise reduction on the low-light areas to obtain a preprocessed image.

[0177] The preprocessed image is then input into the network, where global and local features are extracted by a dual-branch encoder, the attention enhancement module achieves feature fusion for illumination perception, and the reconstruction module generates an enhanced image.

[0178] The enhancement process preserves the original size and color space of the image, outputting an enhanced RGB image that significantly improves brightness, contrast, and detail clarity, while effectively suppressing noise and color distortion.

[0179] The enhanced images can be directly used for subsequent processing in drone applications, such as target detection, scene segmentation, or real-time monitoring, improving overall system performance.

[0180] In this embodiment, a performance comparison analysis of different methods in low-light image enhancement is conducted to evaluate the objective performance indicators of different low-light image enhancement methods. Five methods were compared in the experiment: conventional histogram equalization, multi-scale Retinex, deep learning-based LLNet, SID, and the illumination perception enhancement method proposed in this invention. The experiment used the same test dataset, containing drone images of various low-light scenes (dusk, night, cloudy, etc.). Quantitative evaluation was performed by calculating the peak signal-to-noise ratio (PSNR) (decibels) and the structural similarity index (dimensionless). The box plot on the left of the figure shows the PSNR index, and the box plot on the right shows the structural similarity index. The boxes represent 25%-75%. The data distribution is shown in the figure, with the median as the midline and the whisker lines indicating the data range. Experimental results show that the method of this invention significantly outperforms other methods in two key indicators. In the peak signal-to-noise ratio (PSNR) graph, the median position of the method of this invention is the highest (approximately 25 dB), and the bin positions are generally shifted upwards and more compactly distributed, indicating higher and more stable image enhancement quality. In contrast, the bin positions of the comparative methods, such as histogram equalization, are the lowest (approximately 15 dB) and the distribution is scattered, indicating that its enhancement effect is unstable. In the structural similarity graph, the bin position of the method of this invention is close to 0.88, significantly higher than other methods (the highest being approximately 0.72), and there are fewer outliers, proving that its enhancement results are closest to images under real lighting conditions in terms of structural detail, brightness, and contrast. Experimental results demonstrate that the illumination partitioning processing and dual-branch feature extraction mechanism of this invention can more effectively restore details in dark areas while maintaining the naturalness of bright areas.

Claims

1. A method for enhancing UAV imaging data in low-light scenarios, characterized in that, Includes the following steps: S1. UAV imaging data acquisition and training dataset construction: acquire raw low-light images in low-light scenes, and acquire high-quality images under normal illumination in the same scene as real label images to construct training data. S2. Data preprocessing based on illuminance partitioning: The original low-illuminance image is partitioned by illuminance to generate illuminance masks that identify different brightness areas. Based on the illuminance masks, noise reduction is performed on the low-illuminance areas and detail preservation is performed on the medium-illuminance areas to generate a preprocessed image. S3. Data augmentation model construction and training: Construct and train a data augmentation model, which includes a dual-branch feature extraction network, an attention enhancement module, and a reconstruction module. The training process adopts a partition weighted loss function and an adaptive update mechanism for illumination perception parameters. S4. UAV imaging data enhancement: Load the trained data enhancement model, perform the data preprocessing based on illumination partitioning on the new low-light UAV image in sequence, and process it through the data enhancement model to output the enhanced image.

2. The method for enhancing UAV imaging data in low-light scenarios according to claim 1, characterized in that, The data preprocessing based on illuminance partitioning in S2 includes: Generate low-light region masks and medium-light region masks based on the brightness information of low-light images; Gaussian smoothing convolution is applied to the low-light regions identified by the low-light region mask to suppress noise, and the original values ​​are preserved for the medium-light regions identified by the medium-light region mask to generate a preprocessed image.

3. The method for enhancing UAV imaging data in low-light scenarios according to claim 1, characterized in that, The construction of the dual-branch feature extraction network includes: The global branch uses dilated convolution to expand the receptive field to capture the global illumination context information of the image and outputs a global feature map. The local branch uses standard convolution to focus on local regions to extract high-frequency texture details of the image and outputs a local branch feature map.

4. The method for enhancing UAV imaging data in low-light scenarios according to claim 1, characterized in that, The attention enhancement module is used for: The original low-light image is subjected to parallel average pooling and max pooling operations in the spatial dimension. The pooling results are then concatenated and processed by convolution and activation functions to generate an illumination attention map that reflects the relative brightness of each spatial location. Then, the illumination attention map and its complement are used to weight the global feature map and the local branch feature map respectively to achieve feature fusion and generate a fused feature map.

5. The method for enhancing UAV imaging data in low-light scenarios according to claim 1, characterized in that, The reconstruction module is used for: The fused feature map is input into a cascaded residual set for stepwise feature optimization; The output of the residual group is then added to the preprocessed image, and the final enhanced image is generated through global residual concatenation.

6. The method for enhancing UAV imaging data in low-light scenarios according to claim 1, characterized in that, The data augmentation model in S3 is constructed and trained using a partition-weighted loss function. This partition-weighted loss function assigns higher weights to pixel-level differences and structural information loss in low-light areas, and lower weights to pixel-level differences and structural information loss in medium-light areas.

7. A method for enhancing UAV imaging data in low-light scenarios according to claim 6, characterized in that, The partition-weighted loss function includes a region-weighted loss, which is calculated as follows: L L1 =λ low ||M low ⊙(I enh -I gt )||1+λ mid ||M mid ⊙(I enh -I gt )||1; In the formula, L L1 M represents the region-weighted L1 loss, used to measure the pixel-level difference between the enhanced and real images; low M is the mask for the low-light region. mid For the medium illumination region mask; I enh The enhanced image is represented by ⊙; ⊙ represents the element-wise multiplication operator; λ represents the image itself. low This represents the weighting coefficient for low-light areas, used to increase the weight of dark area loss; I gt This represents the true labeled image, a high-quality image under normal illumination; ||·||1 represents the L1 norm; λ mid This represents the weighting coefficient for the medium illumination area, used to reduce the weighting of the bright area.

8. A method for enhancing UAV imaging data in low-light scenarios according to claim 1, characterized in that, The data augmentation model construction and training in S3 adopts illumination-aware learning rate scheduling, which dynamically calculates the effective learning rate based on the relative proportion of dark areas in the image. When the proportion of dark areas is high, the learning rate is increased.

9. A method for enhancing UAV imaging data in low-light scenarios according to claim 8, characterized in that, The effective learning rate is calculated as follows: or eff =the base ·(0.5+0.5·r) a ; In the formula, η eff This represents the effective learning rate, the actual step size used for updating network parameters; η base represents the base learning rate, which serves as the benchmark for learning rate scheduling; 'a' represents the adjustment factor, used to control the sensitivity of the learning rate to the proportion of dark areas; and 'r' represents the proportion of dark areas.

10. A method for enhancing UAV imaging data in low-light scenarios according to claim 1, characterized in that, The data augmentation model construction and training in S3 employs an illumination-aware adaptive parameter update mechanism, which includes: Based on the illumination attention map and the proportion of dark areas, the importance weights of global branch parameters, local branch parameters, and attention enhancement and reconstruction module parameters are calculated. The corrected gradient vector is calculated based on the total loss gradient, parameter importance weights, and historical momentum, and the scaled gradient step vector is generated by combining the effective learning rate and gradient variance estimation. The trainable parameters are updated using the scaling gradient step vector, and a weight decay regularization term is added to prevent overfitting.

Images