Dual-branch low-light unmanned aerial vehicle image enhancement method based on improved IAT

Abstract

The application provides a double-branch low-light unmanned aerial vehicle image enhancement method based on an improved IAT, and belongs to the technical fields of computer vision and image processing.The method comprises the following steps: obtaining a low-light unmanned aerial vehicle aerial image as an input image; performing local illumination enhancement processing on the input image; performing global color correction processing on the input image; fusing the preliminary enhanced image with a color correction matrix and a gamma parameter, and then correcting color deviation through an adaptive white balance module based on the gray world assumption to output a high-quality normal-light image.The application improves the existing IAT image enhancement model in view of the specific degradation problem of the unmanned aerial vehicle aerial image under low-light conditions, provides a high-quality low-light image enhancement scheme, solves the problems of uneven brightness, color distortion, detail loss and the like, simultaneously suppresses noise amplification, provides high-quality image data support for subsequent high-level computer vision tasks, and meets the actual application requirements of the unmanned aerial vehicle under low-light conditions.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image processing technology, and in particular to a dual-path low-light UAV image enhancement method based on improved IAT. Background Technology

[0002] Drone aerial photography is widely used in many fields due to its unique perspective, but harsh outdoor environments (especially low light conditions) can easily lead to degradation problems in the acquired images, such as low contrast, color deviation, uneven brightness, loss of detail, and halo artifacts, which hinder the development of subsequent high-level computer vision tasks.

[0003] Existing image enhancement methods are divided into traditional methods and deep learning methods: Traditional methods rely on assumptions and prior knowledge, have limited model capacity and lack data support, and have poor adaptability; although deep learning methods have advantages in accuracy, robustness and speed, they have shortcomings such as high computational complexity, lack of physical constraints (the generated results may be unnatural), and difficulty in taking into account detail enhancement and color correction with a single network structure, which cannot meet the actual needs of UAV low-light image enhancement. Summary of the Invention

[0004] This invention provides a dual-branch low-light UAV image enhancement method based on improved IAT. Addressing the unique degradation problem of UAV aerial images under low-light conditions, it improves existing IAT image enhancement models to provide a high-quality low-light image enhancement solution that solves problems such as uneven brightness, color distortion, and loss of detail. Simultaneously, it suppresses noise amplification, providing high-quality image data support for subsequent high-level computer vision tasks (such as target tracking), and meeting the practical application needs of UAVs in low-light scenarios.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: The dual-path low-light UAV image enhancement method based on improved IAT includes the following steps: Acquire low-light drone aerial images as input images; Local illumination enhancement processing is performed on the input image: initial feature extraction is performed through convolutional layers, the extracted features are input into the DCE-Net network for feature enhancement, and then processed through two independent paths via convolutional layers and activation functions respectively to generate multiplication coefficients for brightness adjustment and addition coefficients for contrast shift. The input image is then subjected to pixel-level adaptive adjustment using the multiplication coefficients and addition coefficients to obtain the preliminary enhanced image. Global color correction is performed on the input image: a feature map is obtained by downsampling and feature extraction through a convolutional embedding layer. The feature map is then flattened into a feature sequence after injecting positional information. The reduced key vector and value vector are obtained by the spatial reduction module. The key vector, value vector and trained preset query vector are input into the attention mechanism. After layer normalization and processing by a feedforward network with residual structure, a network adjustment amount is generated. The network adjustment amount is added to the base value that remains unchanged after training to obtain the color correction matrix and gamma parameters. The image is initially enhanced and then fused with a color correction matrix and gamma parameters. The color deviation is then corrected by an adaptive white balance module based on the gray-scale world hypothesis, resulting in a high-quality image with normal illumination.

[0006] In this specification, the DCE-Net network consists of multiple depthwise separable convolutional layers. Each depthwise separable convolutional layer is configured with an activation function and adopts a symmetrical skip connection method: the outputs of different layers in the network are concatenated according to preset rules and then sent to subsequent layers.

[0007] In this specification, the processing procedure of the depth-separable convolutional layer is as follows: first, spatial features are extracted through a depth convolutional layer, and then channel fusion is performed through a pointwise convolutional layer to reduce computational complexity and improve computational efficiency.

[0008] In this specification, the activation function used in the path for generating multiplication coefficients is used to restrict the multiplication coefficients to non-negative numbers, and the activation function used in the path for generating addition coefficients is used to control the adjustment range of addition coefficients to avoid over-enhancement or noise amplification.

[0009] In this specification, the convolutional embedding layer is used to progressively reduce the original resolution of the input image while increasing the feature dimension in order to extract global information related to illumination.

[0010] In this specification, the space reduction module is used to downsample the feature sequence, thereby improving the feature representation capability and reducing computational complexity while maintaining the global receptive field.

[0011] In this specification, the processing procedure of the adaptive white balance module is as follows: calculate the average pixel value of the red, green and blue channels of the input image, use the average pixel value of the green channel as a reference, calculate the correction gain coefficient of the other two channels, multiply the pixel value of each channel with the corresponding correction gain coefficient to complete the color correction, and the correction gain coefficient is limited to a preset range.

[0012] This manual also includes model training steps: degrading the normal lighting image dataset from the drone's perspective to obtain a "normal image-degraded image" paired dataset, which is then used in conjunction with an existing low-light image enhancement dataset for model training.

[0013] In this specification, the loss functions used in the training process include L1 loss, exposure control loss, color constancy loss, and illumination smoothing loss. The total loss is the sum of each loss value multiplied by its corresponding weight. L1 loss is used to ensure image structure consistency, exposure control loss is used to maintain appropriate image brightness, color constancy loss is used to reduce color distortion, and illumination smoothing loss is used to avoid abrupt changes in illumination.

[0014] In this specification, the local illumination enhancement processing restores local image details and balances brightness through feature extraction and pixel-level adaptive adjustment, while the global color correction processing achieves natural adjustment of the overall image tone through global context information capture and parameter optimization. The two processes work together to ensure that image details are not overly smoothed and to effectively suppress noise amplification, resulting in a smoother histogram distribution and better visual quality in the enhanced image.

[0015] In summary, the present invention has at least the following beneficial effects: the enhanced image brightness is smooth and natural, the color correction is accurate, the details such as edge texture are effectively restored, and the noise amplification phenomenon is significantly suppressed; it effectively solves the problems of unnatural generation results and difficulty in balancing detail and color correction in existing methods; it significantly improves the performance of upper-level computer vision tasks for UAVs in low-light scenes, enhances the robustness and accuracy of task execution, and adapts to the actual application scenarios of UAV low-light operations. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the low-light enhancement model structure based on IAT involved in this invention.

[0018] Figure 2 This is a schematic diagram of the DCE-Net network structure involved in this invention.

[0019] Figure 3 This is a schematic diagram of the local reinforcement branch involved in the present invention.

[0020] Figure 4 This is a schematic diagram of the convolutional embedding layer involved in this invention.

[0021] Figure 5 This is a schematic diagram of the spatially reduced attention (SRA) structure involved in this invention.

[0022] Figure 6 This is a schematic diagram of the feedforward network (FFN) structure involved in this invention.

[0023] Figure 7 This is a schematic diagram of the terminal output of the global correction branch involved in this invention.

[0024] Figure 8 This is a schematic diagram of the adaptive white balance module involved in this invention.

[0025] Figure 9 This is a schematic diagram of an example of the enhanced low-light tracking results involved in this invention.

[0026] Figure 10 This is a schematic diagram of Example 2 of the enhanced low-light tracking results involved in this invention.

[0027] Figure 11 This is a schematic diagram of Example 3 of the enhanced low-light tracking results involved in this invention. Detailed Implementation

[0028] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the embodiments of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0029] The following disclosure provides many different implementations or examples for carrying out different structures of the embodiments of the present invention. To simplify the disclosure of the embodiments of the present invention, specific examples of components and arrangements are described below. Of course, these are merely examples and are not intended to limit the embodiments of the present invention. Furthermore, reference numerals and / or reference letters may be repeated in different examples of the embodiments of the present invention; such repetition is for simplification and clarity and does not in itself indicate a relationship between the various implementations and / or arrangements discussed.

[0030] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0031] This embodiment provides a dual-path low-light UAV image enhancement method based on improved IAT, including the following steps: Acquire low-light drone aerial images as input images; Local illumination enhancement processing is performed on the input image: initial feature extraction is performed through convolutional layers, the extracted features are input into the DCE-Net network for feature enhancement, and then processed through two independent paths via convolutional layers and activation functions respectively to generate multiplication coefficients for brightness adjustment and addition coefficients for contrast shift. The input image is then subjected to pixel-level adaptive adjustment using the multiplication coefficients and addition coefficients to obtain the preliminary enhanced image. Global color correction is performed on the input image: a feature map is obtained by downsampling and feature extraction through a convolutional embedding layer. The feature map is then flattened into a feature sequence after injecting positional information. The reduced key vector and value vector are obtained by the spatial reduction module. The key vector, value vector and trained preset query vector are input into the attention mechanism. After layer normalization and processing by a feedforward network with residual structure, a network adjustment amount is generated. The network adjustment amount is added to the base value that remains unchanged after training to obtain the color correction matrix and gamma parameters. The image is initially enhanced and then fused with a color correction matrix and gamma parameters. The color deviation is then corrected by an adaptive white balance module based on the gray-scale world hypothesis, resulting in a high-quality image with normal illumination.

[0032] In some embodiments, the DCE-Net network consists of multiple depthwise separable convolutional layers, each configured with an activation function, and employs a symmetrical skip connection method: the outputs of different layers in the network are concatenated according to a preset rule and then fed into subsequent layers.

[0033] Symmetrical skip connections refer to establishing skip connections between symmetrical positions in a structure, forming a symmetrical feature fusion structure. Specifically, the DCE-Net network used in the local branches of this method has six convolutional layers, where the first layer fuses features with the penultimate layer (the sixth layer). Similarly, the second layer fuses features with the penultimate layer (the fifth layer), and the third layer fuses features with the penultimate layer (the fourth layer). Figure 2 As shown.

[0034] In some embodiments, the processing of the depth-separable convolutional layer is as follows: first, spatial features are extracted through a depth convolutional layer, and then channel fusion is performed through a pointwise convolutional layer to reduce computational complexity and improve computational efficiency.

[0035] In some embodiments, the activation function used in the path for generating multiplication coefficients is used to restrict the multiplication coefficients to non-negative numbers, and the activation function used in the path for generating addition coefficients is used to control the adjustment range of the addition coefficients to avoid over-enhancement or noise amplification.

[0036] In some embodiments, the convolutional embedding layer is used to progressively reduce the original resolution of the input image while increasing the feature dimension to extract global information related to illumination.

[0037] In some embodiments, the space reduction module is used to downsample the feature sequence to improve feature representation capability while maintaining the global receptive field and reducing computational complexity.

[0038] In some embodiments, the adaptive white balance module processes the following steps: calculating the average pixel values ​​of the red, green, and blue channels of the input image; using the average pixel value of the green channel as a benchmark, calculating the correction gain coefficients of the other two channels; multiplying the pixel values ​​of each channel by the corresponding correction gain coefficient to complete color correction; and limiting the correction gain coefficients to a preset range.

[0039] In some embodiments, a model training step is also included: degrading the normal lighting image dataset from the perspective of the UAV to obtain a "normal image-degraded image" paired dataset, which is then used together with an existing low-light image enhancement dataset for model training.

[0040] In some embodiments, the loss function used in the training process includes L1 loss, exposure control loss, color constancy loss, and illumination smoothing loss. The total loss is the sum of each loss value multiplied by its corresponding weight. L1 loss is used to ensure image structure consistency, exposure control loss is used to maintain appropriate image brightness, color constancy loss is used to reduce color distortion, and illumination smoothing loss is used to avoid abrupt changes in illumination.

[0041] In some embodiments, the local illumination enhancement process restores local image details and balances brightness through feature extraction and pixel-level adaptive adjustment, while the global color correction process achieves natural adjustment of the overall image tone through global context information capture and parameter optimization. The two processes work together to ensure that image details are not overly smoothed and to effectively suppress noise amplification, resulting in a smoother histogram distribution and better visual quality in the enhanced image.

[0042] The technical concept of this invention is as follows: This invention proposes a dual-branch collaborative enhancement model based on improved IAT, which achieves image enhancement through the organic combination of local illumination enhancement and global color correction. Local enhancement branch: Based on DCE-Net optimized by Zero-DCE++, after initial feature extraction by convolutional layers, pixel-level multiplication coefficients (brightness adjustment) and addition coefficients (contrast shift) are generated through structures such as depthwise separable convolution and symmetric skip connections to achieve adaptive enhancement of local details and brightness. Global correction branch: Based on the Spatial Reducing Attention (SRA) Transformer module, after downsampling and feature extraction through convolutional embedding layers, global context information is captured to generate color correction matrix and gamma parameters; An adaptive white balance module based on the gray-world hypothesis is additionally integrated to correct color deviations; the dual-branch working collaboratively balances detail preservation and noise suppression, ultimately outputting high-quality, normally lit images. Model training utilizes specific paired datasets and multiple loss functions for optimization, ensuring enhanced performance.

[0043] For brightness equalization, a local enhancement network based on DCE-Net was designed. This network adjusts low-light images by generating multiplicative and additive coefficients. To address potential color distortion and over-enhancement after enhancement, a global color correction branch is introduced. This branch employs a spatially reduced attention mechanism to capture global contextual information and learns the optimal... Correction parameters and color transformation matrix This achieves a natural adjustment to the overall color tone of the image. Simultaneously, combined with an adaptive white balance algorithm based on the gray-scale world assumption, it effectively corrects color deviations.

[0044] In terms of detail preservation, the edge texture information of the image is effectively enhanced through the collaborative work of deep convolutional structures in local branches and attention mechanisms in global branches. Local branches employ a hierarchical feature extraction strategy to gradually restore local details, while global branches highlight important regions through attention weights, preventing details from being overly smoothed. This dual-branch collaborative mechanism results in a smoother and more natural histogram distribution in the enhanced image, effectively suppressing noise amplification while improving visual quality.

[0045] IAT-based low-light enhancement model This invention aims to improve existing IAT image enhancement models to meet the needs of UAVs capturing high-quality images in low-light conditions. The model employs a dual-branch structure, enhancing low-light images through collaborative enhancement of local details and correction of global illumination. In the local enhancement branch, the input image first undergoes feature extraction via a convolutional layer, then enters the DCE-Net network to obtain enhanced feature maps. Finally, adjustment parameter coefficients are generated through two different paths. Addition coefficients A path passes through a convolutional layer and The activation function yields the multiplication coefficients used for brightness adjustment. The other path involves a convolutional layer and... Activation function processing yields additive coefficients used for contrast shifting. .

[0046] In the global correction branch, the input image undergoes feature extraction via two convolutional blocks, and its output is used as the key vector. Sum value vector , and the pre-trained query vector Together, they are input into the SRA-Transformer module, which uses spatial reduction attention to efficiently capture global contextual information of the image and outputs a preliminary color correction matrix. With gamma parameters Then, add it to the base value to obtain the final color correction matrix. With gamma parameters These baseline values ​​remain unchanged after training. Finally, the locally enhanced image is fused with the parameters output by the global correction, and then processed by an adaptive white balance post-processing module to output a high-quality, normally lit image. The improved IAT model is as follows. Figure 1 As shown.

[0047] Local enhancement branches based on Zero-DCE++ The low-light image first passes through a 3x3 convolutional layer. This convolutional layer has a kernel size of 3x3, a stride of 1, and padding of 1. Its function is to perform initial feature extraction, capturing local features from the low-light image to prepare for subsequent processing.

[0048] Subsequent feature enhancement is performed using the parameter estimation network DCE-Net. DCE-Net contains six convolutional layers, each composed of depthwise separable convolutional structures, with ReLU activation function for each layer. It employs symmetric skip connections: the outputs of layers 3 and 4 are concatenated before being fed into layer 5; the outputs of layers 2 and 5 are concatenated before being fed into layer 6; and the outputs of layers 1 and 6 are concatenated before being fed into the next stage. The construction of DCE-Net is as follows: Figure 2 .

[0049] In the depthwise separable convolutional module, the feature map first enters a set of 3x3 depthwise convolutional layers (3x3 kernel, stride 1, padding 1) for spatial feature extraction, and then enters a pointwise convolutional layer (1x1 kernel, stride 1, padding 0) for channel fusion. Through the above process, the depthwise convolutional layer ultimately achieves the goal of reducing computational load and improving computational efficiency.

[0050] The DCE-Net module significantly reduces computational complexity while maintaining the quality of extracted output features. Finally, the intermediate feature maps enhanced by the DCE-Net module are each passed through a 3×3 convolutional layer (3x3 kernel, stride 1, padding 1) to further fuse local spatial context information, and then each is activated by an activation function. and The multiplication coefficients used to predict pixel-level brightness adjustment are respectively. and the additive coefficients used for contrast offset This enables pixel-level adaptive enhancement of the input image.

[0051] Among them, after The final output multiplication coefficients after activation function processing It is restricted to a non-negative number to avoid color distortion caused by introducing negative values ​​during multiplication adjustments; among which, after... The final output of the addition coefficients after activation function processing Limiting it to ±1 ensures that the amplitude of the additive adjustment is controllable, preventing excessive enhancement or noise amplification. Local branches, such as... Figure 3 As shown.

[0052] Use the obtained multiplication coefficients and additive coefficients For the input image The initial enhancement is performed, and the specific formula is as shown in equation (1): (1) Transformer global correction branch based on spatial reduction attention In the global correction branch, the low-light image is first passed through a convolutional embedding layer to convert the input image into a feature map for downsampling and feature extraction. This convolutional embedding layer consists of: a convolutional layer (kernel size 3x3, stride 2, padding 1), a batch normalization layer, a GELU activation function, another convolutional layer (kernel size 3x3, stride 2, padding 1), and a batch normalization layer.

[0053] The role of convolutional embedding layers is to progressively downsample the original resolution of the input image to a lower resolution, while increasing the feature dimension to extract global information related to illumination. The construction of a convolutional embedding layer is as follows: Figure 4 .

[0054] The feature map then enters the SRA-Transformer module, starting with a positional embedding convolutional layer. This layer injects positional information through convolution operations, resulting in a 2D feature map of shape [B, H, W, C]. This positional embedding convolutional layer has a 3x3 kernel size, padding of 1, and a stride of 1. It is then flattened into a sequence [B, N, C], where... .

[0055] The core of the SRA-Transformer is to obtain the key vector after multi-scale spatial reduction through the spatial reduction module SR (Spatial reduction). Sum value vector To enhance the representational power of features and reduce computational complexity while maintaining the global receptive field, the structure of SRA is as follows: Figure 5 .

[0056] Specifically, after the feature sequence is injected with positional information and flattened, the sequence enters the SR module for downsampling processing, and then passes through a linear layer to output the reduced K and V. The sequence lengths of the reduced K and V are... Length times, of which This is the downsampling ratio, set to 16.

[0057] The reduced key K and value V, along with the trained preset parameters Q, are input into the multi-head attention system, with the number of attention heads set to 4. The feature dimension is 64, therefore the dimension d of the attention head is 16. The formula for calculating attention is shown in equation (2).

[0058] (2) The attention output is then processed by layer normalization and fed into a feedforward network (FFN) with a residual structure for further feature enhancement. Figure 6 The FFN is constructed as follows: a fully connected layer 1, a GELU activation function, and a fully connected layer 2. The output of the FFN is residually connected to the input to obtain the final output.

[0059] The output of the feedforward network first passes through a global average pooling layer, followed by two independent linear layers. Within each linear layer, the output goes through the following sequence: fully connected layer - GELU activation function - fully connected layer - Tanh activation function. This ultimately generates the network adjustment values ​​for the color correction matrix and gamma parameters. These network adjustment values ​​are compared to the pre-trained baseline values. and Add them together to output the color correction matrix. and gamma parameters .like Figure 7 .

[0060] Color correction matrix of global branch output and gamma parameters After initial reinforcement via local branches The components are then fused. Simultaneously, an adaptive white balance module is inserted between the color correction matrix and the gamma correction to address image color cast issues.

[0061] Adaptive white balance, based on the gray-world assumption, is a widely used automatic color correction method. Its core idea is that the average values ​​of the red, green, and blue color channels in an entire image should be balanced, ultimately converging into a neutral gray. When the ambient light source is colored, it causes a color cast in the overall image, meaning the overall response of a certain channel is enhanced. The goal of this algorithm is to calculate and compensate for this deviation, automatically correcting the image colors to make them appear closer to what is observed under standard white light. The adaptive white balance module's position in the enhancement model is as follows: Figure 8 As shown.

[0062] The standard implementation process is as follows. First, the algorithm calculates the average value of the pixels in the red, green, and blue channels of the image. Then, using the average value of the green channel pixels as a benchmark, it divides each average value by the average value of the other two channels to calculate the correction gain coefficient for the corresponding channel. Finally, each channel is multiplied by the corresponding correction gain coefficient to complete the correction. The entire process is driven entirely by the image's own data and requires no prior information. The relevant formulas are shown in equations (3) to (6): (3) (4) (5) (6) It is a matrix of pixel values ​​for each channel. It is the pixel average of each channel. These are the gain coefficients of each channel. In this invention, the green channel is used as the reference. This is a very small constant used to prevent excessive gain; in this invention, this value is set to 1e-6. Furthermore, this invention uses the `torch.clamp` function in the code to correct the gain coefficient. It is limited to the range [0,1].

[0063] The adaptive white balance module based on the grayscale world assumption designed in this invention does not have a set trigger condition. When the value of a certain channel in the RGB channels is too large or too small, the correction gain coefficient of that channel is adjusted. The value will decrease or increase accordingly to suppress or enhance the value of this channel. When the values ​​of the three channels are not significantly different, the final calculated correction gain coefficient... The value approaches 1, having minimal impact on each channel. Therefore, no triggering condition was designed.

[0064] The calculation formula for the enhanced image output by the model is shown in equation (7).

[0065] (7) Training process To better adapt to illumination enhancement tasks from a UAV perspective, this invention degrades the UAV123 target tracking dataset under normal illumination using methods such as darkening, color shifting, and noise addition, resulting in a paired dataset of "normal images - degraded images" from UAV123. The final training datasets used include LOLv1, LOLv2, and the degraded UAV123 dataset, with a total of 60 training rounds.

[0066] The loss functions used in training include: L1 loss, exposure control loss, color constancy loss, and illumination smoothing loss.

[0067] The purpose of L1 loss is to compare the pixel-level differences between the enhanced image and the high-quality image, ensuring that the output image is structurally close to the target. Its weight is 1.0.

[0068] The purpose of exposure control loss is to encourage enhanced images to maintain appropriate brightness levels. This is achieved by calculating the absolute deviation of the image from a preset mean (default 0.6). The weighting is 0.15.

[0069] The purpose of color constancy loss is to maintain the consistency between image color channels and reduce color distortion. This is achieved by calculating the sum of squares of the differences between channels. The weighting is 0.2.

[0070] The purpose of the illumination smoothing loss is to promote a smooth transition in the enhanced illumination map and avoid abrupt illumination changes. Its weight is 0.8.

[0071] These loss functions are used after each forward propagation during the training phase, i.e., when the model generates augmented images, the total loss is calculated and used for backpropagation. The total loss value is the product of each loss value and its respective weight, and the formula for calculating the total loss value function is Equation (8).

[0072] (8) Comparative experiment To verify that the method in this invention has a good enhancement effect on images captured by UAVs under low-light conditions and provides a clear performance improvement for higher-level tasks, this invention selects the UAV night tracking task dataset UAVDark135 for testing. First, three no-reference quality assessment methods are used to test the enhanced image quality of this algorithm and five other algorithms: LOE, BRISQUE, and NIQE. The LOE index value symbolizes the quality of the image's brightness perception; a smaller value indicates a smoother and more natural image brightness, while a larger value indicates obvious exposure errors. The BRISQUE index value reveals the consistency of image details and spatial structure before and after enhancement; a smaller value indicates higher consistency, and vice versa. The NIQE index value symbolizes the naturalness of the image; a smaller value indicates less image distortion, and vice versa. Distortion includes color shift, underexposure / overexposure, content distortion, and texture loss. The no-reference evaluation indices are shown in the table below, where bold data represents the optimal value and underlined data represents the suboptimal value: ; Meanwhile, to verify that the proposed enhancement algorithm can significantly improve the performance of upper-layer tracking tasks and improve tracking results in low-light scenes, this invention uses the OStrack target tracking model for testing. First, the proposed enhancement algorithm is used to process the UAVDark135 dataset, and then the OStrack target tracking model is used to track and detect the enhanced UAVDark135 dataset. Precision, success rate, and center error are selected as tracking evaluation metrics. Precision is the core metric for measuring the accuracy of single-frame tracking position, usually calculated based on a given overlap threshold (e.g., 0.5). A higher value indicates a higher degree of overlap between the predicted and ground truth boxes. Success rate assesses the overall robustness of tracking, calculated as the area under the curve of the percentage of frames with overlap exceeding different thresholds (0 to 1). A higher value indicates better stability of the tracker throughout the process. Center error directly reflects the deviation between the predicted and ground truth box centers using pixel distance; a smaller value indicates more accurate localization. The tracking evaluation metrics are shown in the table below: ; Some tracking results (examples 1, 2, and 3 of the enhanced low-light tracking results) are as follows: Figure 9 , Figure 10 and Figure 11 As shown.

[0073] In summary, the method of this invention has a good enhancement effect on images captured by UAVs under low-light conditions, achieving superior no-reference evaluation metrics. Furthermore, it significantly improves the performance of UAV upper-level target tracking in low-light environments.

[0074] The embodiments described above are for illustrative purposes only and are not intended to limit the invention. Therefore, any changes in numerical values ​​or substitutions of equivalent elements should still fall within the scope of this invention.

[0075] The above detailed description will enable those skilled in the art to understand that the present invention can indeed achieve the aforementioned objectives and has complied with the provisions of the present invention law.

[0076] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention. The above descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. It should be noted that any modifications, equivalent substitutions, and improvements made within the spirit and principles of the invention should be included within the scope of protection of the invention.

[0077] It should be noted that the above description of the process is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to the process under the guidance of this specification. However, these modifications and changes remain within the scope of this specification.

[0078] The basic concepts have been described above. Obviously, for those skilled in the art who have read this application, the above disclosure is merely illustrative and does not constitute a limitation of this application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and corrections to this application. Such modifications, improvements, and corrections are suggested in this application, and therefore, such modifications, improvements, and corrections still fall within the spirit and scope of the exemplary embodiments of this application.

[0079] Furthermore, this application uses specific terms to describe its embodiments. For example, "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that "an embodiment," "one embodiment," or "an alternative embodiment" mentioned twice or more in different positions in this specification do not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application can be appropriately combined.

[0080] Furthermore, those skilled in the art will understand that aspects of this application can be described and illustrated through several inventive kinds or situations, including any new and useful combination of processes, machines, products, or substances, or any new and useful improvements thereof. Therefore, aspects of this application can be implemented entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or a combination of hardware and software. All of the above hardware or software can be referred to as a "unit," "module," or "system." Furthermore, aspects of this application can take the form of a computer program product embodied in one or more computer-readable media, wherein computer-readable program code is contained therein.

[0081] The computer program code required for the operation of each part of this application can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, and Python; general programming languages ​​such as C; Visual Basic, Fortran2103, Perl, COBOL2102, PHP, and ABAP; dynamic programming languages ​​such as Python, Ruby, and Groovy; or other programming languages. This program code can run entirely on the user's computer, or as a standalone software package on the user's computer, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any network, such as a local area network (LAN) or wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as Software as a Service (SaaS).

[0082] Furthermore, unless expressly stated in the claims, the order of processing elements and sequences, the use of numbers and letters, or other names described in this application are not intended to limit the order of the processes and methods of this application. Although some currently considered useful embodiments of the invention have been discussed in the foregoing disclosure by way of various examples, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments; rather, the claims are intended to cover all modifications and equivalent combinations that conform to the substance and scope of the embodiments of this application. For example, although the implementation of the various components described above can be embodied in a hardware device, it can also be implemented as a purely software solution, such as an installation on an existing server or mobile device.

[0083] Similarly, it should be noted that, in order to simplify the description of the present application and thus aid in the understanding of one or more embodiments of the invention, the foregoing description of the embodiments of the present application sometimes combines multiple features into a single embodiment, drawing, or description thereof. However, this approach of the present application should not be construed as reflecting an intention that the claimed subject matter requires more features than expressly recited in each claim. Rather, the subject of the invention should possess fewer features than in any single embodiment described above.

Claims

1. A dual-path low-light UAV image enhancement method based on improved IAT, characterized in that, include: Acquire low-light drone aerial images as input images; Local illumination enhancement processing is performed on the input image: initial feature extraction is performed through convolutional layers, the extracted features are input into the DCE-Net network for feature enhancement, and then processed through two independent paths via convolutional layers and activation functions respectively to generate multiplication coefficients for brightness adjustment and addition coefficients for contrast shift. The input image is then subjected to pixel-level adaptive adjustment using the multiplication coefficients and addition coefficients to obtain the preliminary enhanced image. Global color correction is performed on the input image: a feature map is obtained by downsampling and feature extraction through a convolutional embedding layer. The feature map is then flattened into a feature sequence after injecting positional information. The reduced key vector and value vector are obtained by the spatial reduction module. The key vector, value vector and trained preset query vector are input into the attention mechanism. After layer normalization and processing by a feedforward network with residual structure, a network adjustment amount is generated. The network adjustment amount is added to the base value that remains unchanged after training to obtain the color correction matrix and gamma parameters. The initially enhanced image is fused with the color correction matrix and gamma parameters, and then the color deviation is corrected by an adaptive white balance module based on the gray world hypothesis to output a normal lighting image.

2. The dual-branch low-light UAV image enhancement method based on improved IAT according to claim 1, characterized in that, The DCE-Net network consists of multiple depthwise separable convolutional layers, each configured with an activation function, and employs a symmetrical skip connection method: the outputs of different layers in the network are concatenated according to preset rules and then fed into subsequent layers.

3. The dual-branch low-light UAV image enhancement method based on improved IAT according to claim 2, characterized in that, The processing procedure of the depthwise separable convolutional layer is as follows: first, spatial features are extracted through a depthwise convolutional layer, and then channel fusion is performed through a pointwise convolutional layer to reduce computational complexity and improve computational efficiency.

4. The dual-branch low-light UAV image enhancement method based on improved IAT according to claim 1, characterized in that, The activation function used in the path that generates multiplication coefficients is used to restrict the multiplication coefficients to non-negative numbers, while the activation function used in the path that generates addition coefficients is used to control the adjustment range of addition coefficients to avoid over-enhancement or noise amplification.

5. The dual-branch low-light UAV image enhancement method based on improved IAT according to claim 1, characterized in that, The convolutional embedding layer is used to progressively reduce the original resolution of the input image while increasing the feature dimension to extract global information related to illumination.

6. The dual-branch low-light UAV image enhancement method based on improved IAT according to claim 1, characterized in that, The space reduction module is used to downsample the feature sequence, which improves the feature representation ability and reduces computational complexity while maintaining the global receptive field.

7. The dual-branch low-light UAV image enhancement method based on improved IAT according to claim 1, characterized in that, The adaptive white balance module processes the following steps: calculate the average pixel values ​​of the red, green, and blue channels of the input image; use the average pixel value of the green channel as a benchmark to calculate the correction gain coefficients of the other two channels; multiply the pixel values ​​of each channel by the corresponding correction gain coefficients to complete color correction; and limit the correction gain coefficients to a preset range.

8. The dual-branch low-light UAV image enhancement method based on improved IAT according to claim 1, characterized in that, It also includes a model training step: degrading the normal lighting image dataset from the drone's perspective to obtain a normal image-degraded image paired dataset, which is then combined with the existing low-light image enhancement dataset for model training.

9. The dual-branch low-light UAV image enhancement method based on improved IAT according to claim 8, characterized in that, The loss functions used in the training process include L1 loss, exposure control loss, color constancy loss, and illumination smoothing loss. The total loss is the sum of each loss value multiplied by its corresponding weight. L1 loss is used to ensure image structure consistency, exposure control loss is used to maintain appropriate image brightness, color constancy loss is used to reduce color distortion, and illumination smoothing loss is used to avoid abrupt changes in illumination.

10. The dual-branch low-light UAV image enhancement method based on improved IAT according to claim 1, characterized in that, The local illumination enhancement process restores local image details and balances brightness through feature extraction and pixel-level adaptive adjustment, while the global color correction process achieves natural adjustment of the overall image tone through global context information capture and parameter optimization.

Images