Low-illumination unmanned aerial vehicle target tracking system and method

Abstract

The application relates to a low-illumination unmanned aerial vehicle target tracking system and method, which comprises a target tracking module obtained through field self-adaption training, is used for processing and analyzing low-illumination complex dynamic images, and outputs a target tracking result; the target tracking module comprises a pre-trained backbone network and a tracking result output network, the two are connected with an illumination level-oriented double-path interactive feature enhancement network, a time sequence optimization and enhancement layer, and a field self-adaption scoring network and a dynamic sparse and semantic constraint alignment network are arranged; after the field self-adaption training process is completed, model compression is completed through a field adaptation sparse distillation lightweight mechanism, so that the target tracking module is deployed to an industrial-grade unmanned aerial vehicle edge visual camera. Compared with the prior art, the application can solve the problems of tracking drift and interruption caused by illumination interference, day-night cross-domain feature deviation and motion blur in the low-illumination complex dynamic scene, and can also consider the tracking performance and lightweight deployment requirements.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) image processing technology, and in particular to a low-light UAV target tracking system and method. Background Technology

[0002] In the fields of computer vision and UAV technology, target tracking in complex dynamic scenes with low light has always been a very challenging task. Existing UAV tracking algorithms have made significant progress under normal lighting conditions, but when facing nighttime or low-light environments, they often face difficulties in extracting target features due to severe image degradation, extremely low contrast, and a surge in sensor noise.

[0003] Currently, academia and industry often attempt to alleviate this problem by introducing image enhancement techniques or cross-domain learning methods. However, existing image enhancement methods often ignore the hierarchical lighting interference caused by altitude changes during UAV flight (such as low-altitude shadow occlusion and high-altitude atmospheric scattering), and lack the ability to adaptively compensate for complex conditions such as local overexposure, underexposure, and occlusion. This makes trackers prone to confidence fluctuations or even tracking interruptions when dealing with such extreme scenarios. On the other hand, tracking in low-light scenes inherently involves day-night cross-domain feature shift. Although traditional domain adaptation methods attempt to reduce the distribution difference between the source domain (labeled normal illumination data) and the target domain (unlabeled low-light data), most lack a refined semantic constraint alignment mechanism, which easily leads to semantic shifts during cross-domain transformation. In addition, UAVs are often affected by wind interference in actual operations. The dynamic shaking of the fuselage and the long exposure effect under low light intertwine, resulting in severe motion blur. Most existing algorithms lack the ability to combine onboard sensor (such as gyroscope) data for temporal calibration, making it difficult to maintain the continuity of features between frames. Furthermore, at the hardware deployment level, high-performance cross-domain alignment models are often structurally redundant, making it difficult to balance high precision and real-time requirements on the edge-embedded NPU (Neural Processing Unit) platform of drones with limited computing resources. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a low-light UAV target tracking system and method to solve the problems of UAV target tracking being easily affected by illumination interference, day and night cross-domain feature shift, and tracking drift and interruption caused by motion blur in complex dynamic low-light scenarios, while taking into account both tracking performance and lightweight deployment requirements.

[0005] The objective of this invention can be achieved through the following technical solution: a low-light drone target tracking system, including a target tracking module deployed in an industrial-grade drone edge vision camera, wherein the drone edge vision camera is used to acquire low-light complex dynamic images, the target tracking module is used to process and analyze the low-light complex dynamic images, and output target tracking results, wherein the target tracking module is obtained through domain adaptive training. The target tracking module includes a pre-trained backbone network and a tracking result output network. The backbone network and the tracking result output network are sequentially connected by an illumination-level guided dual-path interactive feature enhancement network and a temporal optimization and enhancement layer. The illumination-level guided dual-path interactive feature enhancement network and the temporal optimization and enhancement layer are respectively connected to a domain adaptive scoring network. The temporal optimization and enhancement layer and the tracking result output network are connected by a dynamic sparsity and semantic constraint alignment network. In the domain adaptive training process, the labeled UAV normal illumination dataset is used as the source domain and the unlabeled UAV low illumination dataset is used as the target domain. First, the parameters of the backbone network and the tracking result output network that have been trained on the labeled UAV normal illumination dataset are frozen. The source domain and the target domain are used as datasets to train the illumination level-guided dual-path interactive feature enhancement network, the temporal optimization and enhancement layer. The illumination level-guided dual-path interactive feature enhancement network is used to extract the inherent reflection features of the target and model the global illumination interference. Combined with the physical constraint dynamic sparse operator, non-target interference signals are eliminated. The temporal optimization and enhancement layer is used to adaptively compensate for illumination layer features and complete inter-frame brightness correction, motion jitter calibration and target blur restoration, outputting source domain features and target domain features; The domain adaptive scoring network is used to decode source domain features and target domain features to extract key parameters for multi-index scoring, and feeds the multi-index scores back to the illumination level-guided dual-path interactive feature enhancement network, temporal optimization and enhancement layer to achieve reverse optimization; The Dynamic Sparse and Semantic Constraint Alignment Network is used to minimize the maximum mean difference in feature distributions between the source and target domains and the semantic consistency loss, and introduces the target class prototype distance constraint to avoid semantic shift. After the domain-adaptive training process is completed, the model is compressed through the domain-adaptive sparse distillation lightweight mechanism so that the target tracking module can be deployed to the edge vision camera of an industrial-grade drone in real time.

[0006] Furthermore, the illumination-level guided dual-path interactive feature enhancement network includes an illumination-level dual-domain adaptation module and heterogeneous reflection component extraction branches and illumination constraint state space branches. The reflection component extraction branches extract the inherent reflection features of the target through RMDConv (Residual Multi-Scale Dynamic Convolution) and reflection feature purification operations. The illumination-constrained state space branch utilizes RMDConv to dynamically adjust the state equation parameters based on the illumination hierarchy, thereby achieving global illumination interference modeling with linear complexity. The illumination-level dual-domain adaptation module is used to perform dual-path feature differentiation fusion, in conjunction with a physical constraint dynamic sparse operator, to eliminate non-target interference signals under low illumination.

[0007] Furthermore, the RMDConv adopts a multi-path parallel and dynamic fusion structure design, in which the input features are simultaneously connected to three branches, and feature extraction at different scales is achieved through 1×1 convolution and asymmetric convolution respectively. The output of each branch is then dynamically convolved to complete the adaptive feature representation of the input. Subsequently, the multi-branch features are concatenated and integrated, then residual summation is performed with the original input features, and finally the final feature transformation is completed through depthwise separable convolution.

[0008] Furthermore, the illumination hierarchy dual-domain adaptation module adopts a dual-branch adaptation architecture of illumination base layer and illumination variation layer. The illumination base layer is used to extract general target features across illumination conditions, and the illumination variation layer is used to dynamically capture hierarchical illumination interference corresponding to different flight altitudes of the UAV, including low-altitude near-ground shadow modeling and high-altitude atmospheric scattering modeling. The illumination-level dual-domain adaptation module also includes an illumination gradient weight allocator to perform weighted fusion of the dual-branch features, and then uses a feature decoupling fusion step to separate the strong coupling between the target core features and illumination interference.

[0009] Furthermore, the temporal optimization and enhancement layer is equipped with a hierarchical feature adaptive compensation network and a wind disturbance illumination compensation temporal calibration network. The hierarchical feature adaptive compensation network is equipped with an illumination hierarchical feature prototype library. The output features of the illumination hierarchical feature prototype library are simultaneously split into two parallel processing paths. One path first enhances the feature representation of the weak light region through low-level weak light gain enhancement, while the other path reduces the redundant interference of the strong light region through high-level strong light noise suppression. Subsequently, the outputs of the two parallel processing paths are sequentially connected to the occlusion perception attention module and the adaptive compensation module. Finally, the hierarchical feature integration module merges the processing results of the two parallel processing paths into a complete feature. The working process of the wind disturbance and illumination compensation timing calibration network includes: Wind disturbance features are extracted based on real-time data from the UAV gyroscope, and dynamic optical flow propagation is performed on the input image features. The two work together to drive the generation of the illumination fluctuation compensation factor. The inter-frame brightness difference is first corrected using the illumination fluctuation compensation factor, and then motion jitter calibration and target blur restoration are completed sequentially.

[0010] Furthermore, the domain adaptive scoring network uses a basic large model as its core carrier. First, it extracts key quantitative parameters, including illumination level factor, dynamic sparsity index, semantic consistency score, and feature distribution distance, through a dual-domain feature multi-dimensional encoding and decoding module. Then, it performs multi-index scoring from the dimensions of illumination level, feature adaptation, feature compensation, and temporal calibration. The multi-index scoring results are fed back to the illumination level-guided dual-path interactive feature enhancement network and the temporal optimization and enhancement layer to achieve reverse optimization.

[0011] Furthermore, the dual-domain feature multi-dimensional encoding and decoding module is configured with a source domain latent space representation layer, a target domain latent space representation layer, a dual-domain feature joint mapping layer, a dual-domain contrastive learning encoding layer, and a multi-branch feature quantization decoding layer. The source domain features and target domain features are respectively input to the source domain latent space representation layer and the target domain latent space representation layer to extract the source domain latent space expression features and the target domain latent space expression features respectively. Then, the dual-domain feature joint mapping layer realizes the correlation and fusion of dual-domain features. Subsequently, the dual-domain contrastive learning encoding layer completes the contrastive encoding of features. Finally, the multiple dedicated branches of the multi-branch feature quantization decoding layer output the illumination level factor, dynamic sparsity index, semantic consistency score, and feature distribution distance respectively.

[0012] Furthermore, the dynamic sparse and semantic constraint alignment network adopts a cross-domain alignment architecture with dynamic sparse guidance and semantic constraint reinforcement. The dynamic sparse guidance is used to filter key features and suppress redundant information. The semantic constraint enhancement specifically involves jointly minimizing the maximum mean difference between the feature distributions of the source and target domains and the semantic consistency loss, and introducing a target category prototype distance constraint term to avoid semantic offset issues in cross-domain feature transformation.

[0013] Furthermore, the domain-adaptive sparse distillation lightweight mechanism specifically achieves model compression through domain-invariant parameter sharing, plug-in encapsulation, channel pruning, and parameter distillation techniques.

[0014] A low-light UAV target tracking method, which runs independently within the UAV edge vision camera, includes the following steps: For low-light complex dynamic images acquired by the airborne edge vision camera, the basic image representation features are first obtained through backbone network processing. Then, the illumination hierarchy-guided dual-path interactive feature enhancement network extracts the inherent reflection features of the target and models global illumination interference, as well as enhances the target features and removes illumination interference; Then, timing optimization and enhancement layers were used to solve overexposure, underexposure and occlusion problems in low light, as well as to complete inter-frame brightness correction, motion jitter calibration and target blur restoration; Finally, the target tracking result is obtained by outputting the tracking result through the network.

[0015] Compared with the prior art, the present invention has the following advantages: This invention deploys a target tracking module within the edge vision camera of an industrial-grade UAV. This target tracking module is trained through domain-adaptive learning and includes a pre-trained backbone network and a tracking result output network. The backbone network and the tracking result output network are sequentially connected to an illumination-level guided dual-path interactive feature enhancement network and a temporal optimization and enhancement layer. Furthermore, the illumination-level guided dual-path interactive feature enhancement network and the temporal optimization and enhancement layer are each connected to a domain-adaptive scoring network. A dynamic sparsity and semantic constraint alignment network connects the temporal optimization and enhancement layer to the tracking result output network. In the domain-adaptive training process, a labeled UAV normal illumination dataset is used as the source domain, and an unlabeled UAV low illumination dataset is used as the target domain. First, the parameters of the backbone network and the tracking result output network, already trained on the labeled UAV normal illumination dataset, are frozen. The source and target domains are then used as datasets to train an illumination-level guided dual-path interactive feature enhancement network, temporal optimization, and enhancement layers. After the domain-adaptive training process is completed, a domain-adaptive sparse distillation lightweight mechanism is used to compress the model, enabling real-time deployment of the target tracking module to the industrial-grade UAV edge vision camera. This achieves a complete solution encompassing feature enhancement, domain adaptation, compensation calibration, closed-loop scoring optimization, and lightweight deployment, effectively improving the accuracy and stability of UAV target tracking in complex dynamic low-light scenarios while meeting the real-time and lightweight requirements of UAV edge deployment.

[0016] This invention designs a lighting-level guided dual-path interactive feature enhancement network. The dual paths are divided into a reflection component extraction branch and an lighting-constrained state space branch. The reflection component extraction branch extracts the inherent reflection features (shape and texture) of the target through operations such as novel residual multi-scale dynamic convolution RMDConv and reflection feature purification. The lighting-constrained state space branch uses RMDConv to dynamically adjust the state equation parameters based on the lighting level to achieve linear complexity global lighting interference modeling (such as light source flicker and halo diffusion). Then, the lighting-level dual-domain adaptation module realizes the differential fusion of features from the two paths. Combined with a physically constrained dynamic sparse operator, it can accurately remove non-target interference signals under low illumination, solve the feature confusion problem in low illumination environment, improve the accuracy of core target feature extraction, and provide a high-purity feature foundation for subsequent processing.

[0017] This invention proposes a residual multi-scale dynamic convolution RMDConv, which adopts a multi-path parallel and dynamic fusion structure design: input features are simultaneously connected to three branches, and feature extraction at different scales is achieved through 1×1 convolution and asymmetric convolution, respectively. The outputs of each branch are then subjected to dynamic convolution to complete the adaptive feature representation of the input. Subsequently, the multi-branch features are concatenated and integrated, followed by residual summation with the original input features, and finally the final feature transformation is completed through depthwise separable convolution. This structure not only covers local and global features through multi-scale branches, but also achieves dynamic feature adaptation through dynamic convolution. At the same time, by combining residual connections and depthwise separable convolution, it can effectively reduce the number of parameters and computational costs while ensuring information integrity and representation flexibility, and takes into account the performance and lightweight requirements of the model.

[0018] This invention constructs a dual-branch adaptation architecture in the illumination hierarchy dual-domain adaptation module, comprising an illumination base layer and an illumination variation layer. The illumination base layer focuses on extracting general target features across illumination conditions, ensuring the consistency of core features under different illumination scenarios. The illumination variation layer dynamically captures hierarchical illumination interference corresponding to different UAV flight altitudes, such as low-altitude near-ground shadow occlusion and high-altitude atmospheric scattering effects. A precise weighted fusion of the dual-branch features is performed through an illumination gradient weight assigner. Finally, through a feature decoupling and fusion stage, the strong coupling between the target core features and illumination interference is effectively separated. This illumination hierarchy dual-domain adaptation module can adaptively match the dynamic changes in illumination hierarchy under complex UAV flight scenarios, providing cleaner and more robust target feature support for subsequent feature processing and tracking result output networks.

[0019] This invention sets up a hierarchical feature adaptive compensation network and a wind disturbance illumination compensation temporal calibration network in the temporal optimization and enhancement layer. The core architecture of the hierarchical feature adaptive compensation network revolves around an illumination hierarchical feature prototype library: the output features of the illumination hierarchical feature prototype library are simultaneously split into two parallel processing paths. One path first enhances the feature representation of the weak light region through low-level weak light gain enhancement, while the other path reduces the redundant interference of the strong light region through high-level strong light noise suppression. Subsequently, the outputs of both paths are sequentially connected to the occlusion perception attention module to focus on effective features and filter out invalid information in the occluded region. Then, the features are specifically repaired and enhanced by the adaptive compensation module. Finally, the hierarchical feature integration module merges the processing results of the two paths into a complete feature. This achieves differentiated processing of illumination hierarchical features. Combined with the occlusion perception and adaptive compensation mechanism, it can accurately optimize the feature quality of different illumination regions and efficiently integrate hierarchical information, greatly improving the feature representation effect under complex illumination and occlusion scenes. This solves problems such as local overexposure, underexposure, and occlusion under low light, providing highly recognizable features for tracking and inference. The wind disturbance and illumination compensation temporal calibration network integrates wind field perception, optical flow propagation, and temporal calibration. On the one hand, it extracts wind disturbance features based on real-time data from the UAV gyroscope, and on the other hand, it performs dynamic optical flow propagation on the input image features. The two work together to drive the generation of illumination fluctuation compensation factors. Then, with the illumination fluctuation compensation factor as the core, it first corrects the brightness difference between frames, and then completes motion jitter calibration and target blur restoration in sequence. This can not only reduce the interference of wind disturbance and illumination fluctuation on inter-frame features, but also alleviate the motion blur problem of targets in UAV dynamic flight, effectively reduce the propagation deviation of inter-frame features, and significantly improve the continuity of temporal features in low-light dynamic scenes.

[0020] This invention designs a dynamic sparse and semantically constrained alignment network, constructing a cross-domain alignment architecture guided by dynamic sparseness and reinforced by semantic constraints. Based on the core method of domain adaptation, a dynamic sparse feature guidance mechanism is designed to provide an efficient feature foundation for domain alignment by accurately selecting key features and suppressing redundant information. Simultaneously, during the optimization process, the maximum mean difference in feature distribution between the source domain (normal illumination data) and the target domain (low illumination data) and the semantic consistency loss are jointly minimized. Furthermore, a target category prototype distance constraint term is introduced to effectively avoid semantic shift problems in cross-domain feature transformation. This dynamic sparse and semantically constrained alignment network inherits the results of previous feature extraction and domain adaptation, specifically addressing the tracking drift problem caused by feature distribution shift in low illumination scenarios, and achieving precise control of tracking confidence fluctuations, ultimately achieving accurate alignment of features between the source and target domains.

[0021] This invention designs a domain-adaptive scoring network based on a fundamental large model. This network uses the fundamental large model as its core carrier. First, it extracts key quantitative parameters such as illumination level factors, dynamic sparsity index, semantic consistency score, and feature distribution distance through a dual-domain feature multi-dimensional encoding and decoding module. Then, it scores multiple indicators across dimensions such as illumination level, feature adaptation, feature compensation, and temporal calibration. These indicators are further fed back to the illumination level-guided dual-path interactive feature enhancement network and the temporal optimization and enhancement layer, achieving reverse optimization of the network modules. This allows for precise control of feature differences between the source domain (labeled UAV normal illumination data) and the target domain (unlabeled UAV low illumination data). It leverages the generalization ability of the large model to ensure comprehensive scoring and efficiently improves feature adaptation and performance stability in cross-domain scenarios through closed-loop feedback of multi-dimensional parameters.

[0022] This invention incorporates a dual-domain feature multi-dimensional encoding and decoding module within a domain-adaptive scoring network. This module focuses on cross-domain alignment of source and target domain features. First, it extracts the latent space representations of features from both domains using source and target domain latent space representation layers, respectively. Then, a dual-domain feature joint mapping layer achieves the correlation and fusion of the features. Subsequently, a dual-domain contrastive learning encoding layer completes the contrastive encoding of the features. Finally, multiple dedicated branches of a multi-branch feature quantization decoding layer are used, namely, an illumination attribute quantization branch, a dynamic sparse evaluation branch, a semantic consistency parsing branch, and a cross-domain feature quantization branch. The module innovatively combines the joint mapping of dual-domain features with multi-dimensional quantization decoding. This achieves deep correlation between source and target domain features and accurately outputs multi-dimensional cross-domain feature evaluation indicators. This provides a comprehensive and detailed quantitative basis for subsequent domain adaptive optimization, effectively improving the accuracy and controllability of feature alignment in cross-domain scenarios. It also solves the problem of tracking interruption caused by image degradation in low light, improves tracking accuracy, and provides extreme scenario protection for the tracking process. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the overall structure of the present invention; Figure 2 This is a schematic diagram of the application framework for an example. Figure 3 This is a schematic diagram of the structure of RMDConv in this invention; Figure 4 This is a schematic diagram illustrating the working process of the dual-domain illumination layer adaptation module in this invention. Figure 5 This is a schematic diagram illustrating the working process of the hierarchical feature adaptive compensation network in this invention; Figure 6 This is a schematic diagram illustrating the working process of the wind disturbance illumination compensation timing calibration network in this invention; Figure 7 This is a schematic diagram illustrating the working process of the dual-domain feature multi-dimensional encoding and decoding module in this invention; The markings in the figure are as follows: 1. UAV edge vision camera; 2. Target tracking module; 201. Backbone network; 202. Illumination-level guided dual-path interactive feature enhancement network; 203. Temporal optimization and enhancement layer; 204. Domain adaptive scoring network; 205. Dynamic sparsity and semantic constraint alignment network; 206. Tracking result output network. Detailed Implementation

[0024] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0025] Example To address the shortcomings of existing technologies, this solution designs a closed-loop optimization scheme that integrates illumination hierarchy adaptation, domain adaptive scoring, and wind disturbance compensation calibration. The aim is to build a robust sensing scheme that adapts to complex all-weather working conditions through precise feature alignment and hierarchy compensation, and to achieve a deep integration of sensing accuracy and edge deployment efficiency.

[0026] like Figure 1 As shown, a low-light drone target tracking system includes a target tracking module 2 deployed within an industrial-grade drone edge vision camera 1. The drone edge vision camera 1 is used to acquire low-light complex dynamic images, and the target tracking module 2 is used to process and analyze the low-light complex dynamic images to output target tracking results. The target tracking module 2 is trained through domain adaptive training. The target tracking module 2 is equipped with a pre-trained backbone network 201 and a tracking result output network 206. The backbone network 201 and the tracking result output network 206 are sequentially connected by an illumination-level guided dual-path interactive feature enhancement network 202 and a temporal optimization and enhancement layer 203. The illumination-level guided dual-path interactive feature enhancement network 202 and the temporal optimization and enhancement layer 203 are respectively connected to a domain adaptive scoring network 204. The temporal optimization and enhancement layer 203 and the tracking result output network 206 are connected by a dynamic sparsity and semantic constraint alignment network 205.

[0027] In the domain-adaptive training process, the labeled UAV normal illumination dataset is used as the source domain and the unlabeled UAV low illumination dataset is used as the target domain. First, the network parameters of the backbone network 201 and the tracking result output network 206, which have been trained on the labeled UAV normal illumination dataset, are frozen. The source domain and the target domain are used as datasets to train the illumination level-guided dual-path interactive feature enhancement network 202 and the temporal optimization and enhancement layer 203. The illumination level-guided dual-path interactive feature enhancement network 202 is used to extract the inherent reflection features of the target and model the global illumination interference. Combined with the physical constraint dynamic sparse operator, non-target interference signals are eliminated. The temporal optimization and enhancement layer 203 is used to adaptively compensate for illumination layer features and complete inter-frame brightness correction, motion jitter calibration and target blur restoration, outputting source domain features and target domain features; The domain adaptive scoring network 204 is used to decode the source domain features and target domain features to extract key parameters for multi-index scoring, and feeds the multi-index scores back to the illumination level-guided dual-path interactive feature enhancement network 202 and the temporal optimization and enhancement layer 203 to achieve reverse optimization. The Dynamic Sparsity and Semantic Constraint Alignment Network 205 is used to minimize the maximum mean difference in feature distributions between the source and target domains and the semantic consistency loss, and introduces a target class prototype distance constraint to avoid semantic shift.

[0028] After the domain-adaptive training process is completed, this solution uses a domain-adaptive sparse distillation lightweight mechanism to compress the model, so as to deploy the target tracking module 2 to the industrial-grade UAV edge vision camera 1 in real time, thereby enabling high-precision, robust, and real-time target tracking at the image acquisition end.

[0029] Based on the above system, a low-light UAV target tracking method is implemented, which runs independently in the UAV edge vision camera 1, including the following steps: For the low-light complex dynamic image acquired by the UAV edge vision camera 1, the image basic representation features are first processed by the backbone network 201. Then, the illumination hierarchy-guided dual-path interactive feature enhancement network 202 extracts the inherent reflection features of the target and models global illumination interference, as well as enhances the target features and removes illumination interference; Then, timing optimization and enhancement layer 203 were used to solve the problems of overexposure, underexposure and occlusion under low light, as well as to complete inter-frame brightness correction, motion jitter calibration and target blur restoration; Finally, the target tracking result is obtained by outputting the tracking result through network 206.

[0030] This embodiment applies the above-described solution, such as Figure 2 As shown, the core framework is built around the entire process of feature enhancement, domain adaptation, compensation calibration, closed-loop scoring optimization, and lightweight deployment, mainly including: The illumination-level guided dual-path interactive feature enhancement network adopts a heterogeneous architecture of a reflection component extraction branch and an illumination constraint state space branch. It extracts the inherent reflection features of the target and models global illumination interference through residual multi-scale dynamic convolution. Combined with physical constraint dynamic sparse operators, it removes non-target interference signals. In this network, an illumination-level dual-domain adaptation module is set up. The dual domains refer to the source domain (labeled normal illumination UAV target tracking dataset) and the target domain (unlabeled low illumination UAV target tracking dataset). The illumination-level dual-domain adaptation module has a dual-branch architecture of illumination base layer and illumination variation layer. It extracts cross-illumination general features and captures hierarchical illumination interference. It enhances target features and removes illumination interference through weight allocation and feature decoupling fusion. The temporal optimization and enhancement layer includes a hierarchical feature adaptive compensation network and a wind disturbance illumination compensation temporal calibration network. The hierarchical feature adaptive compensation network is based on an illumination hierarchical feature prototype library. It solves the problems of overexposure, underexposure, and occlusion under low light by strengthening the gain in weak light, suppressing the noise in strong light, and using an occlusion perception attention mechanism. The wind disturbance illumination compensation temporal calibration network integrates UAV gyroscope data and dynamic optical flow propagation to generate illumination fluctuation compensation factors, completes inter-frame brightness correction, motion jitter calibration, and target blur restoration, and improves the continuity of temporal features. The Dynamic Sparse and Semantic Constraint Alignment Network achieves accurate alignment of source and target domain features through dynamic sparsity guidance and joint optimization of multiple loss functions. The domain adaptive scoring network, with a basic large model as its core, uses a dual-domain feature multi-dimensional encoding and decoding module to extract key parameters such as illumination level factors and semantic consistency scores. It scores multi-dimensional indicators and back-optimizes the preceding network to form a closed-loop adaptive adjustment. In this network, a dual-domain feature multi-dimensional encoding and decoding module is set up. Through dual-domain feature joint mapping and multi-branch quantization decoding, it outputs multi-dimensional cross-domain feature evaluation indicators, providing a quantitative basis for domain adaptive optimization. Finally, by combining the domain-adaptive sparse distillation lightweight mechanism with parameter sharing, plug-in encapsulation, and channel pruning techniques, the real-time and lightweight requirements of UAV edge deployment are met.

[0031] Specifically: I. Illumination-level guided dual-path interactive feature enhancement network like Figure 2 As shown, this scheme innovatively designs a lighting-level guided dual-path interactive feature enhancement network. The dual paths consist of a reflection component extraction branch and a lighting-constrained state space branch, forming a heterogeneous architecture. The reflection component extraction branch extracts the inherent reflection features (shape and texture) of the target through innovatively designed novel residual multi-scale dynamic convolution (RMDConv) and reflection feature purification operations. The lighting-constrained state space branch utilizes RMDConv to dynamically adjust the state equation parameters based on the lighting level, achieving linear complexity global lighting interference modeling (such as light source flicker and halo diffusion). A lighting-level dual-domain adaptation module enables differentiated fusion of features from both paths. Combined with a physically constrained dynamic sparse operator, it accurately removes non-target interference signals under low illumination, solving the feature confusion problem in low-light environments and improving the accuracy of core target feature extraction. This provides a high-purity feature foundation for subsequent processing, expressed as: , in, Extract the branch mapping function for the reflection component. Let be the illumination-constrained state-space branch mapping function. , These are the learnable parameters for dual-channel RMDConv. To fuse the weight matrix, For physically constrained dynamic sparse operators For dual-path feature differentiation fusion operation, This is an element-wise multiplication operation. For input image features, Light level factor, It is a dynamic sparsity index.

[0032] Residual Multi-Scale Dynamic Convolution (RMDConv) employs a multi-path parallel and dynamic fusion structure: input features are simultaneously fed into three branches, which extract features at different scales through 1×1 convolution and asymmetric convolution, respectively. The outputs of each branch are then dynamically convolved to complete the adaptive feature representation of the input. Subsequently, the multi-branch features are concatenated and integrated, followed by residual summation with the original input features, and finally, the final feature transformation is completed through depthwise separable convolution. This structure covers both local and global features through multi-scale branches, achieves dynamic feature adaptation through dynamic convolution, and combines residual connections and depthwise separable convolution to effectively reduce the number of parameters and computational costs while ensuring information integrity and representation flexibility, thus balancing model performance and lightweight requirements. Its specific operation is as follows: Figure 3 As shown, for any input tensor, after being mapped to an output tensor by residual multi-scale dynamic convolution, it can be formalized as follows: , in A collection of multi-scale convolutions. To support dynamic convolution operations with multiple scale branches, For feature splicing and integration operations, For depthwise separable convolution operations, For tensor addition operations, For the input feature tensor, This is a label for the size of multi-scale convolution kernels.

[0033] II. Illumination Hierarchy Dual-Domain Adaptation Module like Figure 4 As shown, this solution innovatively designs a dual-domain adaptation module for illumination levels, constructing a dual-branch adaptation architecture of an illumination base layer and an illumination variation layer. The illumination base layer focuses on extracting general target features across illumination conditions, ensuring the consistency of core features under different illumination scenarios. The illumination variation layer dynamically captures hierarchical illumination interference corresponding to different flight altitudes of the UAV, such as low-altitude near-ground shadow occlusion and atmospheric scattering effects in high-altitude environments. A precise weighted fusion of the dual-branch features is performed through an illumination gradient weight allocator. Finally, through a feature decoupling and fusion stage, the strong coupling between the target core features and illumination interference is effectively separated. This module can adaptively match the dynamic changes in illumination levels under complex UAV flight scenarios, providing cleaner and more robust target feature support for subsequent feature processing and tracking result output networks. This process can be represented by the following formula: in This is the feature extraction function for the illumination base layer. This is the feature extraction function for the illumination variation layer. For feature decoupling and fusion operations, , Output weights for the illumination gradient weight allocator. For weighted multiplication operations, For tensor addition operations, For input features, This refers to the flight altitude of the drone.

[0034] III. Hierarchical Feature Adaptive Compensation Network like Figure 5 As shown, this solution innovatively designs a hierarchical feature adaptive compensation network. Its core architecture revolves around a prototype library of illumination hierarchical features. The output features of this prototype library are simultaneously split into two parallel processing paths. One path first enhances the feature representation of low-light regions through low-level weak-light gain enhancement, while the other path reduces redundant interference in strong-light regions through high-level strong-light noise suppression. Subsequently, the outputs of both paths are sequentially connected to an occlusion perception attention module to focus on effective features and filter out invalid information in occluded regions. Then, an adaptive compensation module performs targeted repair and enhancement of the features. Finally, a hierarchical feature integration module merges the processing results of the two paths into a complete feature. This network innovation achieves differentiated processing of illumination hierarchical features. Combining occlusion perception and adaptive compensation mechanisms, it can accurately optimize the feature quality of different illumination regions and efficiently integrate hierarchical information, significantly improving the feature representation effect under complex illumination and occlusion scenes. This solves problems such as local overexposure, underexposure, and occlusion under low light conditions, providing highly recognizable features for tracking and inference. Specifically, this can be expressed by the following formula: in Output features for the illumination level feature prototype library. For low-level low-light gain enhancement operation, For high-level strong light noise suppression operation, To obscure the perceptual attention mechanism, , For adaptive compensation module, This is a hierarchical feature integration operation.

[0035] IV. Wind Disturbance and Light Illumination Compensation Timing Calibration Network like Figure 6As shown, this scheme innovatively designs a wind disturbance and illumination compensation temporal calibration network, integrating wind field perception, optical flow propagation, and temporal calibration into a unified mechanism. On the one hand, it extracts wind disturbance features based on real-time data from the UAV gyroscope; on the other hand, it performs dynamic optical flow propagation on the input image features. The two work together to drive the generation of illumination fluctuation compensation factors. With the illumination fluctuation compensation factor as the core, it first corrects the brightness difference between frames, and then sequentially completes motion jitter calibration and target blur restoration. This not only weakens the interference of wind disturbance and illumination fluctuations on inter-frame features, but also alleviates the motion blur problem of targets during UAV dynamic flight, effectively reducing inter-frame feature propagation deviation and significantly improving the continuity of temporal features in low-light dynamic scenes. This network can accept cross-domain aligned features to achieve joint optimization in the spatiotemporal dimensions, and its process can be represented by the following formula: , , in This is the wind disturbance feature extraction function. For dynamic optical flow propagation operation, This is the generation function for the illumination fluctuation compensation factor. This is an operation to correct for inter-frame brightness differences. For motion jitter calibration operation, For the target fuzzy restoration operation, As a compensation factor for illumination fluctuations, For the features of the current frame, G represents the features of the previous frame, and G represents the real-time data from the UAV gyroscope.

[0036] V. Dynamic Sparse and Semantic Constraint Alignment Network This innovative solution employs a dynamic sparse and semantically constrained alignment network to construct a cross-domain alignment architecture guided by dynamic sparseness and reinforced by semantic constraints. Based on a domain-adaptive core method, a dynamic sparse feature guidance mechanism is designed to provide an efficient feature foundation for domain alignment by accurately selecting key features and suppressing redundant information. Simultaneously, during optimization, the maximum mean difference in feature distribution between the source domain (normal illumination data) and the target domain (low illumination data) is jointly minimized, along with semantic consistency loss. Furthermore, a target category prototype distance constraint term is introduced to effectively avoid semantic shift issues in cross-domain feature transformation. This network inherits the results of previous feature extraction and domain adaptation, specifically addressing the tracking drift problem caused by feature distribution shifts in low-illumination scenarios and achieving precise control over tracking confidence fluctuations. Ultimately, it achieves accurate alignment of features between the source and target domains, a process that can be represented by the following formula: , in The loss function is the maximum mean difference of the characteristic distribution. Let the semantic consistency loss function be... The loss function is defined by the prototype distance constraint of the target category. These are key features after source domain filtering. Key features selected after filtering the target domain. For the target category prototype, , For loss weighting coefficients, For tensor addition operations, This is a weighted multiplication operation.

[0037] VI. Domain Adaptive Scoring Network Based on the Basic Large Model This innovative solution employs a domain-adaptive scoring network based on a fundamental large model. This network is the core module for achieving cross-domain feature alignment and performance optimization. Using the fundamental large model as its core carrier, the network first extracts key quantitative parameters such as illumination level factors, dynamic sparsity index, semantic consistency score, and feature distribution distance through a dual-domain feature multi-dimensional encoding and decoding module. Then, it scores multiple indicators across dimensions such as illumination level, feature adaptation, feature compensation, and temporal calibration. These indicators are further fed back to the illumination level-guided dual-path interactive feature enhancement network and the temporal optimization and enhancement layer, achieving reverse optimization of the network modules. This allows for precise control of feature differences between the source domain (labeled UAV normal illumination data) and the target domain (unlabeled UAV low illumination data). It leverages the generalization ability of the large model to ensure comprehensive scoring while efficiently improving feature adaptation and performance stability in cross-domain scenarios through closed-loop feedback of multi-dimensional parameters. By integrating illumination level factors, dynamic sparse feature stability index, and semantic consistency score, an unsupervised scoring function is constructed. This function can evaluate the model's adaptability to scenarios such as illumination fluctuations in real time without target domain annotation. It automatically selects the optimal combination of illumination level adaptation coefficient and dynamic sparsity rate to adapt to complex low-light scenarios such as dawn-dusk transitions. The score is strongly correlated with the actual tracking accuracy, forming an adaptive closed-loop optimization, which can be expressed by the following formula: , in For multi-indicator fusion scoring function, For gradient calculation operations on network parameters, Rate the lighting levels. For feature fitting scoring, For semantic consistency score, For characteristic distribution distance, For the comprehensive scoring results, These are the initial network parameters. These are the parameters after reverse optimization. For learning rate, For parameter update subtraction operation, This is a weighted multiplication operation.

[0038] VII. Dual-domain feature multi-dimensional encoding and decoding module like Figure 7 As shown, this scheme designs a dual-domain feature multi-dimensional encoding and decoding module in the domain adaptive scoring network. This module's core objective is cross-domain alignment of source and target domain features. First, it extracts the latent space representations of the features from both domains through the source and target domain latent space representation layers, respectively. Then, a dual-domain feature joint mapping layer achieves the correlation and fusion of the features. Subsequently, a dual-domain contrastive learning encoding layer completes the contrastive encoding of the features. Finally, multiple dedicated branches of the multi-branch feature quantization decoding layer, namely the illumination attribute quantization branch, dynamic sparse evaluation branch, semantic consistency parsing branch, and cross-domain feature fluctuation branch, are used to further refine the feature representation. This module outputs multi-dimensional quantitative parameters such as illumination level factor, dynamic sparsity index, semantic consistency score, and feature distribution distance. It innovatively combines joint mapping of dual-domain features with multi-dimensional quantization decoding, achieving deep correlation between source and target domain features and accurately outputting multi-dimensional cross-domain feature evaluation indicators. This provides comprehensive and detailed quantitative basis for subsequent domain adaptive optimization, effectively improving the accuracy and controllability of feature alignment in cross-domain scenarios, solving the tracking interruption problem caused by image degradation in low-light conditions, improving tracking accuracy, and providing extreme scenario protection for the tracking process. Specifically, this can be expressed by the following formula: , in Let be the latent space representation function of the source domain. Let be the latent space representation function of the target domain. This is a joint mapping operation for features from two domains. For two-domain contrastive learning encoding operations, For multi-branch feature quantization decoding operations, Light level factor, It is a dynamic sparsity index. For semantic consistency score, For characteristic distribution distance, Original features of the source domain These are the original features of the target domain.

[0039] VIII. Domain-Adaptive Sparse Distillation Lightweight Mechanism This innovative design employs a domain-adaptive sparse distillation lightweight mechanism, utilizing domain-invariant parameter sharing and adaptive parameter sparse distillation. On one hand, the domain-invariant parameter sharing strategy enables parameter reuse of core modules across different domain scenarios. On the other hand, core components such as the illumination-level guided dual-path interactive feature enhancement network, the hierarchical feature adaptive compensation network, and the wind disturbance illumination compensation temporal calibration network are encapsulated in a plug-in manner (supporting flexible plug-and-play adaptation to different scenarios). Combined with channel pruning and parameter distillation techniques, redundant parameters are precisely optimized. This significantly reduces the number of model parameters and inference time while maintaining detail retention capabilities across various optical zoom scenarios. This mechanism not only meets the real-time processing requirements of UAV platforms but also provides a feasible path for the lightweight deployment and engineering implementation of algorithms. Its process can be represented by the following formula: , in For domain-invariant shared parameters, These are parameters specific to the task. For adaptive parameter sparse distillation operation, For channel pruning operations, For distillation parameters, These are the parameters of the lightweight model. This is a parameter set merging operation.

[0040] By deploying the target tracking module designed in this solution within an industrial-grade UAV edge vision camera based on an embedded NPU platform, edge computing can be achieved to quickly and accurately obtain target tracking results. Specifically, this edge computing refers to acquiring low-light, complex, and dynamic image data on the UAV itself using an onboard edge vision camera, and independently implementing edge visual intelligence within that camera. This means that the low-light UAV target tracking method proposed in this solution achieves high-precision, robust, and real-time tracking results at the image data acquisition end. Applying this solution in practice, the obtained target tracking results can be used for rapid human-UAV interaction, communication data optimization, real-time response operations, intelligent analysis applications, privacy protection, and data security. This ensures that the number and amount of communication with the cloud platform are minimized, thereby reducing waiting time and computational costs. Furthermore, by deploying the UAV low-light target tracking task to the image data acquisition end, this solution can effectively reduce backbone network congestion, alleviate bandwidth occupation, achieve lower latency, improve processing efficiency, speed up response to requests, and further improve the target tracking quality of UAVs in complex dynamic low-light environments. Moreover, the speed of this solution after edge deployment can reach real-time processing speed, and its tracking accuracy and success rate meet the requirements of industrial applications.

Claims

1. A low-light unmanned aerial vehicle (UAV) target tracking system, characterized in that, It includes a target tracking module (2) deployed in an industrial-grade UAV edge vision camera (1). The UAV edge vision camera (1) is used to acquire low-light complex dynamic images. The target tracking module (2) is used to process and analyze the low-light complex dynamic images and output the target tracking results. The target tracking module (2) is obtained through domain adaptive training. The target tracking module (2) includes a pre-trained backbone network (201) and a tracking result output network (206). The backbone network (201) and the tracking result output network (206) are connected in sequence by an illumination level-guided dual-path interactive feature enhancement network (202) and a temporal optimization and enhancement layer (203). The illumination level-guided dual-path interactive feature enhancement network (202) and the temporal optimization and enhancement layer (203) are respectively connected to a domain adaptive scoring network (204). The temporal optimization and enhancement layer (203) and the tracking result output network (206) are connected by a dynamic sparsity and semantic constraint alignment network (205). In the adaptive training process of the domain, the labeled UAV normal illumination dataset is used as the source domain and the unlabeled UAV low illumination dataset is used as the target domain. First, the parameters of the backbone network (201) and the tracking result output network (206) that have been trained on the labeled UAV normal illumination dataset are frozen. The source domain and the target domain are used as datasets to train the illumination level-guided dual-path interactive feature enhancement network (202) and the temporal optimization and enhancement layer (203). The illumination level-guided dual-path interactive feature enhancement network (202) is used to extract the inherent reflection features of the target and model the global illumination interference. Combined with the physical constraint dynamic sparse operator, the non-target interference signal is eliminated. The temporal optimization and enhancement layer (203) is used to adaptively compensate for illumination layer features and complete inter-frame brightness correction, motion jitter calibration and target blur restoration, outputting source domain features and target domain features; The domain adaptive scoring network (204) is used to decode the source domain features and target domain features to extract key parameters for multi-index scoring, and feeds the multi-index scores back to the illumination level-guided dual-path interactive feature enhancement network (202) and the temporal optimization and enhancement layer (203) to achieve reverse optimization; The Dynamic Sparse and Semantic Constraint Alignment Network (205) is used to minimize the maximum mean difference in feature distributions between the source and target domains and the semantic consistency loss, and introduces the target class prototype distance constraint to avoid semantic shift; After the domain adaptive training process is completed, the model is compressed through the domain adaptation sparse distillation lightweight mechanism so that the target tracking module (2) can be deployed to the industrial-grade UAV edge vision camera (1) in real time.

2. The low-light UAV target tracking system according to claim 1, characterized in that, The illumination-level guided dual-path interactive feature enhancement network (202) includes an illumination-level dual-domain adaptation module and a heterogeneous reflection component extraction branch and illumination constraint state space branch. The reflection component extraction branch extracts the inherent reflection features of the target through RMDConv and reflection feature purification operations. The illumination-constrained state space branch utilizes RMDConv to dynamically adjust the state equation parameters based on the illumination hierarchy, thereby achieving global illumination interference modeling with linear complexity. The illumination-level dual-domain adaptation module is used to perform dual-path feature differentiation fusion, in conjunction with a physical constraint dynamic sparse operator, to eliminate non-target interference signals under low illumination.

3. A low-light UAV target tracking system according to claim 2, characterized in that, The RMDConv adopts a multi-path parallel and dynamic fusion structure design. The input features are simultaneously connected to three branches, and feature extraction at different scales is achieved through 1×1 convolution and asymmetric convolution, respectively. The output of each branch is then dynamically convolved to complete the adaptive feature representation of the input. Subsequently, the multi-branch features are concatenated and integrated, then residual summation is performed with the original input features, and finally the final feature transformation is completed through depthwise separable convolution.

4. A low-light UAV target tracking system according to claim 2, characterized in that, The illumination hierarchy dual-domain adaptation module adopts a dual-branch adaptation architecture of illumination base layer and illumination variation layer. The illumination base layer is used to extract general target features across illumination conditions, and the illumination variation layer is used to dynamically capture hierarchical illumination interference corresponding to different flight altitudes of the UAV, including low-altitude near-ground shadow modeling and high-altitude atmospheric scattering modeling. The illumination-level dual-domain adaptation module also includes an illumination gradient weight allocator to perform weighted fusion of the dual-branch features, and then uses a feature decoupling fusion step to separate the strong coupling between the target core features and illumination interference.

5. A low-light UAV target tracking system according to claim 1, characterized in that, The temporal optimization and enhancement layer (203) is equipped with a hierarchical feature adaptive compensation network and a wind disturbance illumination compensation temporal calibration network. The hierarchical feature adaptive compensation network is equipped with an illumination hierarchical feature prototype library. The output features of the illumination hierarchical feature prototype library are simultaneously split to two parallel processing paths. One path first enhances the feature performance of the weak light region through low-level weak light gain enhancement, while the other path reduces the redundant interference of the strong light region through high-level strong light noise suppression. Subsequently, the outputs of the two parallel processing paths are sequentially connected to the occlusion perception attention module and the adaptive compensation module. Finally, the hierarchical feature integration module merges the processing results of the two parallel processing paths into a complete feature. The working process of the wind disturbance and illumination compensation timing calibration network includes: Wind disturbance features are extracted based on real-time data from the UAV gyroscope, and dynamic optical flow propagation is performed on the input image features. The two work together to drive the generation of the illumination fluctuation compensation factor. The inter-frame brightness difference is first corrected using the illumination fluctuation compensation factor, and then motion jitter calibration and target blur restoration are completed sequentially.

6. A low-light UAV target tracking system according to claim 1, characterized in that, The domain adaptive scoring network (204) takes the basic large model as its core carrier. First, it extracts key quantitative parameters including illumination level factor, dynamic sparsity index, semantic consistency score and feature distribution distance through the dual-domain feature multi-dimensional encoding and decoding module. Then, it performs multi-index scoring from the dimensions of illumination level, feature adaptation, feature compensation and time-series calibration. The multi-index scoring results are fed back to the illumination level-guided dual-path interactive feature enhancement network (202) and the time-series optimization and enhancement layer (203) to achieve reverse optimization.

7. A low-light UAV target tracking system according to claim 6, characterized in that, The dual-domain feature multi-dimensional encoding and decoding module is configured with a source domain latent space representation layer, a target domain latent space representation layer, a dual-domain feature joint mapping layer, a dual-domain contrastive learning encoding layer, and a multi-branch feature quantization decoding layer. The source domain features and target domain features are respectively input to the source domain latent space representation layer and the target domain latent space representation layer to extract the source domain latent space expression features and the target domain latent space expression features respectively. Then, the dual-domain feature joint mapping layer realizes the correlation and fusion of dual-domain features. Subsequently, the dual-domain contrastive learning encoding layer completes the contrastive encoding of features. Finally, the multi-branch feature quantization decoding layer outputs the illumination level factor, dynamic sparsity index, semantic consistency score, and feature distribution distance through multiple dedicated branches.

8. A low-light UAV target tracking system according to claim 1, characterized in that, The dynamic sparse and semantic constraint alignment network (205) adopts a cross-domain alignment architecture with dynamic sparse guidance and semantic constraint reinforcement. The dynamic sparse guidance is used to filter key features and suppress redundant information. The semantic constraint enhancement specifically involves jointly minimizing the maximum mean difference between the feature distributions of the source and target domains and the semantic consistency loss, and introducing a target category prototype distance constraint term to avoid semantic offset issues in cross-domain feature transformation.

9. A low-light UAV target tracking system according to claim 1, characterized in that, The domain-adaptive sparse distillation lightweight mechanism specifically achieves model compression through domain-invariant parameter sharing, plug-in encapsulation, channel pruning, and parametric distillation techniques.

10. A low-light unmanned aerial vehicle (UAV) target tracking method, implemented based on the low-light UAV target tracking system as described in claim 1, characterized in that, The method runs independently within the UAV edge vision camera (1) and includes the following steps: For low-light complex dynamic images acquired by the UAV edge vision camera (1), the basic image representation features are first obtained by processing through the backbone network (201); Then, the target's inherent reflection features are extracted and global illumination interference is modeled by the illumination hierarchy-guided dual-path interactive feature enhancement network (202), as well as the target features are enhanced and illumination interference is removed; Then, the timing optimization and enhancement layer (203) is used to solve the problems of overexposure, underexposure and occlusion under low light, as well as to complete the inter-frame brightness correction, motion jitter calibration and target blur restoration; Finally, the target tracking result is obtained by outputting the tracking result through the tracking result output network (206).

Images