A low-altitude field multi-target detection method and system based on digital twinning

Abstract

This invention discloses a multi-target detection method and system based on digital twins for the low-altitude domain. The method proposes the DT-YOLO end-to-end detection framework, addressing the challenges of small target detection from a UAV perspective, poor adaptability to complex environments, and high training data acquisition costs. Through the collaborative design of four core modules—a lightweight backbone network, a high-resolution feature pyramid, multi-scale adaptive enhancement, and hybrid balanced loss—it achieves high-precision, lightweight, and weather-adaptive multi-target detection. The system employs a graphical user interface for intuitive human-computer interaction. This invention combines deep learning methods with digital twin technology to perform real-time identification, localization, and detection of various targets, including vehicles and pedestrians, in UAV aerial images, and possesses robust detection capabilities under various weather conditions, flight altitudes, and viewing angles.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, artificial intelligence, digital twins, and intelligent transportation, specifically to a method and system for multi-target detection in the low-altitude domain based on digital twins. Background Technology

[0002] With the acceleration of urbanization and the rapid development of intelligent transportation systems, drone-based traffic monitoring is becoming an important tool for modern urban management. In recent years, the rapid rise of the low-altitude economy has opened up new development paths for intelligent urban management. Compared to traditional fixed-view monitoring systems, drones offer flexible aerial perspectives, can cover a wider area, can be quickly deployed to traffic hotspots, and adapt to complex and ever-changing urban environments. This flexibility and mobility give drones unique advantages in scenarios such as emergency response, large-scale event monitoring, and key area monitoring.

[0003] However, traffic target detection from the perspective of drones faces many technical challenges:

[0004] First, small target detection is difficult. In aerial view scenarios, targets such as vehicles and pedestrians occupy very few pixels in the image, usually less than 32×32 pixels, making it difficult for traditional algorithms to accurately detect these tiny targets.

[0005] Second, interference from complex environments. Urban environments present interference factors such as dense shading, dynamic lighting, and extreme weather conditions, which seriously affect detection accuracy.

[0006] Third, data acquisition costs are high. Acquiring real training data under large-scale, multi-scenario, and multi-weather conditions is costly and time-consuming, and in some extreme scenarios, it may even pose security risks and legal restrictions.

[0007] In recent years, deep learning technology has driven the rapid development of the object detection field. The YOLO (You Only Look Once) series of single-stage object detection algorithms are widely popular for their excellent balance between real-time performance and accuracy. However, existing algorithms still have significant limitations when dealing with small object detection tasks from the perspective of drones. The feature map resolution of the standard detection layer is too coarse for tiny targets, resulting in a high false negative rate.

[0008] Digital twin technology offers an innovative solution to overcome the aforementioned bottlenecks. By constructing a high-fidelity virtual copy of the physical world, digital twins achieve a two-way mapping between physical and digital spaces. In traffic monitoring research, digital twin platforms can provide a fully controllable experimental environment, allowing researchers to precisely adjust all environmental parameters. CARLA, as an open-source autonomous driving simulation platform, has become an ideal choice for constructing digital twin traffic scenarios due to its powerful sensor simulation system.

[0009] Based on the above, this invention proposes a multi-target detection method and system for low-altitude domains based on digital twins. Summary of the Invention

[0010] The purpose of this invention is to propose a multi-target detection method and system based on digital twins for low-altitude environments, addressing the challenges of small target detection, poor adaptability to complex environments, and high training data acquisition costs from the perspective of unmanned aerial vehicles (UAVs). This invention employs deep learning methods combined with digital twin technology to perform real-time identification, localization, and detection of various targets, including vehicles and pedestrians, in UAV aerial images, and possesses robust detection capabilities under diverse weather conditions, flight altitudes, and viewing angles.

[0011] To achieve the above objectives, the present invention adopts the following technical solution: A digital twin-based multi-target detection method for low-altitude domains includes the following steps: Using drones to acquire drone aerial images; A digital twin-based YOLO (DT-YOLO) end-to-end detection framework is proposed. The obtained UAV aerial images are input into the pre-trained digital twin-based YOLO (DT-YOLO) end-to-end detection framework, and the multi-target detection results are output. The digital twin-based YOLO (DT-YOLO) end-to-end detection framework includes: The Lightweight Star Backbone Network (LSB) extracts features from the input image through the synergistic effect of grouped convolution, channel shuffling, and star aggregation mechanism, and outputs multi-scale feature maps. The efficient weighted feature pyramid network (EWFPN) is connected to the lightweight star backbone network (LSB) to receive multi-scale feature maps and constructs an enhanced multi-scale feature pyramid by introducing learnable weighted fusion, high-resolution enhancement and bidirectional information propagation mechanisms. The Multi-Scale Adaptive Enhancement Module (MSAE), connected to the Efficient Weighted Feature Pyramid Network (EWFPN), is used to adaptively enhance the enhanced multi-scale feature pyramid through a three-dimensional attention mechanism to generate detection features; The detection head, connected to the multi-scale adaptive enhancement module (MSAE), is used to perform target classification and bounding box regression based on the detection features, and output multi-target detection results; The YOLO (DT-YOLO) end-to-end detection framework based on digital twins is trained using the Hybrid Balanced Loss Technique (HBLT), which optimizes network parameters by combining soft label classification loss, size-aware localization loss, and distribution prediction loss.

[0012] Preferably, the efficient weighted feature pyramid network (EWFPN) introduces a P2 layer (higher resolution, smaller stride) to participate in feature fusion and detection; the P2 layer has a higher resolution than the minimum detection layer in the standard feature pyramid, which is used to enhance the spatial feature representation of small targets; The learnable weighted fusion mechanism assigns learnable weight parameters to each feature branch, enabling the fusion operation to adaptively adjust the importance of features according to task requirements. The bidirectional information propagation mechanism is used to ensure the effective coupling of semantic information and detailed features in the pyramid hierarchy; The multi-scale feature fusion process can be represented functionally as follows:

[0013] in, This represents the fused output feature map; Represents the learnable weight parameters; Let i be the i-th input feature map; This represents a numerically stable term.

[0014] Preferably, the three-dimensional attention mechanism specifically includes: Scale-aware attention is used to dynamically learn and adjust the importance of features at different scales through a self-attention mechanism. Spatial awareness attention is used to generate spatial attention weights using deformable convolution, so that the receptive field can be dynamically adjusted according to the shape of the target. Task-aware attention is used to learn feature weights for the target classification and bounding box regression branches respectively, thereby achieving task-specific feature enhancement.

[0015] Preferably, the total loss function of the Hybrid Balanced Loss Technique (HBLT) is expressed as:

[0016] in, This is a soft-label classification loss, used to reduce the weight of easily classified samples and focus on difficult-to-classify samples by using a dynamic modulation factor; Size-aware localization loss is used to adaptively weight targets of different sizes and difficulty levels using a non-monotonic modulation function; This is the distributed prediction loss, used to model bounding box regression as a probability distribution prediction; , , These are the weighting coefficients for each loss.

[0017] Preferably, the pre-training process of the YOLO (DT-YOLO) end-to-end detection framework based on digital twins includes the step of constructing a synthetic dataset based on digital twin technology, specifically including: Utilize a simulation platform to construct a high-fidelity digital twin of urban traffic scenarios; In the constructed digital twin urban traffic scenario, flight parameters, weather conditions, and traffic density are programmed to generate diverse monitoring scenarios; Images are collected from the constructed digital twin urban traffic scene, and corresponding target bounding box annotations are automatically generated based on the 3D target information in the scene to form a digital twin dataset for pre-training.

[0018] Preferably, the flight parameters include flight altitude, pitch angle, and azimuth angle; the flight altitude is configured to include low-altitude monitoring altitude, standard cruise altitude, and high-altitude monitoring altitude; the pitch angle is configured to include top-down and tilt angles; and the azimuth angle is configured to be sampled at preset intervals within the range of 0°-360°. The weather conditions are configured to include at least two of the following: sunny, light rain, heavy fog, and night.

[0019] A digital twin-based multi-target detection system for low-altitude environments includes: The data acquisition module is used to acquire aerial images taken by the drone; The target detection module integrates a pre-trained digital twin-based YOLO (DT-YOLO) end-to-end detection framework for processing UAV aerial images and outputting multi-target detection results. The visualization and interaction module is used to receive and display multi-target detection results and provide a human-computer interaction interface.

[0020] Preferably, the visualization interaction module is built on the PyQt framework, adopts a modular architecture that separates the data layer, algorithm layer and presentation layer, and uses a multi-threaded architecture to allocate image acquisition, model inference and interface rendering to different threads for execution.

[0021] The present invention further protects a computer device, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the instruction, program, code set or instruction set being loaded and executed by the processor to implement the above-mentioned digital twin-based multi-target detection method for low-altitude domains.

[0022] The present invention further protects a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the instruction, program, code set, or instruction set is loaded and executed by a processor to implement the above-described digital twin-based multi-target detection method for low-altitude domains.

[0023] Compared with existing technologies, this invention provides a digital twin-based multi-target detection method and system for low-altitude domains, which has the following advantages: The multi-person behavior recognition method proposed in this invention significantly improves classification accuracy by enhancing the baseline network model, while also providing interpretability for the classification results. The proposed baseline model uses ResNet as the backbone to extract visual features for each frame, then employs a DETR-style Transformer decoder to obtain token representations. These tokens are then aggregated into clip-level features via temporal and aggregation modules, and finally, a fully connected classifier outputs the group behavior category. Specifically, it offers the following advantages: (1) Improve recognition accuracy and robustness: Through adaptive spatial resampling enhancement, the loss of discriminative information caused by fixed aggregation is effectively alleviated, making the network more focused on key spatiotemporal regions related to group behavior.

[0024] (2) Structured joint reasoning of relationships and interactions: The relationship-interaction combined reasoning head explicitly constructs and combines human-human relationships and human-object interactions to form a branchable and supervised reasoning link, which is conducive to learning more comprehensive group representations and improving the ability to identify complex group behaviors.

[0025] (3) It has a causal interpretable mechanism with clear intervention semantics: the saliency partition counterfactual learning completes the counterfactual construction and consistency constraint in the label space, which avoids the high overhead of pixel-level generation and can output explanatory information on "how the model prediction changes after the key evidence is replaced", which is convenient for manual verification and model auditing.

[0026] (4) Easy to deploy in engineering: The enhancement module and counterfactual learning mainly occur in the training phase or label space, and the inference phase does not increase or only slightly increases the amount of computation; the system can be deployed on the server or edge computing device to meet real-time or near-real-time requirements. Attached Figure Description

[0027] Figure 1 The qualitative visualization comparison results of DT-YOLO in a real drone scenario on VisDrone 2019, as mentioned in Embodiment 2 of this invention; Figure 2 Visualization of the cross-domain detection performance of DT-YOLO in the real world and digital twin world mentioned in Embodiment 2 of the present invention; Figure 3 This is an analysis of the training performance metrics of DT-YOLO on the VisDrone2019 dataset mentioned in Embodiment 2 of the present invention; Figure 4This demonstrates the visualization of multi-weather condition detection using DT-YOLO in the CARLA digital twin environment, as mentioned in Embodiment 2 of this invention. Figure 5 This refers to the multi-dimensional detection visualization of DT-YOLO in the CARLA digital twin environment mentioned in Embodiment 2 of the present invention; Figure 6 The training performance visualization of DT-YOLO mentioned in Embodiment 2 of this invention on the CARLA Digital Twin Dataset includes: (a) confusion matrix; (b) data distribution statistics; (c) bounding box correlation; (d) precision-confidence curve; (e) F1-confidence curve; (f) recall-confidence curve; and (g) precision-recall curve. Figure 7 This is a comparison between the LSTM traffic flow prediction based on DT-YOLO detection results and the actual measured values ​​mentioned in Embodiment 3 of the present invention; Figure 8 This is the system initial interface mentioned in Embodiment 3 of the present invention; Figure 9 This refers to the real-time monitoring interface of the system mentioned in Embodiment 3 of the present invention. Detailed Implementation

[0028] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0029] Traffic target detection from a drone's perspective faces three fundamental challenges that traditional detection frameworks struggle to address simultaneously. First, the inherent extreme scale differences of aerial photography compress vehicles and pedestrians into tiny spatial footprints of only 16×16 to 32×32 pixels, while traditional multi-scale feature pyramids operate at P3-P5 resolution (step sizes 8-32), failing to retain sufficient spatial detail for accurate localization. Second, airborne and edge deployment platforms impose stringent computational constraints, requiring lightweight architectures to maintain real-time inference capabilities without sacrificing detection accuracy. Third, validating detection algorithms through real-world data collection faces practical obstacles, including airspace control, weather dependence, and the inability to systematically generate extreme scenarios, limiting the diversity and scale of training datasets.

[0030] To systematically address these interconnected challenges, this invention proposes the DT-YOLO (Digital Twin YOLO) end-to-end detection framework. Through the collaborative design of four core modules—a lightweight backbone network, a high-resolution feature pyramid, multi-scale adaptive enhancement, and hybrid balanced loss—it achieves high-precision, lightweight, and weather-adaptive multi-target detection. DT-YOLO does not treat lightweight deployment and small target detection as isolated goals, but rather establishes a coherent design philosophy where each component contributes to a specific aspect of the overall solution while enhancing the capabilities of other modules. The Efficient Weighted Feature Pyramid Network (EWFPN) extends the spatial resolution to the P2 layer (160×160) and introduces learnable fusion weights to dynamically emphasize high-resolution features crucial for small target localization, establishing the feature representation foundation upon which subsequent modules rely. The Multi-Scale Adaptive Enhancement (MSAE) module achieves dynamic detection through a 3D attention mechanism, adaptively adjusting scale distribution, spatial structure, and task requirements, transforming the detection head from a static feature processor into an adaptive enhancement unit. The Hybrid Balanced Loss Technique (HBLT) addresses training challenges by using soft-label classification and size-aware localization losses, mitigating class imbalance and scale sensitivity issues that disproportionately affect minority classes in traffic scenarios. These three core modules operate consistently across all deployment configurations, while the backbone network architecture provides a scalable implementation—a standard enhanced backbone for high-performance scenarios and a lightweight star backbone (LSB) for resource-constrained edge devices—adapting to heterogeneous hardware platforms while maintaining a unified detection framework.

[0031] The synergistic integration of these four modules creates emergent capabilities that surpass their individual contributions. EWFPN introduces a P2 layer and constructs a high-resolution feature pyramid through weighted fusion using Bidirectional Feature Pyramid Network (BiFPN), providing a fine-grained, multi-scale feature foundation for the subsequent MSAE dynamic detection head. MSAE adaptively extracts and enhances target information from the P2-P4 layers of features through the 3D attention mechanism of DyHead (DyHeadDynamic Head). The deformable features provided by LSB (DCNv3 in Full configuration) offer geometric flexibility for the spatial adaptation of MSAE. The dynamic detection head of MSAE adaptively utilizes the rich feature representations established by EWFPN, concentrating computational resources on informative spatial regions and appropriate scale ranges. HBLT's adaptive loss weighting enables the network to fully leverage the representational capabilities established by EWFPN and MSAE, ensuring that training resources are allocated according to detection difficulty rather than sample frequency. In deployment scenarios requiring maximum efficiency, the LSB backbone seamlessly integrates with this detection pipeline, significantly reducing computational overhead while maintaining feature extraction quality. This architectural coherence distinguishes DT-YOLO from the traditional approach of simply stacking independent modules, establishing a unified framework specifically optimized for traffic monitoring from the perspective of drones.

[0032] The core innovations of the above-mentioned DT-YOLO end-to-end detection framework are: (1) introducing a high-resolution detection layer to enhance the feature expression of small targets, and improving the feature response of small targets from a single pixel to a larger receptive field area; (2) achieving adaptive adjustment of scale, space and task dimensions through a three-dimensional attention mechanism; (3) effectively alleviating the problems of class imbalance and scale sensitivity by adopting a hybrid loss strategy; (4) using digital twin technology to construct a synthetic dataset with multiple weather conditions, multiple angles and multiple dimensions, systematically covering extreme scenes that are difficult to obtain in the real world.

[0033] To fundamentally overcome the inherent limitations of real-world data acquisition, this invention further proposes a digital twin verification framework based on the CARLA simulation platform. By programmatically controlling environmental parameters including altitude, viewpoint, weather conditions, and traffic density, a large-scale synthetic dataset is constructed, systematically covering scenes that are difficult or impossible to capture in reality. Cross-domain verification between the real VisDrone 2019 dataset and the synthetic CARLA dataset demonstrates the effective transfer capability of the proposed method and system from the virtual to the real domain, verifying the robustness of the DT-YOLO design and the feasibility of developing simulation-based aerial traffic monitoring algorithms.

[0034] The following description, in conjunction with relevant accompanying drawings and specific examples, further illustrates the proposed digital twin-based multi-target detection method and system for low-altitude domains.

[0035] Example 1: This invention proposes a multi-target detection method and system based on digital twins for low-altitude domains, specifically including: (a) Lightweight Star Backbone Network (LSB) In scenarios where UAV onboard platforms have limited resources, the computational complexity of standard backbone networks is too high to meet the requirements of real-time detection. Therefore, this invention adopts a lightweight backbone network for improvement. This network achieves a significant reduction in parameters and computational load through three collaborative mechanisms: grouped convolution, channel shuffling, and star-shaped aggregation.

[0036] Grouped convolution divides the input channels into multiple non-overlapping groups, with each group undergoing independent convolution followed by concatenation of the outputs, effectively reducing computational complexity. Channel shuffling systematically rearranges the channel order, ensuring that each group in subsequent layers receives information from all groups in the previous layer, enhancing feature interaction capabilities. The star-shaped aggregation mechanism reduces the complexity of traditional dense connections from quadratic to linear, significantly reducing computational overhead while maintaining feature fusion effectiveness.

[0037] (ii) Efficient Weighted Feature Pyramid Network (EWFPN) In UAV traffic monitoring missions, the detector's accuracy for small targets is crucial to the overall system performance. The standard YOLO architecture uses only three feature layers (P3-P5, stride 8, 16, 32), with a minimum feature map resolution of 80×80. For a small target occupying only 16×16 pixels, this corresponds to only one pixel response on the P3 layer, which cannot provide sufficient spatial degrees of freedom for bounding box regression.

[0038] This invention integrates three mechanisms within a bidirectional pyramid architecture: learnable weighted fusion, high-resolution enhancement, and bidirectional information propagation. By introducing a P2 layer (higher resolution, smaller stride) to participate in feature fusion and detection, the P2 layer provides four times the spatial detail compared to the P3 layer. A 16×16 pixel target occupies a larger receptive field area on the P2 layer instead of a single pixel, fundamentally alleviating the difficulty of locating small targets.

[0039] A learnable weighted fusion mechanism assigns learnable weight parameters to each feature branch, enabling the fusion operation to adaptively adjust feature importance according to task requirements. A bidirectional information propagation mechanism ensures effective coupling of semantic information and detailed features within the pyramid hierarchical structure. (1) in For learnable weight parameters; Fi Refers to the first i Each input feature map represents a feature tensor from different scales (or branches); F out The fused output feature map is generated using learnable weight parameters. The final feature representation obtained by weighted summation (and normalization) of all input features is used for subsequent object detection tasks; It is a numerically stable term.

[0040] (III) Multi-scale Adaptive Augmentation Module (MSAE) High-resolution feature pyramid networks construct multi-scale features that provide fine-grained spatial representations for small targets, but fixed detection heads struggle to adapt to the drastic changes in target scale, deformation, and occlusion in UAV scenarios. MSAE achieves adaptive enhancement through the DyHead dynamic detection head mechanism, which includes a three-dimensional attention mechanism.

[0041] Scale-aware attention learns the relative importance of features at different scales through a self-attention mechanism and dynamically adjusts the weights of each layer based on the statistical properties of the input features, enabling the network to adaptively emphasize the most relevant feature layers according to the scale distribution of the current input.

[0042] Spatial awareness attention is enhanced by deformable features provided by deformable convolution modules. Spatial saliency maps are extracted through cross-channel average pooling to generate spatial attention weights, enabling the receptive field of the detection head to be dynamically adjusted according to the target shape, thereby improving robustness to complex geometric changes.

[0043] Task-aware attention achieves task-specific feature enhancement by learning feature weights for the classification and localization branches respectively. It allows the classification branch to emphasize channels with strong semantic discriminative power, while the localization branch emphasizes boundary-sensitive channels, thus avoiding conflicts in feature optimization.

[0044] (iv) Hybrid Balanced Loss Technology (HBLT) Traffic monitoring from a drone's perspective faces two fundamental challenges at the training level: class imbalance and scale sensitivity. In real-world datasets, vehicle class samples dominate, while minority classes such as bicycles and motorcycles account for only a small proportion. This extreme class imbalance causes models to tend to ignore minority class samples. Simultaneously, small targets are extremely sensitive to bounding box localization errors; the same pixel-level offset results in a significantly greater decrease in IoU (Intersection over Union) for small targets than for large targets.

[0045] This invention employs a hybrid loss strategy, which utilizes the synergistic effect of Focaler soft label classification loss, WIoU (WiseIntersection over Union) size-aware localization loss, and DFL (Distribution FocalLoss) distribution prediction, as shown in formula (2).

[0046] (2) in For soft label classification loss, For size-aware positioning loss, For distributed prediction loss, , , These are the weighting coefficients for each loss.

[0047] The soft-label classification loss introduces a dynamic modulation factor to reduce the loss weight of easily classified samples, thus focusing training on difficult-to-classify samples. This allows high-quality minority class samples to receive comparable training attention as majority class samples in imbalanced scenarios, effectively mitigating the class imbalance problem.

[0048] Size-aware localization loss achieves adaptive optimization through a combination of dynamic weighting and enhancement functions. This mechanism exhibits non-monotonicity: it avoids over-focusing on extremely difficult samples, maintains sufficient focus on moderately difficult samples, and appropriately reduces weights for simple samples. This non-monotonicity makes the training process more robust, and is particularly crucial for small targets—by amplifying the loss on low IoU samples, the network is forced to learn more accurate bounding box predictions.

[0049] The distributed prediction loss extends bounding box regression from deterministic single-point prediction to probabilistic distributed prediction, enabling the network to explicitly model the uncertainty of the target location and establishing a more principled approach to handling location uncertainty.

[0050] (v) Construction of digital twin dataset This invention utilizes the CARLA high-fidelity simulation platform to construct a virtual urban traffic scenario, and achieves precise configuration of environmental parameters through programmed control.

[0051] Flight parameter dimensions: Three typical altitudes are set: low-altitude monitoring (30 meters), standard cruise (60 meters), and high-altitude monitoring (100 meters); pitch angles are configured as three typical viewing angles: 45°, 60°, and 90°; azimuth angles are sampled at 45° intervals within the range of 0°-360°, for a total of 8 azimuth angles.

[0052] Weather conditions dimension: Four typical scenarios are designed: sunny, light rain, heavy fog, and night.

[0053] The target categories include five core categories: pedestrians, bicycles, motorcycles, cars, and trucks. Bounding box annotations are extracted directly from the CARLA 3D scene graph and calculated by projecting the vertex coordinates of the target's 3D bounding box onto the camera's 2D image plane.

[0054] The final digital twin dataset contains more than 8,500 images, of which 1,835 are high-quality labeled training images.

[0055] (vi) System Visual Interface Design To facilitate the engineering application of the algorithm of this invention, a visual interactive system based on PyQt (Python Qt, a Python graphical user interface framework) was developed. This system adopts a modular architecture design, separating the core object detection algorithm from the front-end presentation layer, thus achieving an organic combination of algorithm capabilities and user interaction.

[0056] The system's overall architecture is divided into three layers: the data layer is responsible for reading and preprocessing image / video data; the algorithm layer integrates the DT-YOLO detection model to achieve target recognition and localization; and the presentation layer uses PyQt to build a graphical user interface, providing an intuitive human-computer interaction experience. Data is transferred between the layers through standardized interfaces to ensure the system's scalability and maintainability. The system adopts a multi-threaded architecture design, separating image acquisition, model inference, and interface rendering into different threads to ensure smooth interface response.

[0057] Example 2: Based on Embodiment 1, but with a difference, the effectiveness and advantages of the proposed digital twin-based multi-target detection method and system for low-altitude domains are characterized using specific example data, specifically including: (i) Detection accuracy has been significantly improved Table 1: Comparison of quantization performance on the VisDrone 2019 validation set

[0058] Table 1 presents detailed quantitative comparison results of DT-YOLO and current mainstream object detection methods on the VisDrone2019 validation set. The comparison methods cover the latest variants of the YOLO family (YOLO11n), methods specifically optimized for UAV perspectives (Drone-YOLO, YOLORACE, OSD-YOLOv10, LEAF-YOLO), and advanced algorithms designed for small object detection (SRM-YOLO, LES-YOLO, Dual-YOLO, S-YOLO). All methods were compared fairly under the same training configuration, trained using the VisDrone2019 training set, and their performance was evaluated on the validation set.

[0059] As can be seen from Table 1, DT-YOLO achieves significant advantages in both detection accuracy and computational efficiency. The DT-YOLO proposed in this invention provides two versions to meet the needs of different application scenarios: DT-YOLO-Full uses the standard YOLO11n backbone network and is designed for high-performance cloud scenarios; DT-YOLO-Lite replaces the standard backbone with a lightweight LSB (Lightweight StarBackbone) backbone network and is designed for edge device deployment.

[0060] In terms of detection accuracy, DT-YOLO-Full achieved an mAP@0.5 of 47.3%, surpassing all comparable methods and improving upon the current best method, S-YOLO, by 2.5 percentage points (a relative improvement of 5.6%).

[0061] More notably, it achieved 28.6% on the more stringent mAP@0.5:0.95 metric, also outperforming S-YOLO (27.8%) by 0.8 percentage points. Precision and recall reached 57.3% and 44.2% respectively, demonstrating an excellent balance between precision and recall.

[0062] Compared to the baseline model YOLO11n, DT-YOLO achieved significant improvements across all metrics. mAP@0.5 increased from 33.1% to 47.3%, an absolute gain of 14.2 percentage points and a relative improvement of 43.0%; mAP@0.5:0.95 increased from 19.2% to 28.6%, a relative improvement of 49.0%. This performance improvement is primarily attributed to the synergistic effect of its four core modules: EWFPN significantly improves the feature representation of small targets through BiFPN weighted fusion and P2 layer high-resolution feature enhancement; MSAE integrates the DyHead dynamic detection head, enabling adaptive multi-scale detection; and HBLT (Hybrid Balanced Loss Technique) employs a hybrid loss strategy to effectively mitigate class imbalance and size sensitivity issues. Although DT-YOLO's computational cost (25.90 GFLOPs) is higher than the baseline, it maximizes accuracy while maintaining similar computational efficiency (close to S-YOLO) by adopting the C3k2 lightweight fusion node and an optimized P2-P4 three-layer detection configuration (removing the redundant P5 layer).

[0063] Among the methods optimized for UAV perspectives, Drone-YOLO (38.1% mAP@0.5), YOLO-RACE (31.6%), OSD-YOLOv10 (33.4%), and LEAF-YOLO (39.7%) all significantly outperformed DT-YOLO. Specifically, compared to the best-performing LEAF-YOLO, DT-YOLO's mAP@0.5 was 7.6 percentage points higher, representing a relative improvement of 19.1%. It's worth noting that while LEAF-YOLO has extremely low parameter counts (1.20M), it requires 800 training epochs to converge, resulting in high time costs. Although OSD-YOLOv10 has 1.60M parameters, its detection accuracy is even lower than the baseline YOLO11n, indicating that excessive lightweighting may sacrifice feature representation capabilities. DT-YOLO, through systematic design, achieves the optimal trade-off between accuracy and efficiency while maintaining a reasonable model size.

[0064] (ii) Enhanced small target detection capability Among the methods optimized for small target detection, the DT-YOLO model designed in this invention demonstrates a powerful ability to capture small targets. Compared to Dual-YOLO, DT-YOLO achieves higher accuracy (47.3% vs 43.1%) with fewer parameters (3.00M vs 4.56M). This is mainly due to the key role of the three attention mechanisms in DyHead within the MSAE module in UAV scenarios: scale-aware attention dynamically adjusts the weights of feature maps at different scales, making the network focus more on high-resolution features containing small targets (P2 layer); spatial-aware attention learns the importance distribution of spatial locations, accurately locating small target regions; and task-aware attention adaptively adjusts feature representations according to the different needs of classification and localization tasks. Statistical analysis shows that the mAP@0.5 for small targets increased from 21.5% to 38.2%, a relative improvement of 77.7%, fully validating the effectiveness of introducing the P2 layer in EWFPN to work in conjunction with MSAE.

[0065] S-YOLO, a recently published method, employs an adaptive gating strategy and dynamic convolution mechanism, achieving an excellent performance of 44.8% mAP@0.5. DT-YOLO surpasses S-YOLO by 2.5 and 0.8 percentage points in mAP@0.5 and mAP@0.5:0.95, respectively. Comparative analysis shows that S-YOLO's adaptive gating strategy mainly optimizes the detection of medium to large targets, while DT-YOLO's design is better suited to the small target scenarios that account for over 70% of the VisDrone dataset. Furthermore, EWFPN's BiFPN weighted fusion mechanism automatically balances the contributions of features at different scales through learnable weights. Post-training weight analysis shows that the P2 layer has the highest weight (wP2=0.65), validating the network's adaptive learning ability for key features of small targets.

[0066] (iii) Lightweight design meets edge deployment requirements To suit edge deployment scenarios, this invention specifically designs a lightweight DT-YOLO model—DT-YOLO-Lite. This lightweight design primarily achieves its purpose by introducing an LSB module and employing grouped convolutions and channel shuffling mechanisms based on the StarNet architecture, compressing the number of parameters from 3.00M to 2.46M, ranking third in parameter count among all compared methods. Although mAP@0.5 is slightly lower than the full version by 1.5 percentage points to 45.8%, it still significantly outperforms all compared methods except S-YOLO. Of particular note is DT-YOLO-Lite's precision of 57.4%, even slightly higher than the full version (57.3%), the highest in the table, indicating that the lightweight backbone network reduces the false positive rate through regularization. The Lite version, with only 1.73 times the number of parameters as LES-YOLO, achieves a 0.8 percentage point improvement in accuracy, demonstrating the optimal precision-efficiency trade-off among lightweight models.

[0067] (iv) Improved training efficiency From the perspective of training epochs, DT-YOLO achieves optimal performance with 300 training epochs, while LEAF-YOLO requires 800 epochs. This is due to the adaptive loss balancing strategy of the HBLT module. HBLT combines Focaler Loss and WIoU Loss: Focaler Loss achieves optimal performance with (1... The pt)γ modulation factor reduces the loss weight of easily classified samples, allowing the model to focus on difficult samples. WIoU Loss introduces size-aware localization loss, balancing the localization error of targets of different sizes through a dynamic non-monotonic modulation factor. Experiments show that after adopting HBLT, the model can achieve the accuracy of the baseline 300 rounds in the first 100 rounds, accelerating model convergence and improving the final accuracy, significantly reducing the overall cost of algorithm development and deployment. To further verify the effectiveness of the proposed DT-YOLO framework, this invention conducts a systematic qualitative visualization analysis on complex real-world UAV scenarios in the VisDrone2019 dataset. Specific results are as follows: Figure 1 As shown in the figure (the figure shows the detection performance of different models at different confidence thresholds, and provides a magnified view of the local area of ​​fine-grained category confusion). Figure 1 The detection results of three model variants, Baseline, DT-YOLO Full and DT-YOLO Lite, were compared at confidence thresholds of 0.2 and 0.6, respectively. The scenarios covered a variety of typical traffic conditions, including nighttime roads, complex urban intersections, dense traffic, and significant changes in lighting.

[0068] At a confidence threshold of 0.2, the Baseline model misclassified the tricycle as a regular car. This error primarily stemmed from the highly similar outlines of the targets from the UAV's top-down view, and the Baseline network's insufficient ability to model multi-scale detail features. In contrast, both DT-YOLO Full and DT-YOLO Lite accurately identified the true category of the target, fully demonstrating the significant advantages of the LSB module and EWFPN structure in fine-grained discrimination and multi-scale semantic enhancement. When the confidence threshold was increased to 0.6, the Baseline exhibited significant missed detections, while DT-YOLO could still stably detect small targets with a size of only about 12×12 pixels, thus verifying the effectiveness of the MSAE module in adaptive enhancement of targets at different scales. DT-YOLO also demonstrated stronger robustness in complex nighttime lighting scenarios, further illustrating the superiority of the dual-domain joint training strategy in reducing the distribution differences between the simulation and real domains.

[0069] (v) Strong cross-domain generalization ability To fully verify DT-YOLO's detection performance and cross-domain generalization ability, Figure 2 The results show the comparison between the algorithm in the VisDrone 2019 real-world and CARLA digital twin simulation environments (the figure covers the detection performance of all ten types of traffic participants, dense target recognition in real complex scenes, and viewpoint generalization performance in the digital twin environment).

[0070] The visualization results showcase the algorithm's performance across three dimensions. First, in terms of class coverage, the algorithm successfully detected targets at all scales, from Pedestrian to Bus, ensuring that the detection boxes closely fit the target boundaries even in scenes with densely packed small targets. Second, regarding robustness in complex environments, the middle scene demonstrates the algorithm's stable performance under conditions of intersection occlusion, long-distance detection on urban main roads, and shadow interference in residential areas. Finally, in terms of cross-domain generalization, the lower image confirms that DT-YOLO achieves high-quality zero-shot transfer without fine-tuning in unseen digital twin environments, maintaining high confidence output even when targets exhibit significant perspective distortion or are in low-light conditions.

[0071] At the level of quantitative indicators and mechanism analysis Figure 3 The training performance of DT-YOLO is comprehensively presented (the figure includes the normalized confusion matrix, data distribution statistics, and multiple precision and recall curves). The data distribution plot shows that the dataset exhibits a significant long-tail structure, with the vehicle class accounting for nearly half of the data, while some smaller sample classes have extremely low percentages.

[0072] To address this severe class imbalance issue, DT-YOLO dynamically adjusts the loss weights using Focaler Loss and Dice Loss in HBLT, significantly improving the detection accuracy for small-sample classes compared to the baseline. Simultaneously, correlation analysis reveals that, due to a large number of targets concentrated in specific pixel ranges and the proportion of small targets exceeding 70%, this framework significantly enhances the recall performance for small-scale targets by leveraging the high-resolution P2 layer of EWFPN and the scale-aware attention of MSAE. Although the confusion matrix shows some degree of ambiguity between vehicle subcategories at extremely long distances, DT-YOLO effectively mitigates this problem through its task-aware attention mechanism. Combined with trends in F1-Confidence curves, it can be seen that DT-YOLO achieves a better recall balance while maintaining high accuracy.

[0073] The combined results of visualization and quantitative analysis demonstrate that DT-YOLO exhibits excellent performance in both the real world and the digital twin world, successfully covering all 10 categories. The algorithm maintains stable detection capabilities under various challenging conditions, and its cross-domain generalization performance proves the practical value of this framework in traffic monitoring from a UAV perspective, providing strong algorithmic support for building intelligent transportation systems that integrate real and virtual environments.

[0074] Based on the digital twin simulation platform CARLA, this invention conducts a systematic quantitative and visual comprehensive experimental analysis of the DT-YOLO method. In the quantitative experimental part, this invention focuses on the performance of DT-YOLO in virtual scenarios and its verification value. From an experimental design perspective, the core objective of the CARLA experiment is to verify the effectiveness and generalization ability of the algorithm in a digital twin environment. The value of digital twin technology lies not only in evaluating the algorithm's ability to migrate from the real domain (VisDrone2019) to the virtual domain, but also in its adaptive analysis of multi-dimensional changing factors under controlled environments. This cross-domain verification strategy demonstrates the robustness of the algorithm under different data distributions, thus providing reliable performance guarantees for practical deployment. The evaluation focus of this invention is to conduct a comprehensive sensitivity analysis and performance boundary exploration of DT-YOLO through multi-dimensional scenario design, thereby jointly supporting the core assertion of the algorithm's cross-domain generalization ability with the real-domain experiments.

[0075] Table 2: Performance Comparison of Real Domain (VisDrone2019, DOTA 1.0) and Digital Twin Domain (CARLA)

[0076] Table 2 shows the performance comparison of the DT-YOLO series models in the real-world domain (VisDrone2019, DOTA1.0) and the digital twin domain (CARLA). Experimental results show that DT-YOLO-Lite achieves an mAP@0.5 of 98.10% on the CARLA validation set, with precision and recall reaching 96.90% and 95.40%, respectively. Compared with the real dataset, the absolute values ​​of performance metrics are higher in the virtual environment, mainly due to the higher annotation accuracy and relatively balanced target scale distribution in the digital twin scenario. More importantly, the algorithm demonstrates stable and consistent detection capabilities across domains with different feature distributions: in the VisDrone2019 task, DT-YOLO-Lite improved mAP@0.5 by 12.68% (from 33.10% to 45.78%) compared to the baseline model yolo11n; in the DOTA1.0 task, DT-YOLO-Lite achieved a 4.32% mAP@0.5 gain despite having fewer parameters (2.46M) than the baseline model (2.60M); in terms of efficiency, DT-YOLO-Lite reduced computational cost (GFLOPs) by approximately 6.6% compared to the Full version while maintaining extremely high accuracy, demonstrating excellent potential for deployment. In terms of relative performance, DT-YOLO showed improved recall on CARLA, mainly due to clearer target boundaries and fewer occlusions. Regarding cross-domain generalization ability, the DT-YOLO series exhibited stronger domain adaptability. Specifically, the baseline model YOLO 11n exhibits a performance fluctuation of 63.26% between the real-world domain (VisDrone: 33.10%) and the digital twin domain (CARLA: 96.36%), while the cross-domain fluctuation of DT-YOLO-Lite is only 52.32% (from 45.78% to 98.10%). This difference indicates that the four core modules proposed in this study—LSB, EWFPN, MSAE, and HBLT—not only improve the overall detection accuracy but also significantly enhance the algorithm's robustness and generalization ability under different data distributions, enabling it to demonstrate more stable detection performance in real-world complex scenarios.

[0077] (vi) Strong multi-dimensional adaptability Figure 4The demonstration showcases DT-YOLO's detection performance under four typical weather conditions in the CARLA digital twin environment (from top to bottom: Clear, Fog, Rainy, and Nighttime; each row displays detection results from five different perspectives and scenes. Detection categories include cars, pedestrians, motorcycles, trucks, and bicycles; the values ​​next to the detection boxes are confidence scores). The system validates the algorithm's robustness under complex environmental interference.

[0078] Under clear conditions, DT-YOLO demonstrates optimal detection performance, serving as a benchmark for ideal environments. The first row of visualizations shows that the algorithm accurately detects multiple target categories, including cars, pedestrians, motorcycles, and trucks, with confidence levels generally ranging from 0.78 to 0.94. At different viewing angles, the multi-scale feature fusion strategy of the EWFPN module ensures stable bounding box output, while the scale-aware attention mechanism of MSAE enables reliable detection of small, distant targets. This high performance in clear weather provides a benchmark for subsequent performance evaluation under adverse weather conditions.

[0079] Fog conditions introduce significant visibility reduction and feature blurring interference. As shown in the second row of visualizations, despite the overall hazy image, DT-YOLO can still accurately identify vehicles at intersections, maintaining a confidence level between 0.62 and 0.83. The star-shaped operation in the LSB backbone network enhances target feature representation under blurred conditions through nonlinear feature mapping, while the WIoU loss function in HBLT effectively reduces the false detection rate by adaptively adjusting weights for low-quality samples. Even in extremely low visibility scenarios, the algorithm successfully detects nearby vehicle targets, demonstrating DT-YOLO's basic functional capabilities under extreme weather conditions.

[0080] In rainy conditions, water droplet noise and road surface reflection constitute a dual interference. The visualization results in the third row show that DT-YOLO exhibits good adaptability to rainy scenes, maintaining a detection confidence level within the range of 0.65–0.97. In well-lit rainy scenes, the algorithm achieves accurate detection of small-scale targets such as pedestrians and bicycles that are easily obscured by raindrops, thanks to the enhanced representation capability of the P2 high-resolution detection layer in EWFPN for small targets. The specular reflection caused by road surface water did not lead to significant false detections, demonstrating the effective suppression of difficult samples by Focaler Loss.

[0081] Nighttime conditions represent the most challenging detection scenarios. As observed in the fourth row of visualizations, DT-YOLO maintains reliable detection performance under low-light conditions, achieving confidence levels of 0.84–0.94 for vehicle targets. The dynamic attention mechanism of DyHead in the MSAE module effectively enhances the distinction between targets and background in nighttime scenes by adaptively adjusting feature weights at different scales and spatial locations, fully utilizing the limited light source information provided by vehicle taillights and streetlights. However, the detection performance for small targets such as pedestrians and bicycles deteriorates in nighttime scenes (some confidence levels fall below 0.6). This is because the feature information of small targets is severely attenuated under low-light conditions, making it difficult for even the high-resolution P2 layer to provide sufficient discriminative features.

[0082] Visual analysis of four weather conditions demonstrates that DT-YOLO maintains stable detection capabilities across the entire spectrum of meteorological environments, from ideal to extreme conditions, indicating minimal performance fluctuations under varying weather conditions. The synergistic effect of the four core modules—LSB, EWFPN, MSAE, and HBLT—enables the algorithm to effectively address diverse environmental challenges such as reduced visibility, feature blurring, noise interference, and insufficient lighting, fully showcasing the unique value of digital twin technology in algorithm verification.

[0083] Figure 5 The results of DT-YOLO's multi-dimensional detection on the CARLA digital twin dataset were further demonstrated. Through controlled experiments covering various weather conditions, flight altitudes, and observation angles, the robustness of the algorithm in complex dynamic scenarios was systematically verified.

[0084] From a category perspective, DT-YOLO demonstrates stable detection capabilities across all five target classes. For large-scale targets such as cars and trucks, the algorithm achieves accurate localization detection under various weather conditions. For small-scale targets such as pedestrians, motorcycles, and bicycles, the EWFPN's introduction of a high-resolution P2 detection layer provides ample spatial detail, enabling effective identification of these easily missed targets in different scenarios. Particularly under low-visibility conditions such as fog and nighttime, the detection boxes for small-scale targets remain clearly visible, fully validating the adaptive enhancement effect of the dynamic attention mechanism in the MSAE module for multi-scale targets.

[0085] From a spatial perspective, it can be seen that changes in flight altitude and observation angle directly affect the imaging scale and morphological characteristics of the target. Figure 5The diagram on the right illustrates the differences in field of view when the UAV flies at low, medium, and high altitudes. At low altitudes, targets are larger but the field of view is limited, while at high altitudes, targets tend to be smaller but the coverage area is wider. The visualization results show that DT-YOLO effectively integrates multi-level feature information through the bidirectional feature fusion strategy of EWFPN, enabling the algorithm to maintain consistent detection performance at different flight altitudes. Regarding the viewing angle, the change from a direct overhead view to an oblique view introduces varying degrees of perspective distortion, which the LSB backbone network captures through nonlinear feature mapping, thus ensuring stable output of the detection boxes under multi-angle conditions.

[0086] From the perspective of cross-sectional observation of weather and spatial dimensions, performance under extreme conditions is particularly crucial. DT-YOLO still accurately captures targets under extreme combinations of nighttime, high altitude, and oblique view, demonstrating the algorithm's robustness under multiple adverse factors. In contrast, the baseline model exhibits more significant performance fluctuations under the same complex conditions. This qualitative observation indicates that DT-YOLO effectively reduces its sensitivity to environmental conditions and significantly improves detection stability in complex and variable scenarios through the collaborative design of its four core modules.

[0087] Comprehensive multi-dimensional visualization analysis reveals that DT-YOLO demonstrates excellent overall adaptability across four dimensions: weather, altitude, angle, and category. This multi-dimensional controlled experimental design fully leverages the advantages of digital twin technology, providing intuitive performance evaluation and technical support for the algorithm's deployment in real-world UAV traffic monitoring scenarios.

[0088] Figure 6 The training performance visualization results of DT-YOLO on the CARLA digital twin dataset are presented (the peak F1 score for all classes is concentrated in the range of 0.8-0.9, showing balanced detection performance and verifying the optimization effect of the HBLT module on class imbalance). Multi-dimensional analysis fully verifies the learning characteristics and detection capabilities of the algorithm in the virtual environment.

[0089] The confusion matrix reveals that the classification accuracy for all five categories in the CARLA dataset exceeds 90%, significantly higher than the average accuracy of approximately 70% for the VisDrone2019 dataset. This performance improvement is primarily attributed to clearer target boundaries and less inter-class confusion in the digital twin environment. Specifically, the Car class achieves 98% accuracy with only a slight 2% confusion with Truck, while Pedestrian achieves 95% accuracy with only a 3% confusion with Bicycle. Compared to the up to 40% confusion rate between vehicle subcategories in real-world scenarios, the performance in the CARLA environment demonstrates the advantages of high-quality annotation and clear visual features in virtual data, while also defining the performance ceiling of DT-YOLO's class discrimination ability under ideal conditions.

[0090] Data distribution statistics show that the CARLA dataset exhibits a relatively balanced distribution across the five categories. The Car class accounts for approximately 40%, while the other four classes account for between 12% and 18%, with a class ratio of approximately 4:1. This contrasts sharply with the extremely long-tailed distribution of VisDrone2019. This balance means that the Focaler Loss in the HBLT module does not need to overcompensate weights, resulting in relatively consistent learning performance across categories. Nevertheless, slight differences remain in the peak F1 scores for different categories. The peak F1 score for Car is 0.92, while for Bicycle it is 0.85, indicating that the physical size and visual feature complexity of the target remain key factors affecting detection difficulty.

[0091] Correlation analysis of the bounding boxes revealed the spatial distribution patterns of the targets. Target locations were mainly concentrated in the central image region, with aspect ratios mostly falling between 0.2 and 0.4. Due to the relatively uniform distribution of target scales, with small targets accounting for approximately 35% and medium-to-large targets accounting for approximately 65% ​​combined, this distribution is beneficial for the model to be trained sufficiently at different scales. The three-layer detection configuration of EWFPN was fully utilized on the CARLA dataset, and the contribution of each detection layer was highly consistent with the target scale distribution.

[0092] The F1-Confidence curves illustrate the performance trade-offs of different classes at different confidence thresholds. The peak F1 scores for all classes are concentrated in the range of 0.80 to 0.92, significantly higher than performance in real-world scenarios. The differences in peak confidence reflect the varying detection difficulty; large targets such as cars and trucks can maintain high F1 scores at higher confidence levels, while small targets require a lower threshold to balance precision and recall. This pattern aligns with observations in real-world data, demonstrating the similarity between virtual and real-world data in terms of target physical characteristics.

[0093] The recall-confidence curves further validated the algorithm's recall performance. At a confidence threshold of 0.3, the recall rate for all classes exceeded 95%, and even when the threshold was increased to 0.5, it remained above 90%. This high recall performance is attributed to the high-resolution features of EWFPN's P2 layer and the dynamic detection head of MSAE, enabling the algorithm to effectively capture the vast majority of target instances. Meanwhile, the precision-recall curves showed that the mAP@0.5 for all classes was above 0.95, and the curve shape was close to an ideal rectangle. This indicates that the algorithm maintains a high recall rate while keeping the false positive rate at an extremely low level.

[0094] In summary, the above analysis shows that DT-YOLO exhibits near-ideal detection performance in a digital twin environment, including extremely low inter-class confusion, high recall, and excellent average precision. These results not only validate the effectiveness of the algorithm design but also fully demonstrate the unique value of digital twin technology in algorithm verification.

[0095] Through comprehensive multi-dimensional quantitative and visual analysis, DT-YOLO demonstrated excellent overall adaptability in cross-validation across four dimensions: weather, altitude, angle, and category. The dual-domain validation system of CARLA and VisDrone 2019 complements each other, using real data to assess actual performance while virtual data explores performance limits. This evaluation model provides reliable technical support and comprehensive performance assurance for the practical deployment of DT-YOLO in UAV traffic monitoring systems.

[0096] (vii) Data acquisition costs have been significantly reduced. By constructing a virtual dataset using digital twin technology, this system systematically covers long-tail samples such as extreme weather and dangerous scenarios that are difficult to obtain in the real world, fundamentally overcoming the security and economic constraints of real-world data collection. The digital twin dataset contains over 8,500 images with pixel-level annotation accuracy, completely avoiding the inherent subjective errors of manual annotation.

[0097] Example 3: Based on Embodiments 1-2, but with some differences, the following describes the proposed digital twin-based multi-target detection method and system for low-altitude domains with specific examples. The aim is to achieve high-precision, lightweight, and multi-weather-adaptive traffic target detection from the perspective of UAVs. The specific content is as follows.

[0098] (a) Traffic flow prediction applications To verify the practicality of the DT-YOLO algorithm in intelligent transportation systems, this invention introduces a Long Short-Term Memory (LSTM) network to predict traffic flow at urban intersections. This prediction module utilizes vehicle detection results from the CARLA digital twin dataset to establish a closed-loop verification framework connecting detection accuracy with downstream traffic management applications.

[0099] The LSTM network processes the detected vehicle count time series at 5-second intervals using a sliding window of 5 time steps. The training data consists of 100 minutes of continuous traffic flow (1200 sample points), while the test data covers 20 minutes with 240 sample points. The network architecture uses a 3-layer LSTM structure with 50 hidden units, trained for 100 epochs using the Adam optimizer with a learning rate of 1×10⁻⁶. -3 .

[0100] Please see Figure 7 , Figure 7 This paper presents a comparison between predicted and actual traffic flow on the CARLA intersection dataset. The solid red line represents actual traffic volume, and the dashed blue line represents the LSTM prediction. Quantitative evaluation results show that the model has excellent predictive performance, with a mean squared error (MSE) of 0.6048, a root mean squared error (RMSE) of 0.7777, and a mean absolute error (MAE) of 0.5226 vehicles. Observing the curves in the figure, it can be seen that the predicted curve, represented by the dashed blue line, closely tracks the actual traffic flow pattern represented by the solid red line. The model performs robustly under different traffic density scenarios, fully covering the numerical range from approximately 12 vehicles during peak hours to nearly 0 vehicles during low-flow hours.

[0101] The achieved prediction accuracy validates two key advantages of the digital twin approach. First, DT-YOLO provides sufficiently accurate detection results to support reliable traffic flow estimation. Second, the virtual environment can collect comprehensive time-series data that is difficult to obtain in real drone deployments, thus providing a complete testing platform for the development and validation of prediction models before physical implementation.

[0102] This application demonstrates the synergistic effect of the detection algorithm and the digital twin method. By proving its effectiveness in both spatial accuracy of target detection and temporal patterns of traffic flow prediction, this study establishes DT-YOLO as a feasible solution for UAV-based intelligent traffic monitoring systems in the context of a low-altitude economy. This end-to-end validation framework not only proves the algorithm's detection performance but also demonstrates its practical value in real-world traffic management applications, thus laying a solid foundation for future deployment on real UAV platforms.

[0103] (ii) System integration testing To facilitate system deployment and migration, this system is developed using Python and relies on the PyTorch deep learning framework and the PyQt graphical interface library. The system supports Windows and Linux operating systems and can generate standalone executable files using packaging tools, eliminating the need for users to configure complex runtime environments.

[0104] The system's initial interface adopts a simple design style, such as... Figure 8 As shown, the central area displays the system name "Digital Twin-Based Multi-Target Detection System for Low-Altitude Domain," with two main function buttons below: "Enter System" and "Exit System." After the user clicks "Enter System," the system will load the detection model and redirect to the real-time monitoring interface.

[0105] The real-time monitoring interface adopts a three-column layout design, such as... Figure 9 As shown. The left area is the raw input screen display area, used to display the raw aerial images or video frames captured by the drone; the middle area is the detection result screen display area, which presents the detection results processed by the DT-YOLO algorithm in real time, including target bounding boxes, category labels, and confidence scores; the right area is the detection statistics panel, which dynamically displays the real-time count of various targets, including five categories: pedestrians, cars, trucks, motorcycles, and bicycles.

[0106] The detection statistics panel uses a dual encoding method of numbers and colors. Category names are displayed in gray font, while the corresponding counts are presented in striking red numbers, allowing users to quickly obtain key information. The system supports real-time updates of statistical results; the statistics panel will refresh synchronously when the number of targets in the detection screen changes.

[0107] The system has a function button area at the bottom, including two operation buttons: "Load Traffic Scene Image" and "Return to Main Menu". Users can use the "Load Traffic Scene Image" button to select local image or video files for detection, or use the "Return to Main Menu" button to return to the system's main interface.

[0108] The system supports multiple data source inputs, including local image files, video files, and real-time camera streams. In image detection mode, the system supports batch processing of multiple images; in video detection mode, the system processes images in real time on a frame-by-frame basis and overlays detection boxes and label information onto the detection results; in real-time detection mode, the system can connect to drone image transmission devices or surveillance cameras to achieve continuous real-time detection.

[0109] The detection results visualization employs a color-coding strategy, with different colored bounding boxes used to label different categories of targets. This color-coding mechanism helps users quickly identify and distinguish between different types of targets. Above each detection box is labeled the target category and confidence score, in the format "Category Confidence" (e.g., "car 0.87"), with the confidence score rounded to two decimal places.

[0110] The above description is merely a preferred embodiment of the present invention and is illustrative rather than restrictive. However, the scope of protection of the present invention is not limited thereto. Any equivalent substitutions, modifications, omissions, combinations, or other changes made by those skilled in the art within the technical scope disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be included within the scope of protection of the present invention.

Claims

1. A low-altitude field multi-target detection method based on digital twinning, characterized in that, Includes the following steps: Using drones to acquire drone aerial images; A YOLO end-to-end detection framework based on digital twins is proposed. The obtained UAV aerial images are input into the pre-trained YOLO end-to-end detection framework based on digital twins, and the multi-target detection results are output. The YOLO end-to-end detection framework based on digital twins includes: A lightweight star-shaped backbone network extracts features from the input image through the synergistic effect of grouped convolution, channel shuffling, and star-shaped aggregation mechanism, and outputs multi-scale feature maps. An efficient weighted feature pyramid network, connected to the lightweight star-shaped backbone network, is used to receive multi-scale feature maps and constructs an enhanced multi-scale feature pyramid by introducing learnable weighted fusion, high-resolution enhancement and bidirectional information propagation mechanisms. A multi-scale adaptive enhancement module, connected to an efficient weighted feature pyramid network, is used to adaptively enhance the enhanced multi-scale feature pyramid through a three-dimensional attention mechanism to generate detection features. The detection head, connected to the multi-scale adaptive enhancement module, is used to perform target classification and bounding box regression based on the detection features, and output multi-target detection results. The YOLO end-to-end detection framework based on digital twins is trained using a hybrid balanced loss technique, which optimizes network parameters by combining soft label classification loss, size-aware localization loss, and distribution prediction loss.

2. The method according to claim 1, characterized in that, The efficient weighted feature pyramid network introduces a P2 layer to participate in feature fusion and detection; the P2 layer has a higher resolution than the minimum detection layer in the standard feature pyramid, which is used to enhance the spatial feature representation of small targets. The learnable weighted fusion mechanism assigns learnable weight parameters to each feature branch, enabling the fusion operation to adaptively adjust the importance of features according to task requirements. The bidirectional information propagation mechanism is used to ensure the effective coupling of semantic information and detailed features in the pyramid hierarchy; The functional representation of the multi-scale feature fusion process is as follows: in, This represents the fused output feature map; Represents the learnable weight parameters; Let i be the i-th input feature map; This represents a numerically stable term.

3. The method according to claim 1, characterized in that, The three-dimensional attention mechanism specifically includes: Scale-aware attention is used to dynamically learn and adjust the importance of features at different scales through a self-attention mechanism. Spatial awareness attention is used to generate spatial attention weights using deformable convolution, so that the receptive field can be dynamically adjusted according to the shape of the target. Task-aware attention is used to learn feature weights for the target classification and bounding box regression branches respectively, thereby achieving task-specific feature enhancement.

4. The method according to claim 1, characterized in that, The total loss function of the hybrid balance loss technique is expressed as follows: in, This is a soft-label classification loss, used to reduce the weight of easily classified samples and focus on difficult-to-classify samples by using a dynamic modulation factor; Size-aware localization loss is used to adaptively weight targets of different sizes and difficulty levels using a non-monotonic modulation function; This is the distributed prediction loss, used to model bounding box regression as a probability distribution prediction; , , These are the weighting coefficients for each loss.

5. The method according to claim 1, characterized in that, The pre-training process of the YOLO end-to-end detection framework based on digital twins includes the step of constructing a synthetic dataset based on digital twin technology, specifically including: Utilize a simulation platform to construct a high-fidelity digital twin of urban traffic scenarios; In the constructed digital twin urban traffic scenario, flight parameters, weather conditions, and traffic density are programmed to generate diverse monitoring scenarios; Images are collected from the constructed digital twin urban traffic scene, and corresponding target bounding box annotations are automatically generated based on the 3D target information in the scene to form a digital twin dataset for pre-training.

6. The method according to claim 5, characterized in that, The flight parameters include flight altitude, pitch angle, and azimuth angle; the flight altitude is configured to include low-altitude monitoring altitude, standard cruise altitude, and high-altitude monitoring altitude; the pitch angle is configured to include top-down and tilt angles; the azimuth angle is configured to be sampled at preset intervals within the range of 0°-360°. The weather conditions are configured to include at least two of the following: sunny, light rain, heavy fog, and night.

7. The digital twin-based multi-target detection system for low-altitude domains, using the method described in any one of claims 1-6, is characterized in that... include: The data acquisition module is used to acquire aerial images taken by the drone; The target detection module integrates a pre-trained YOLO end-to-end detection framework based on digital twins, which is used to process drone aerial images and output multi-target detection results. The visualization and interaction module is used to receive and display multi-target detection results and provide a human-computer interaction interface.

8. The system according to claim 7, characterized in that, The visualization interaction module is built on the PyQt framework and adopts a modular architecture that separates the data layer, algorithm layer and presentation layer. It also uses a multi-threaded architecture to allocate image acquisition, model inference and interface rendering to different threads for execution.

9. A computer device, characterized in that, The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set, or instruction set, and the instruction, program, code set, or instruction set is loaded and executed by the processor to implement the digital twin-based multi-target detection method for low-altitude domains as described in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, at least one program, code set, or instruction set, which is loaded and executed by a processor to implement the digital twin-based multi-target detection method for low-altitude domains as described in any one of claims 1-6.

Images