Detection and tracking of drone swarms using lidar

WO2026221713A3PCT designated stage Publication Date: 2026-11-12UNIV OF MASSACHUSETTS +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/US2025/053787
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2024-11-04
Filing Date
2025-11-03
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

Existing methods for detecting and tracking drone swarms are limited by radar's inability to effectively detect non-metallic drones near the ground, visual tracking's daylight dependency, and audio tracking's difficulty in differentiating drones from birds and at long distances.

Method used

Employing LiDAR for precise localization and using machine learning to generate and classify point clouds, distinguishing drones from other objects, and tracking them in real-time.

Benefits of technology

Enables real-time detection and tracking of large drone swarms, improving monitoring and defense capabilities by accurately identifying and following drone trajectories.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure US2025053787_12112026_PF_FP_ABST
    Figure US2025053787_12112026_PF_FP_ABST
Patent Text Reader

Abstract

An exemplary system and method that employs LiDAR (light detection and ranging) to detect and track drone swarms by generating a LiDAR point cloud, filtering the point clouds, applying machine learning (ML) to recognize point clouds as drones to distinguish them from other objects such as birds, and tracking the identified drones. The exemplary system may be employed on a large scale of up to hundreds of drones in real time. The exemplary system may implement a pipeline operation on stationary or mobile platforms.
Need to check novelty before this filing date? Find Prior Art

Description

Attorney Docket No. 11555-017WO1UMA No. 25-006 DETECTION AND TRACKING OF DRONE SWARMS USING LIDAR RELATED APPLICATIONThis application claims priority to, and the benefit of, U. S. Provisional Patent Application No. 63 / 716,096, filed November 04, 2024, entitled “Detection and Tracking of Drone Swarms Using Lidar,” which is incorporated by reference herein in its entirety.BACKGROUND

[0001] The detection and tracking of drone swarms pose a considerable challenge. Radar is often limited since drones are small, in many cases composed of non -metallic content that doesn’t reflect radar waves well, and fly relatively close to the ground to be easily obscured by¬ ground clutter. Visual tracking of drone swarms is limited to daylight operation only, and drones can have difficulty differentiating from birds at a distance. Audio tracking of drones is feasible when there is a small number of drones, but has difficulty differentiating individual drones in a large swarm, and suffers at long distances.

[0002] There is a benefit to improving the detection and tracking of drone swarms.SUMMARY

[0003] An exemplary system and method are disclosed that employs LiDAR (light detection and ranging) to detect and track drone swarms. Lidar has the advantages of precise localization, accuracy of identification, and robust operation at night. The exemplary system generates a LiDAR point cloud, filters the point clouds, and classifies the point cloud using machine learning (ML) or other Al (such as neural networks) to recognize the individual points as drones to distinguish them from other objects such as birds, and tracks the identified drones. The exemplary system may be employed on a large scale of up to hundreds of drones in real¬ time. The exemplary system may implement a pipeline operation.

[0004] Drone swarms have become increasingly commonplace both in civilian and military applications, such as light shows and combat operations. The exemplary system can provide real-time detection and tracking of large drone swarms. For drone light show's, it allows monitoring and tracking of, for example, faulty drones that may have dropped out of the formation and are in need of recovery. For a battlefield, drone swarm tracking operations maybe used to protect against attack from drone swarms and can be used as part of a counter-drone strategy to interdict enemy drones. The drone swarm monitoring operations may be used to monitor sensitive civilian and military locations and facilities, such as airports, prisons, buildings, and bases.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0005] In an aspect, a system is disclosed comprising: a processor; and a memory having instructions stored thereon for detecting and monitoring a drone swarm, wherein execution of the instructions by the processor causes the processor to: receive LiDAR measurement of a LiDAR scan; generate a point cloud data object for the LiDAR measurement; generate, via a computer vision operation, a cluster of the point cloud data object; merge the cluster, or an intermediate data object (e.g., alphashape, Delaunay triangle, or other mesh objects); and classify, via a trained Al classifier, drones in the point cloud data object and exclude non-drone objects, wherein the classified outputs are used for detecting and monitoring a drone swarm. The use and merging of multiple time scans improves the detection of drones and removal of outliers (non-drones) while the multiple time scans also provides for tracking (e.g., trajectory of drones and changes in trajectory).

[0006] In some embodiments, the instructions to generate the point cloud data object include: instructions to merge the shape object of the cluster generated at different time scans (e.g., via a window approach), wherein the classified outputs are generated from the merged shape object of the cluster.

[0007] In some embodiments, the trained Al classifier is a binary classifier.

[0008] In some embodiments, the trained Al classifier was trained using the shape object of the cluster of LiDAR scans.

[0009] In some embodiments, the system is a stationary platform.

[0010] In some embodiments, the system is a mobile platform.

[0011] In some embodiments, the instructions, when executed by the processor, further cause the processor to track individual drones within the swarm.

[0012] In some embodiments, the instructions to track the individual drones include: instructions to: merge a plurality of the classified outputs at different time scans to generate a time sequence of the classified outputs; interpolate the time sequence of the classified outputs; and reconstruct trajectories of each classified output in the interpolated time sequence of classified outputs.

[0013] In some embodiments, the instructions to track the individual drones include instructions to plot the time sequence of the classified outputs and a trajectory path of the individual drones.

[0014] In some embodiments, the classified outputs are used for the detection of a drone or drone swarm being in proximity to a pre-defined location, and the classified outputs are used to generate an alert.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0015] In some embodiments, the classified outputs are used for the detection of a drone or drone swarm being out of formation of a pre-defined swarm formation, and the classified outputs are used to generate an alert or for control of the swarm.

[0016] In some embodiments, the execution of the instructions further causes the processor to: receive (i) a second LIDAR measurement of a second LiDAR scan and / or (ii) a second sensor measurement of a second sensor scan (e.g., thermal scan), each acquired concurrently with the LiDAR measurement, wherein the second LiDAR scan and / or the second sensor scan are time synchronized with the LiDAR measurement; generate a respective point cloud data object for the second LiDAR measurement or the second measurement; generate, via a computer vision operation, a respective cluster of the point cloud data object; generate, via a triangulation operation, a respective shape object of the respective cluster (e.g., alphashape, Delaunay triangle, or other mesh objects); merge the shape object of the cluster generated at different time scans and the respective shape object of the respective cluster at the same different time scans; and classify, via the trained Al classifier, drones in the merged point cloud data object and exclude non-drone objects.

[0017] In some embodiments, the second sensor measurement or the second sensor scan is a thermal measurement by a thermal camera.

[0018] In another aspect, a method is disclosed comprising: receiving LiDAR measurement of a LiDAR scan; generating a point cloud data object for the LiDAR measurement; generating, via a computer vision operation, a cluster of the point cloud data object; generating, via a triangulation operation, a shape object of the cluster (e.g., alphashape, Delaunay triangle, or other mesh objects); and classifying, via a trained Al classifier, drones in the point cloud data object and exclude non-drone objects, wherein the classified outputs are used for detecting and monitoring a drone swarm.

[0019] In some embodiments, the classification of the drones includes: merging the shape object of the cluster generated at different time scans (e.g., via a window approach), wherein the classifier output is generated from the merged shape object of the cluster.

[0020] In some embodiments, the trained Al classifier is a binary classifier.

[0021] In some embodiments, the trained Al classifier was trained using the shape object of the cluster of LiDAR scans.

[0022] In some embodiments, the method described herein further includes: tracking individual drones within the swarm.

[0023] In some embodiments, the tracking of the individual drones includes: merging a plurality of the classified outputs at different time scans to generate a time sequence ofAttorney Docket No. 11555-017WO1UMA No. 25-006 classified outputs; interpolating the time sequence of classified outputs; and reconstructing trajectories of each classified output in the interpolated time sequence of classified outputs.

[0024] In yet another aspect, a non-transitory computer-readable medium having instructions stored thereon is disclosed, wherein execution of the instructions causes a processor to: receive LiDAR measurement of a LiDAR scan; generate a point cloud data object for the LiDAR measurement; generate, via a computer vision operation, a cluster of the point cloud data object; generate, via a triangulation operation, a shape object of the cluster (e.g., alphashape, Delaunay triangle, or other mesh objects); and classify, via a trained Al classifier, drones in the point cloud data object and exclude non-drone objects, wherein the classified outputs are used for detecting and monitoring a drone swarm.

[0025] In some embodiments, the classification of the drones includes: merging the shape object of the cluster generated at different time scans (e.g., via a window approach), wherein the classifier output is generated from the merged shape object of the cluster.

[0026] In some embodiments, the trained Al classifier is a binary classifier.

[0027] In some embodiments, the trained Al classifier was trained using the shape object of the cluster of LiDAR scans.

[0028] In some embodiments, the instructions, when executed by the processor, further cause the processor to track individual drones within the swarm.

[0029] In some embodiments, the tracking of the individual drones includes: merging a plurality of the classified outputs at different time scans to generate a time sequence of classified outputs; interpolating the time sequence of classified outputs; and reconstructing trajectories of each classified output in the interpolated time sequence of classified outputs.

[0030] Other systems, methods, features, and / or advantages will be or may become apparent to one with skill in the art upon examination of the following drawings and detailed descriptions. It is intended that all such additional systems, methods, features, and / or advantages be included within this description and be protected by the accompanying claims.BRIEF DESCRIPTION OF DRAWINGS

[0031] Figs. 1 A - IB each shows an example system for detecting and tracking a drone or a drone swarm, in accordance with an illustrative embodiment.

[0032] Fig. 2 shows an example operation flow for the exemplary system, in accordance with an illustrative embodiment.

[0033] Fig. 3A shows an example LiDAR system for drone swarm detection and tracking in accordance with an illustrative embodiment.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0034] Fig. 3B shows point clouds of unmanned aerial vehicles (UAVs) (e.g., UAV1, UAV2, UAV3) and a Mallard bird (Anas platyrhynchos) at 3 different frame times.

[0035] Fig. 3C shows example point clouds of UAV1 & 4 different birds with varying poses at different ranges with a 500ms frame time.

[0036] Fig. 4 shows an example drone or drone swarm detection and tracking system, in accordance with an illustrative embodiment.

[0037] Fig. 5 shows an example algorithmic implementation of a detection and classification operation in the exemplary system.

[0038] Fig. 6 shows an example algorithmic implementation of a tracking operation in the exemplary system.

[0039] Fig. 7 shows the reconstructed three-dimensional (3D) convex hulls of the 3 UAVs (e.g., UAV1, UAV2, UAV3) used in experiments at 3 different frame times to visualize the impact of frame time on shape recognition.

[0040] Fig. 8A shows an experimental setup used to scan the drones in two different outdoor scenarios.

[0041] Fig. 8B shows 4 neural network (NN) classifiers’ generalized performance for different frames.

[0042] Fig. 8C shows the 4 NN classifiers’ distance -based performance at 4 available frames.

[0043] Fig. 8D shows the NN classifier’s velocity -based performance at different frames.

[0044] Fig. 8E shows the NN classifier’s lighting condition-based performance at different frames.

[0045] Fig. 8F shows trajectories reconstructed for 2 different UAVs at different ranges and speed scenarios.

[0046] Fig. 8G shows the experimental device’s extracted trajectory vs. the GPS data of UAV1.

[0047] Fig. 8H shows a spatial resolution test with 3 static UAV s using their alphashapes.

[0048] Fig. 81 shows the point cloud of UAV1 with a runaway propeller, and the reconstructed trajectory of both objects.

[0049] Fig. 8 J shows an experimental setup using Livox Tele- 15 to scan the drone swarm at a first drone show (referred to as Droneshow I).

[0050] Fig. 8K shows the red-green-blue (RGB) images and corresponding point cloud representations (both time-synchronized) of 2 different swarm formations from the first drone show Droneshow 1) and the second drone show (Droneshow2), respectively.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0051] Fig. 8L exhibits a painting on a cylindrical surface from Droneshow2 and presents the other aspect of 3D visualization of the swarm at a convenient viewing plane.

[0052] Fig. 8M shows the trajectory visualization of 6 such drones, which are part of Droneshowl’s star formation.

[0053] Fig. 8N shows a mixture of the drone swarm and bird swarm.

[0054] Fig. 80 shows the accuracy plots and confusion matrix of the experiment instance.

[0055] Fig. 8P shows a drone swarm scalability test.

[0056] Fig. 8Q shows an experimental device with multiple LIDAR and thermal measurement an example implementation of the system of Fig. IB.DETAILED DESCRIPTION

[0057] Some references, which may include various patents, patent applications, and publications, are cited in a reference list and discussed in the disclosure provided herein. The citation and / or discussion of such references is provided merely to clarify the description of the disclosed technology and is not an admission that any such reference is “prior art” to any aspects of the disclosed technology described herein. In terms of notation, “[n]” corresponds to the nth reference in the list. For example, [1] refers to the first reference in the list. All references cited and discussed in this specification are incorporated herein by reference in their entireties and to the same extent as if each reference was individually incorporated by¬ reference.

[0058] Disclosed are components that can be used to perform the disclosed methods and systems. These and other components are disclosed herein, and it is understood that when combinations, subsets, interactions, groups, etc. of these components are disclosed that while specific reference of each various individual and collective combinations and permutation of these may not be explicitly disclosed, each is specifically contemplated and described herein, for all methods and systems. This applies to all aspects of this application, including, but not limited to, steps in disclosed methods. Thus, if there are a variety of additional steps that can be performed, it is understood that each of these additional steps can be performed with any specific embodiment or combination of embodiments of the disclosed methods.

[0059] While traditionally used by UAVs for ground mapping and object detection, LiDAR is employed in a stationary or mobile platform to monitor the sky and track flying objects like drones and birds. Li-DAR systems may be based on mechanical, solid-state, flash, and FMCW. Mechanical LiDAR offers high resolution and range but has wear-prone moving parts

[0063] . Flash LiDAR provides fast data acquisition but has limited range and resolution [54, 79 J.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0060] FMCW LiDAR can provide high precision and interference immunity, but is expensive and complex [70, 84]. Solid-state LiDAR is robust and potentially cheaper, but with a shorter range and lower resolution [47, 78], An example system with a solid-state LiDAR is provided herein that is practical, scalable, and low-cost LiDAR-based swarm sensing, though other LiDAR systems described or referenced herein may be used in the various embodiments.

[0061] Example System

[0062] Figs. 1A - IB each shows an example system 100 (shown as 100a and 100b) for detecting and tracking a drone or a drone swarm, in accordance with an illustrative embodiment. In Fig. 1 A, the exemplary system 100 includes (i) a controller 102 having a data preprocessing operation 104, a drone detector 106, and a drone tracker 108, and (ii) a LiDAR sensor 110. In Fig. IB, the exemplary system 100b includes multiple sensors and sensor types, including (i) one or more LiDAR sensors 110a - lOOn, and a second type of sensor 111 (e.g., thermal sensor). In some embodiments, the exemplary system 100 is a stationary platform. In some embodiments, the exemplary system 100 is a mobile platform. Fig. 8Q shows an experimental device with multiple LIDAR and thermal measurement an example implementation of the system of Fig. IB.

[0063] Data Preprocessing (104). In the example shown in Fig. 1 A, the data preprocessing module 104 is configured to receive, from the LiDAR sensor 110, a LiDAR measurement 112 (e.g., a stream of data points). The data preprocessing module 104 is then configured to (i) convert the LiDAR measurement 112 into a predefined format, (ii) denoise the LiDAR measurement 112 based on reflectivity and spatial location, (iii) filter the LiDAR measurement 112 (e.g., using the RANSAC algorithm), and (iv) output a preprocessed sensor measurement 114.

[0064] Drone Detector (106). The drone detector 106 includes a detection module 116 (also referred to as a detection phase), a merge module 120 (also referred to as a merge phase), and a frame optimizer and classifier 124 (also referred to as a frame optimization and classification phase). In some embodiments, the drone detector 106 is implemented using the algorithm in Fig. 5.

[0065] The detection module 116 is configured to (i) receive the preprocessed sensor measurement 114, and (ii) generate, from the preprocessed sensor measurement 114, a point cloud data object (also referred to as point cloud) (e.g., single-frame). The detection module 116 is then configured to (1) generate, via a computer vision operation, a cluster of the point cloud data object 118 by aggregating densely packed points and identifying points in low-Attorney Docket No. 11555-017WO1UMA No. 25-006 density regions as noise or outliers, and (ii) generate, via a triangulation operation, a shape object (e.g., alphashape, Delaunay triangle, or other mesh objects) of the cluster 118.

[0066] The merge module 120 is configured to (i) receive, from the detection module 116, the cluster of the point cloud data object 118, and (ii) concatenate / merge, via a sliding window method, the shape object of the cluster 118, generated at different time scans, into a merged shape object 122, also referred to as merged frame (e.g., with 3,, 10,..., etc., smaller frames). In some embodiments, the merged shape object 122 is labeled and stored in a multi-frame temporal storage to subsequently train the frame optimizer and classifier 124. The use and merging of multiple time scans improve the detection of drones and removal of outliers (nondrones) while the multiple time scans also provide for tracking (e.g., trajectory of drones and changes in trajectory).

[0067] The frame optimizer and classifier 124 is configured to (i) receive, from the merge module 120, the merged shape object 122, and (ii) classify, via a trained artificial intelligence (Al) classifier 125 (e.g., a convolutional neural network, etc.), drones (e.g., 126) in the point cloud data object using the merged shape object 122, and exclude non-drone objects. The frame optimizer and classifier 124 is then configured to send the classified outputs 126 to the drone tracker 108 for subsequent detecting and monitoring a drone or a drone swarm.

[0068] In some embodiments, the classified outputs 126 are used for the detection of a drone swarm in proximity to a predefined location, to generate an alert. In some embodiments, the classified outputs 126 are used for the detection of a drone swarm out of formation of a predefined swarm formation (e.g., in a drone show), to generate an alert or control the drone swarm.

[0069] Drone Tracker (108). The drone tracker 108, operatively coupled to the frame optimizer and classifier 124, is configured to track individual drones within the drone swarm. In some embodiments, the drone tracker 108 is implemented using the algorithm in Fig. 6.

[0070] Specifically, the drone tracker 108 is configured to merge the classified outputs (e.g., 126), and frames thereof, to generate a time sequence point of the classified outputs that resembles a time -lapse of the flight path of the drone swarm. The drone tracker 108 is then configured to (i) interpolate the time sequence of the classified outputs, and (ii) reconstruct a flightpath 128 (e.g., trajectory ) of each classified output in the interpolated time sequence of the classified outputs. In some embodiments, the drone tracker 108 is configured to plot the time sequence of the classified outputs and a flightpath (e.g., trajectory) of individual drones within the drone swarm.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0071] Multi-Sensor Configuration. In the example shown in Fig. IB, the data preprocessing module 104 is configured to concurrently receive (i) one or more LiDAR sensor measurements 112, from one or more LiDAR sensors 110a - 110n, and (ii) a second-type sensor measurement 113, from a second-type sensor 113 (e.g., thermal sensor). The data preprocessing module 104 is then configured to time synchronize the LiDAR sensor measurements 112 and the second-type sensor measurement 113, before converting the time-synchronized sensor measurements into a predefined format, as described in Fig. 1A. Other configurations of the system 100b are the same as the system 100a. An example of the synchronization is precision time protocol (PTP). Precision Time Protocol (PTP) is a network protocol used for synchronizing clocks throughout a computer network with high precision, often achieving accuracy in the sub-microsecond range.

[0072] Trained Al Classifier (125). In some embodiments, the trained Al classifier 125 is a binary classifier trained using the shape object of the cluster of LiDAR scans.

[0073] Example Method

[0074] Fig. 2 shows an example operation flow 200 for the exemplary system, in accordance with an illustrative embodiment. The method 200 includes receiving (202) a LiDAR measurement (e.g., 112, Figs. 1A - IB) of a LiDAR scan (e.g., 110, Figs. 1A - IB). The method 200 includes generating (204) a point cloud data object for the LiDAR measurement. The method 200 includes generating (206), via a computer vision operation, a cluster of the point cloud data object (e.g., 118, Figs. 1A - IB). The method 200 includes merging (208) the cluster, or an intermediate object generated therefrom, at different time scan. In some embodiments, the intermediate object is generated, via a triangulation operation, as a shape object (e.g., alphashape, Delaunay triangle, or other mesh objects) of the cluster (e.g., 118, Figs. 1 A - IB). The method 200 includes classifying (210), via a trained Al classifier (e.g., 125, Figs. 1A - IB), drones (e.g., 126, Figs. 1A - IB) in the point cloud data object and excluding non-drone objects. In some embodiments, the classified outputs are used for detecting and monitoring a drone swarm. The method 200 includes tracking (212) the drones within the drone swarm.

[0075] In some embodiments, the trained Al classifier (e.g., 125, Figs. 1 A - IB) is a binary classifier trained using the shape object of the cluster (e.g., 118, Figs. 1A - IB) of LiDAR scans.

[0076] In some embodiments, the tracking (212) of the drones within the drone swarm includes (i) merging the classified outputs (e.g., 126, Figs. 1 A - IB) at different time scans to generate a time sequence of classified outputs, (ii) interpolating the time sequence of classifiedAttorney Docket No. 11555-017WO1UMA No. 25-006 outputs, and (iii) reconstructing trajectories (e.g., 128, Figs. 1A - IB) of each classified output in the interpolated time sequence of classified outputs.

[0077] Example Drone Swarm Detection and Tracking System

[0078] Fig. 3A shows an example LiDAR system for drone swarm detection and tracking in accordance with an illustrative embodiment.

[0079] Fig. 3B shows Point clouds of UAV1, UAV2, UAV3, and Mallard bird (Anas platyrhynchos) at 3 different frame times.

[0080] Fig. 3C shows example point clouds of UAV1 and 4 different birds with varying poses at different ranges with a 500ms frame time.

[0081] Fig. 4 shows an example drone or drone swarm detection and tracking system (referred to as the LiSWARM system). In Fig. 4, the exemplary system includes a LiDAR sensor-based hardware setup (e.g., 110, Figs. 1A - IB) for point cloud data collection and a combination of computer vision-based algorithms for spatial clustering, model fitting with outlier data, manipulation of three-dimensional (3D) point clouds, and deep learning-based classifiers (e.g., Figs. 5 - 6). Fig. 5 shows an example algorithmic implementation of a detection and classification operation in the exemplary system. Fig. 6 shows an example algorithmic implementation of a tracking operation in the exemplary system.

[0082] Detection Algorithm. In Fig. 5, the filtered data points are first fed into the detection phase. The DBSCAN (Density-Based Spatial Clustering of Applications with Noise)

[0022] algorithm may be used as a means of spatial clustering to detect individual arbitrary objects, since this is a case of unsupervised learning from unlabeled data. Another clustering algorithm that aggregates densely packed points and identifies points in low-density regions as noise or outliers may be used as the algorithm that can detect clusters of irregular shapes in noisy datasets. The clustering step may break down (determine) a single frame point cloud into a number of cluster point clouds (e.g., 118, Figs. 1A - IB) present in that specific input point cloud frame. Because the cluster point clouds may be of arbitrary shape and point density due to the pose, distance, motion, or ambient light during data collection, the exemplary system may employ a signal processing operation on each detected object cluster (e.g., 118, Figs. 1A - IB) to create an alphashape from each cluster.

[0083] The exemplary system may facilitate the temporal feature of reflected point reception by the LiDAR system, which mimics the “dark adaptation” of the human eye by accumulating light over time. A shape object of a point cloud data object, at different frames, may be concatenated into sequences of 3, 5, and 10 frames (300ms, 500ms, and Is) using a sliding window technique.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0084] The point cloud data of all the detected object clusters (e.g., 118, Figs. 1A - IB) may be evaluated for different frame times, e.g., 100ms, 300ms, 500ms, and Is.

[0085] The recognition operation may employ frame optimization and classification. In the frame optimizer operation (e.g., 124, Figs. 1A - IB), the specific point cloud frame time out may be selected from the available frame times for each cluster based on preset conditions, resulting, e.g., from the range, velocity, and lighting. The exemplary system may check factors including i) the distance of the cluster from the sensor, ii) the number of points in the cluster, and iii) the volume of the cluster (using the actual volume range for the used drones and observed birds) for potential motion blur due to the object’s high speed. The conditions may be set after investigating the impact of all the factors.

[0086] The exemplary system may identify which duration cluster point cloud frame is fed to the next section for each individual case. A binary classifier (e.g., 125, Figs. 1 A - IB) may be implemented using the labeled dataset, e.g., a deep neural network classifier model. One example classifier is the PointNet

[0014] because of its lightweight nature, simplicity, and efficiency in handling unordered 3D point cloud data. Other classifiers, e.g., kNN, SVM, and Random Forest, may be used to compare the classification performance.

[0087] Example Adaptive Sensing

[0088] The exemplary system is configured to perform signal processing tasks on the point cloud data used in the detection algorithm. In the detection step (e.g., 106, Figs. 1 A - IB), the exemplary system may use the shape object (e.g., alphashape, Delaunay triangulation, convex hull, etc.) of each cluster as an additional descriptor to be used for each object cluster. The generation of the geometric feature may also facilitate an operation where points are sparse. Alphashape is a concept from computational geometry and is a generalization of convex hulls used in object recognition. Alphashape and other geometric transforms (e.g., Convex hull and the like) may be employed to reconstruct the shapes of the object from the point cloud clusters (e.g., 118, Figs. 1A - 1B).

[0089] Fig. 7 shows the reconstructed 3D convex hulls of the 3 UAVs used in the experiments at 3 different frame times to visualize the impact of frame time in shape recognition.

[0090] In the frame merger operation, the exemplary system may employ a sliding window approach to concatenate consecutive numbers of frames to merge, e.g., 3, 5, or 10 frames for the dataset. Concatenation can also reduce the data size by a corresponding degree (e.g., 3, 5 & 10 times) for the merged frames. The operation can beneficially maintain the same data size for each frame time since larger variations in data size can impact the classifier performanceAttorney Docket No. 11555-017WO1UMA No. 25-006 by influencing training time, generalization ability, feature importance discernment, and computational efficiency.

[0091] Example Tracking Algorithm

[0092] Fig. 6 provides a summary of an example tracking operation. Once points associated with drones are recognized (e.g., per Fig. 5), they may be passed over to the next block in the pipeline: the tracking algorithm. In Fig. 6, the operation may be performed in 3 parts: (i) frame merging for only drone point clouds to meet the required duration of drone flight, (ii) trajectory reconstruction for that flight duration for missing points in the point cloud, and (iii) trajectory visualization.

[0093] The drone frame merging operation may use the duration as input to merge consecutive drone point cloud frames to be concatenated to produce a point cloud resembling a time-lapse of the drone flight path. Afterward, the lime-lapse drone point clouds may then be used to reconstruct the trajectory of the drone (e.g., 128, Figs. 1A - IB). Since the point cloud is the discrete 3D representation of any object, discontinuities in 3D space along the path of the drone may result due to drone motion, pose, distance, or an anomaly in reflected point reception of the LiDAR itself, among others. To reduce or eliminate spatial discontinuities, linear interpolation or other interpolators may be used to predict and fill in points in the blank space due to discontinuities. Once the points are distributed continuously across the drone’s flight path, the trajectory may be reconstructed. In the study (later described herein), the trajectory (e.g., 128, Figs. 1A - IB) may be viewed with continuous lines. The output from this tracking algorithm block is drone trajectory plots in 3D space. These trajectories are compared to the GPS flight data of a drone to calculate the trajectory accuracy of a drone when tracked via LiDAR.

[0094] Example Artificial Intelligence (Al) and Machine Learning (ML) Classifiers

[0095] Machine Learning. In addition to the machine learning features described above, the exemplary system can be implemented using one or more artificial intelligence and machine learning operations. The term “artificial intelligence” can include any technique that enables one or more computing devices or computing systems (i.e., a machine) to mimic human intelligence. Artificial intelligence (Al) includes but is not limited to knowledge bases, machine learning, representation learning, and deep learning. The term “machine learning” is defined herein to be a subset of Al that enables a machine to acquire knowledge by extracting patterns from raw data. Machine learning techniques include, but are not limited to, logistic regression, support vector machines (SVMs), decision trees, Naive Bayes classifiers, and artificial neural networks. The term “representation learning” is defined herein to be a subset of machineAttorney Docket No. 11555-017WO1UMA No. 25-006 learning that enables a machine to automatically discover representations needed for feature detection, prediction, or classification from raw data. Representation learning techniques include, but are not limited to, autoencoders and embeddings. The term “deep learning” is defined herein to be a subset of machine learning that enables a machine to automatically discover representations needed for feature detection, prediction, classification, etc., using layers of processing. Deep learning techniques include, but are not limited to, artificial neural networks or multilayer perceptron (MLP).

[0096] Machine learning models include supervised, semi-supervised, and unsupervised learning models. In a supervised learning model, the model learns a function that maps an input (also known as a feature or features) to an output (also known as a target) during training with a labeled data set (or dataset). In an unsupervised learning model, the algorithm discovers patterns among data. In a semi-supervised model, the model learns a function that maps an input (also known as a feature or features) to an output (also known as a target) during training with both labeled and unlabeled data.

[0097] Neural Networks. An artificial neural network (ANN) is a computing system including a plurality of interconnected neurons (e.g., also referred to as “nodes”). This disclosure contemplates that the nodes can be implemented using a computing device (e.g., a processing unit and memory as described herein). The nodes can be arranged in a plurality of layers, such as an input layer, an output layer, and optionally one or more hidden layers with different activation functions. An ANN having hidden layers can be referred to as a deep neural network or multilayer perceptron (MLP). Each node is connected to one or more other nodes in the ANN. For example, each layer is made of a plurality of nodes, where each node is connected to all nodes in the previous layer. The nodes in a given layer are not interconnected with one another, i.e., the nodes in a given layer function independently of one another. As used herein, nodes in the input layer receive data from outside of the ANN, nodes in the hidden layer(s) modify the data between the input and output layers, and nodes in the output layer provide the results. Each node is configured to receive an input, implement an activation function (e.g., binary step, linear, sigmoid, tanli, or rectified linear unit (ReLU), and provide an output in accordance with the activation function. Additionally, each node is associated with a respective weight. ANNs are trained with a dataset to maximize or minimize an objective function. In some implementations, the objective function is a cost function, which is a measure of the ANN’S performance (e.g., error such as LI or L2. loss) during training, and the training algorithm tunes the node weights and / or bias to minimize the cost function. This disclosure contemplates that any algorithm that finds the maximum or minimum of the objective functionAttorney Docket No. 11555-017WO1UMA No. 25-006 can be used for training the ANN. Training algorithms for ANNs include, but are not limited to, backpropagation. It should be understood that an ANN is provided only as an example machine learning model. This disclosure contemplates that the machine learning model can be any supervised learning model, semi-supervised learning model, or unsupervised learning model. Optionally, the machine learning model is a deep learning model. Maclaine learning models are known in the art and are therefore not described in further detail herein.

[0098] A convolutional neural network (CNN) is a type of deep neural network that has been applied, for example, to image analysis applications. Unlike traditional neural networks, each layer in a CNN has a plurality of nodes arranged in three dimensions (width, height, depth). CNNs can include different types of layers, e.g., convolutional, pooling, and fully- connected (also referred to herein as “dense”) layers. A convolutional layer includes a set of filters and performs the bulk of the computations. A pooling layer is optionally inserted between convolutional layers to reduce the computational power and / or control overfitting (e.g., by downsampling). A fully-connected layer includes neurons, where each neuron is connected to all of the neurons in the previous layer. The layers are stacked similarly to traditional neural networks. GCNNs are CNNs that have been adapted to work on structured datasets such as graphs.

[0099] Other Supervised Learning Models. A logistic regression (LR) classifier is a supervised classification model that uses the logistic function to predict the probability of a target, which can be used for classification. LR classifiers are trained with a data set (also referred to herein as a “dataset”) to maximize or minimize an objective function, for example, a measure of the LR classifier’s performance (e.g., an error such as LI or L2 loss), during training. This disclosure contemplates that any algorithm that finds the minimum of the cost function can be used. LR classifiers are known in the art and are therefore not described in further detail herein.

[0100] A Naive Bayes (NB) classifier is a supervised classification model that is based on Bayes’ Theorem, which assumes independence among features (i.e., the presence of one feature in a class is unrelated to the presence of any other features). NB classifiers are trained with a data set by computing the conditional probability distribution of each feature given a label and applying Bayes’ Theorem to compute the conditional probability distribution of a label given an observation. NB classifiers are known in the ait and are therefore not described in further detail herein.

[0101] A k-NN classifier is an unsupervised classification model that classifies new data points based on similarity measures (e.g., distance functions). The k-NN classifiers are trainedAttorney Docket No. 11555-017WO1UMA No. 25-006 with a data set (also referred to herein as a “dataset”) to maximize or minimize a measure of the k-NN classifier’s performance during training. This disclosure contemplates any algorithm that finds the maximum or minimum. The k-NN classifiers are known in the art and are therefore not described in further detail herein.

[0102] Other classifiers may be used, e.g., the Random Forest classifier and the Genetic Algorithm classifier, among others described or referenced herein.

[0103] Experimental Results and Additional Examples

[0104] A study was conducted to develop and evaluate an experimental drone swarm management system (also referred to as LiSWARM) using low-cost solid-state LiDAR and other LiDAR. The experimental system employed the processing of point cloud data and adaptively incorporated temporal frames to enable the accurate detection and real-time tracking of hundreds of drones, distinguishing them from other flying objects. The study evaluated the practical evaluations with two drone lightshows with 150 and 500 drones at distances of hundreds of meters. The study performed a scalability analysis for a large swarm of up to 10,000 drones.

[0105] IMPLEMENTATION. The experimental system employed a data acquisition block that can provide LiDAR point cloud data via a wired connection between the LiDAR sensor and the host computer. The point cloud data was collected similarly to a video stream. Since a LiDAR can scan only within its field of view (FOV), careful positioning of LIDAR was performed to keep the target within FOV. The experimental system converted the raw point cloud stream (. Ivx) into point cloud snapshots or 100ms unit point cloud frames (.las or.csv). Because the raw stream is large and memory-intensive, containing detailed 3D representations of various objects, the study utilized the temporal features of the point cloud by combining unit frames to analyze object motion and range effectively. After converting to single frame point clouds, the study denoised them by removing outlier points based on reflectivity and spatial location. Next, the study removed the ground from the point cloud frames using the RANSAC algorithm, which iteratively fits a plane to the ground points and removes the inliers, eliminating unnecessary ground points.

[0106] Then, the experimental system filtered out points from trees, buildings, and other infrastructure considered clutter. The experimental system used another filter to keep only flying objects by maintaining a Z-axis threshold, but this is skipped for special scenarios like static drones on the ground, resting birds, or swimming birds. The filtered point cloud frames were passed into a detection algorithm block with three sub-blocks: detection, frame merger, and recognition. Once the drones were recognized with sufficient accuracy, the associated pointAttorney Docket No. 11555-017WO1UMA No. 25-006 cloud frames were fed into a tracking algorithm block to reconstruct and visualize their trajectories. During detection, the experimental system created a 3D LiDAR-based point cloud dataset of drones and birds under various conditions.

[0107] The study used an off-the-shelf LiDAR device (Livox Tele- 15), a computer (Alienware M17 R3 P45E Gaming Laptop), and a network layer device (Netgear Nighthawk AX6 Router) for the connection between the LiDAR sensor and the computer. The software components of the system included software packages: Matlab R2023b, Jupiter Notebook! Anaconda Distribution), Livox Viewer 0.11.0 (64bit)-Windows. Specific algorithms used included RANSAC (Random Sample Consensus) for ground removal, DBSCAN (Density-Based Spatial Clustering of Applications with Noise) for spatial clustering, and classifier models such as machine learning-based kNN, SVM, Random Forest & deep learning¬ based PointNet.

[0108] PERFORMANCE IN SPARSE DRONE SCENARIO

[0109] Experimental Setup. The study employed three quadcopter drones, UAV1, UAV2, and UAV3, as different airborne objects for detection and tracking. Fig. 8A shows the experimental setup used to scan the drones in two different outdoor scenarios.

[0110] In Fig. 8A, the experimental setup is shown for Livox Tele-15 to scan 2 drones and birds at a university campus (subpanel (a)) and a lake (subpanel (b)).

[0111] The study also used the LiDAR sensor to scan birds resting, swimming, and flying while maintaining all safety measures for the wildlife. The study focused on the LiDAR sensor so that birds or other natural objects can be captured within the FOV of the sensor, and in the case of drones, the study maintained all the drone flights that are covered by the LiDAR sensor. To maintain variability in collected data regarding lighting condition, speed, and range, drone scan experiments were performed at different outdoor locations during noon, afternoon, and evening, and the drone flight paths were associated with a wide range of distances covered at different speeds. The study scanned and stored point frames continuously in point cloud streams for further analysis in the next steps. In the study, each single point cloud frame is 100ms in duration.

[0112] Evaluation Metrics. The study evaluated the individual LiDAR-based subsystems with respect to the detection, recognition, and classification of drones from other flying objects, mainly birds, and then tracked the specific drone by reconstruction and analysis of its trajectory. The experimental system analyzed the existence of a drone and its flight path based on scanned point clouds by the LiDAR, and the parameters that impact the performance of any LiDAR device were target distance from the sensor, speed of the target object, lightingAttorney Docket No. 11555-017WO1UMA No. 25-006 condition during data collection, proximity or spatial separation between two or more drones to identify them uniquely. The analysis covered such parameters during system performance evaluation. Recognition and classification performance for individual drones with respect to such parameters are presented with accuracy plots and confusion matrices while compared with ground truth, as the study used labeled data in the classification. Tracking performance was evaluated in terms of trajectory analy sis reconstructed from point clouds, and an error accuracy¬ plot was determined in comparison to the drone’s GPS-based flight path.

[0113] Detection. Detection of an object included identifying and locating the object within images, videos, or point clouds in the experimental system. The first task was to identify flying objects, including drones, birds, or any other flying object within the LIDAR range and coverage area known as the FOV. The study performed the detection task of drones by flying them from 5m to 300m at varied speeds and at different ambient light conditions. The study was able to detect UAV1 up to 220m, UAV2 up to 180m, and UAV3 up to 220m. And in the case of scanning birds, the study was able to detect different birds up to 180m. Using the data preprocessing and detection algorithm described earlier, by this point the study created segmented point clouds of flying objects, including drones, birds, etc. in single frames of 100ms and then into 3, 5, and 10 frames (higher frame time was not created since 1000ms provides sufficient point density and severity of motion blur grows at longer durations) point clouds using a sliding window approach which ultimately was used to create a comprehensive point cloud dataset of equal data length.

[0114] Classification. Once the flying objects, including drones, were detected, the next task was to recognize drones among them. The study classified the labeled input dataset comprising the point clouds of flying objects clustered into 2 classes: drones and other flying objects. The study used kNN, SVM, Random Forest, and PointNet classifiers in binary classification tasks. This classification task is performed with respect to 3 distinct parameters: target range, target velocity, and lighting condition, where the classifier models are trained and tested with datasets from diverse target ranges, velocities, and lighting conditions. The study-evaluated the generalized classifier performance where the dataset has no dependency on the 3 above-mentioned parameters. Fig. 8B shows all 4 classifiers’ generalized performance for different frames.

[0115] Fig. 8B shows the classification performance. From the test accuracy plots, it is evident that the PointNet-based NN classifier exhibits the best performance with an accuracy of 98% for 10 frames and 91% for single frames.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0116] Different Ranges. The study trained and tested the 3 ML and the NN classifier with datasets segregated into short distance (<50m), medium distance (51-100m), and long-distance (>100m). Fig. 8C shows the 4 classifiers’ distance-based performance at all 4 available frames.

[0117] Fig. 8C shows test accuracy for different range targets. From the test accuracy plots of all 4 classifiers at different frame times of Fig. 8C, it can be observed that the NN-based classifier has the highest accuracy barring for the 5 -frame case at a long distance. The highest accuracies obtained at different scenarios for the NN classifier are 98%’ for 5 frames and 10 frames at short distances, and 98% for 10 frames at both medium and long distances. One trend is evident from these test results: accuracy decreases with an increase in target distance from the sensor, which results from the fact that LiDAR receives fewer reflected points as the target moves further, which was investigated in [1]. The phenomenon appears to impede classification accuracy at longer distances.

[0118] The study proceeded to mimic the human eye’ s “dark adaptation” technique, which accumulates light over time when it experiences sudden darkness. In the methodology, the study concatenated multiple frames to accumulate more reflected points from a target. The approach proved to work efficiently based on accuracy analysis and evaluated plots and showed that accuracy increased as the number of frames merged together increased.

[0119] Different Target Speeds. Since the NN classifier appears to perform in general over most distances, the study selected the NN classifier for further performance evaluations. The study trained and tested the NN classifier with a dataset segregated into static targets at the ground and hovering targets in the sky that are either slow (0-5m / s), medium (5-10m / s), or fast (10-20m / s). Here, the dataset is distance-independent in that all speed-based categories contain point clouds from different distances. Fig. 8D shows the NN classifier’s velocity-based performance at different frames.

[0120] Fig. 8D shows test accuracy for different target velocities. In Fig. 8D, the highest accuracies at varied categories obtained are 98% for static targets, 96% for hovering targets, and 90%’ for three cases of motion. The accuracy plots follow the trend in which accuracy decreases with the increase in target speed across all frames, and the trend supports the intuition that a target with higher velocity will incorporate a greater amount of motion blur. And contrary to expectations, the performance analysis at different ranges shows, in this case, that accuracy is better for fewer frames when the target is in motion.

[0121] Different Light Conditions - Day / Nighttime. The NN classifier had been trained and tested with datasets segregated into daytime and nighttime data, where objects were scannedAttorney Docket No. 11555-017WO1UMA No. 25-006 during daylight and at night. Fig. 8E shows the NN classifier’s lighting condition-based performance at different frames.

[0122] Fig. 8E shows test accuracy for day and night-time data. The highest accuracies during daytime and nighttime were 89% and 88% respectively. The accuracies appear to decrease slightly compared to distance-based and speed-based results. The accuracy plots appear to establish that the classifier performance is not significantly impacted by the presence of ambient light.

[0123] Trajectory Reconstruction and Analysis. The study reconstructed the path a drone in the experiment followed over time during its flight from the stream of point cloud data scanned. To address discontinuities that may appear along the drone path in the time-lapse point cloud, the study reconstructed a continuous trajectory or flight path from the point cloud, which needs to be done at different velocities and ranges for the drone to test the consistency and efficiency of this process. The study verified the trajectory reconstruction process at various ranges and speed scenarios and also verified the trajectory estimation accuracy by comparing it with the GPS flight data of the drone itself.

[0124] Trajectory Reconstruction at Different Ranges and Speeds. The study conducted experiments with 2 UAVS at 2 different ranges and speed scenarios and analyzed their trajectories for different durations. Fig. 8F shows the 2 specific cases for the drones.

[0125] Fig. 8F shows trajectories reconstructed for 2 different UAVs at different ranges and speed scenarios. In Fig. 8F, subpanel (a) shows point clouds of UAV1 and UAV2 in motion, 14.75m / s up to 120m and 6m / s up to 26m, respectively. In Fig. 8F, subpanel (b) shows reconstructed trajectories of UAV1 and UAV2. The study used a UAV1 trajectory with 33s duration at a speed of 14.75m / s from a distance of 20 meters to 120 meters, and a UAV2 trajectory with a 21 -second duration at a speed of 6m / s from a distance of 5 meters to 26 meters. The study created the point clouds capturing the flight paths of drones by concatenating consecutive frames to match the required duration. The trajectories of those drones were then extracted using linear interpolation to handle the spatial discontinuities among frames.

[0126] Trajectory Accuracy Analysis from GPS Flight Path and LiDAR Trajectory Comparison. The study reconstructed the drone trajectory from the point cloud data using LiDAR scans. To verify the accuracy of the drone trajectory from LiDAR, the study used an external sensor and compared the measurements with the ground truth. The study considered the ground truth to be the GPS-based location data, which was extracted as a drone flight path obtained from the drone itself and then compared the deviation in 3D space by comparing the GPS flight path and LiDAR-estimated trajectory.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0127] Fig. 8G shows the 4 main ingredients of this accuracy analysis: original point cloud, the reconstructed trajectory from it, the GPS flight path, and the deviation of points obtained from 2 modalities. The result shows that both trajectories are overlaid fairly close to each other.

[0128] Fig. 8G shows the experimental system’s extracted trajectory vs. the GPS data of UAV1. In Fig. 8G, subpanel (a) shows the LiDAR point cloud, subpanel (b) shows the GPS flight path, subpanel (c) shows the LiDAR reconstructed trajectory and the GPS flight path overlaid, and subpanel (d) shows the error plot of LiDAR points corresponding to the GPS points.

[0129] Sensitivity Analysis. The study evaluated sensitivity analysis for granularity and collision avoidance.

[0130] Granularity. The study evaluated the granularity of LiDAR point cloud data by analyzing the spatial resolution that can be achieved. The study determined the proximity of 2 objects in space, both static & in motion, and still be identified as distinct objects. While tested with 3 static drones at close proximity, the study found the distance to be 30 cm, and tliis is shown in Fig. 8H using both point cloud data and its alphashape representations. Fig. 8H shows a spatial resolution test with 3 static UAVs using their alphashapes. In Fig. 8H, subpanel (a) shows point cloud of 3 UAVs at close proximity, and subpanel (b ) shows the alphashapes of the 3 UAVs.

[0131] While conducting such tests with drones in flight, the study received “Obstacle Detection” notifications in the drone’s original flight log whenever flying UAV2 and UAV3 at such close proximity that it triggered an obstacle detection warning, which occurred at 300 cm or 3 m. The point clouds of the 2 drones were resolved as distinct drones.

[0132] Collision Avoidance. The study used LiDAR point clouds for collision avoidance and to detect faulty drones. During the regular drone flight test, the study encountered an irregular incident when one of the propellers of UAV1 broke off, and the drone fell down. Analysis of the point cloud stream and the reconstructed trajectory of both the fallen drone and the detached propeller led to their retrieval. Fig. 81 shows the incident in terms of the point cloud and the extracted trajectory from it. The data illustrates that LiDAR may be used to detect such anomalous cases to prevent faulty drones from creating collisions or damage. In Fig. 81, subpanel (a) shows the point cloud of UAV 1 with a runaway propeller, and subpanel (b) shows the reconstructed trajectory of both objects.

[0133] PERFORMANCE IN THE DRONE SWARM SCENARIO

[0134] Setup & Evaluation Metrics. The study used the same Li vox Tele- 15 LiDAR device as the sensor and the hardware setup; however, in tliis experiment, the targets were differentAttorney Docket No. 11555-017WO1UMA No. 25-006 drones used in the light shows. The study detected and tracked a swarm of drones together. The study specifically collected LiDAR point clouds of drone swarms from 2 drone show events: drone show with 150 drones on April 15, 2024 (“Droneshow1”) & drone show with 500 drones on May 25, 2024 (“Droneshow2”). Because all the drone shows took place at night and in a secure area, all the LiDAR scans of drone swarms were acquired during nighttime and from a certain distance outside the secure area.

[0135] Fig. 8J shows an experimental setup using Livox Tele-15 to scan the drone swarm at Droneshowl. The study focused on the 3D visualizations of the drone swarm in diverse 3D formations, visualization of individual trajectories of drones within the swarm, recognition of each drone within the swarm, recognition of drones from birds in scenarios where drone swarms are artificially intertwined or mixed together, and finally the scalability analysis which tests how well the experimental system scales with increasing numbers of drones in the swarm. In that regard, the study used several metrics to test the efficiency of the experimental system in analyzing drone swarms: recognition accuracy, data processing rate, latency of the experimental system, and how many drones or objects within the swarm the experimental system can handle in real-time before being overwhelmed.

[0136] 3D Visualization of Drone Swarms. The study shows that the 3D visualization of LiDAR point clouds from drone swarms can (i) detect unlit drones invisible within the swarm, and (ii) provide both location and depth information, allowing the scene to be viewed from any plane, revealing swarm formations or structures not fully perceived from one viewpoint.

[0137] Fig. 8K shows the RGB images and corresponding point cloud representations (both are time-synchronized) of 2 different swarm formations from Droneshowl and Droneshow2 on the left and right, respectively.

[0138] In Fig. 8K, subpanel (a) shows RGB image frames of drone swarm formation 1 and 2 with both lit and unlit drones, and areas of interest scanned by lidar are annotated. In Fig.8K, subpanel (a), the annotated boxes mark the field of view' (FOV) covered by the LiDAR. In Fig. 8K, subpanel (b), the annotated circles mark the individual drones that were unlit during that specific instance. If the ’star’ formation of Droneshowl between its visual RGB and LiDAR representation is compared, drones located at the lower part of the formation are unlit and invisible in the visual RGB image; however, those drones are detected and visualized in the point cloud. Similarly, in the ’sphere’ formation on the right, there are 4 drones in each column distributed across the sphere surface, which is visible in the LiDAR point cloud, but the last drone at each column is unlit and therefore not visible in the RGB frame.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0139] Fig. 8L exhibits a painting on a cylindrical surface from Droneshow2 and presents the other aspect of 3D visualization of the swarm at a convenient viewing plane.

[0140] From the visual point of view, looking up at the drone cloud, where the sensor was also placed, it is not possible to comprehend the full picture that the formation was trying to convey. However, by using LiDAR and its reflected 3D point cloud, the study was able to reconstruct the entire formation and then rotate it into the most convenient viewing plane to perceive the complete message of the scene.

[0141] Swarm Trajectory Visualization. Each drone swarm included many individual drones, and during the flight of each such swarm, all the individual drones maintained a complicated flight path to recreate diverse 3D shapes, letters, or scenes. Therefore, it seems challenging to reconstruct and visualize the trajectories of individual drones within a swarm. However, using the experimental system, which benefits from the superior spatial resolution, the study was able to reconstruct drone trajectories within the swarm. Fig. 8M shows the trajectory visualization of 6 such drones, which are part of the ’star’ formation of Droneshow I.

[0142] Here, the study first selected the drones of interest from which the study localized the point cloud frames from the complete point cloud stream, then the study concatenated the point cloud frames of localized drones over a duration (12s in this case) in which the study want to visualize their trajectories, and then plot and visualize the 6 individual reconstructed trajectories.

[0143] Recognition Accuracy. The study evaluated the accuracy of the LiDAR-based drone detection and tracking system. The study evaluated the recognition accuracy in distinguishing between drones and birds.

[0144] The study used real drone swarm data collected from Droneshow2 and then simulated bird swarm at a similar location to the drone swarm. The synthetic bird swarm was created from actual bird point clouds by replicating them in 3D space. Then, both the drone swarm and bird swarm were mixed together, as shown in Fig. 8N. The experimental system then performed recognition tasks and classified the objects in the mixed swarm either as drones or not. Then, the study evaluated the classification performance using ML classifiers (kNN & SVM) and PointNet-based NN classifier. Fig. 80 shows the accuracy plots and confusion matrix of the test instance.

[0145] In Fig. 80, the figure shows drone swarm recognition performance with simulated bird swarms. In Fig. 80, subpanel (a) shows recognition accuracies for NN and ML classifiers, and subpanel (b) shows confusion matrices for NN and ML classifiers (only kNN shown forAttorney Docket No. 11555-017WO1UMA No. 25-006 ML). The recognition performance shows the NN-based classifier performed better with 89% test accuracy.

[0146] Scalability. The study analyzed the scalability of the detection and tracking system with an increasing number of drones in the swarm with continuous inclusion of drones in the swarm. The study conducted tests on a computer with the following configuration: Intel(R) Core(TM) i7-10750H CPU © 2.60GHz, 6 cores, 12 logical processors, and 16GB RAM. The study performed this scalability analysis for processing time, including spatial clustering, data processing, and then the recognition task from the mixture of drone and bird swarms. The study also investigated the experimental system’s processing rate to obtain the delay of the system pipeline as a function of the number of drones processed per unit time.

[0147] Fig. 8P shows the total processing time at each major operational step as the drone swarm is scaled up in quantity. Specifically, Fig. 8P shows the drone swarm scalability test by replicating drones in the actual drone show. In Fig. 8P, subpanel (a) shows an actual drone swarm point cloud, and subpanel (b) shows a simulated drone swarm replicated 10 times.

[0148] For each number of drones, the vast majority of the processing time is taken up by the clustering and data processing times, while the inference time due to the classification is relatively small.

[0149] As expected, total computational time rises with the number of drones in a swarm, but if we look at the normalized rate of computational time per drone in Fig. 8P, subpanel b), we see that there appears to be a trend towards decreasing latency per drone as the swarm size increases. If 5 seconds is the threshold for near-instantaneous real-time tracking of a drone swarm, then the exemplary system can track at least 150 drones simultaneously, which should handle many cases for drone swarm tracking.

[0150] Fig. 8Q shows an experimental device with multiple LIDAR and thermal measurement an example implementation of the system of Fig. IB.

[0151] Additional experimental results are provided in

[0087] , which is incorporated by¬ reference herein in its entirety.

[0152] Discussion

[0153] Motivations. A robust and reliable 3D drone s 'arm would enable a wide range of transformative capabilities across various domains, including agriculture, delivery services, search-and-rescue, complex environmental monitoring, defense, entertainment, and others. For example, in agriculture, it allows for precision farming by efficiently applying fertilizers and monitoring crop health, boosting yield, and reducing waste

[0062] ,

[0074] , In logistics, it optimizes delivery routes and handles larger package volumes with enhanced reliability

[0044] , DuringAttorney Docket No. 11555-017WO1UMA No. 25-006 disaster response and search- and -rescue operations, a drone swarm can swiftly cover extensive areas, identify survivors, assess damage, and deliver supplies with high precision, which is critical for saving lives and resources [2, 26, 34], For wildlife tracking, it enables comprehensive monitoring of animal movements and behaviors, aiding conservation efforts and habitat management [40, 64, 65 j. In climate studies, drones can collect and analyze atmospheric data, monitor environmental changes, and track weather patterns, providing crucial insights into climate change impacts [8],

[0012] ,

[0032] ,

[0033] . In defense, it allows for strategic surveillance, reconnaissance, and coordinated maneuvers, enhancing situational awareness and mission effectiveness [27, 35, 36, 69]. Last but not least, 3D drone swarms are replacing traditional fireworks, creating similarly exciting visual displays without the associated environmental pollution and safety risks

[0066] . Although these concepts are exciting, managing real-time operations of the coordination of these 3D swarms is the main hurdle preventing these ideas from being fully realized.

[0154] Prior approaches and limitations. Detection and tracking of drones have been research directions with various approaches such as camera-based [19, 76], acoustic-based [10, 30], radar-based [11, 48], RF-based [56, 57], LiDARbased [29, 38], and multimodality-based systems of these sensing methods extensively studied in both academia and industry. However, none of the existing approaches have been confirmed to reliably detect and track 3D swarms at long distances.

[0155] In particular, camera-based systems leverage advanced computer vision techniques to identify and track drones visually, providing high-resolution data but facing challenges in low-visibility conditions and only performing well at short distances (around 50m). Acousticbased systems detect the unique sound signatures of drones, offering a passive detection method but struggling with ambient noise interference. Radar-based systems, widely used in military applications, provide robust detection capabilities over long distances but struggle both with detecting drones made from plastic materials, which are common in commercial and industrial drones today, as well as ground clutter exacerbated by low-flying drones. RF-based systems utilize the radio frequencies emitted by drones for detection and tracking, offering the potential for long-range monitoring but requiring the drones to actively transmit signals. Current LiDAR -based techniques do not differentiate between drones and other flying objects like birds, and none of the existing work can track a swarm of drones. One of the common LiDAR-based sensing systems is SLAM (Simultaneous Localization and Mapping)

[0021] , which creates a real-time map of the surroundings for autonomous systems indoors and outdoors. However, the performance of SLAM in drone swarm monitoring is unclear.Attorney Docket No. 11555-017WO1UMA No. 25-006 Multimodality-based systems combine these different methods to enhance detection accuracy and reliability, yet they face complexities in integration and high costs.

[0156] Exemplary approach. To fill the gaps, the study investigated the development of a LiDAR-based system, named LiSWARM, for real-time 3D drone detection and tracking, with the potential to scale up and simultaneously track thousands of drones. By analyzing the LiDAR signals reflected off the drone’s body, the exemplary system can differentiate drones from other flying objects and perform algorithms to adaptively and intelligently grasp the information in both the spatial and time domains to reliably track the drone’s trajectory in real¬ time.

[0157] The exemplary system provides a robust LiDAR tracking system for drone swarms that can differentiate drones and other flying objects, such as birds, which may have identical reflected signal intensity. The exemplary system can address LiDAR signals that are sensitive to the distance, speed, and behavior of the tracking objects, where the static configuration of the point cloud observation with a fixed frame may lead to high errors in detection and large gaps in real-time tracking. The exemplary system can monitor the drone trajectory of a swarm having drones that can cross paths with each other in 3D.

[0158] In the study, the method utilized LiDAR signals reflected off flying objects and detect and recognize drones that are at least demonstrated up to 22.0m. The study developed algorithms to analyze the drone’s point cloud, enabling the simultaneous tracking of a swarm of hundreds of drones with high temporal and spatial resolutions. The study developed an algorithm to track the trajectory of drones within the swarm and confirm that the exemplary system matches the GPS-based tracking method. The study performed a thorough analysis to validate system performance under varying ranges, speeds, and lighting conditions. The study conducted a scalability analysis with detailed benchmarking to evaluate the feasibility of monitoring up to 10,000 drones.

[0159] RELATEDWORKS. Studies on anti-drone techniques can be categorized into detecting, tracking, and identifying drones [6],

[0013] ,

[0049] . The critical issue in flying object detection systems is differentiating drones from other flying objects using ground-based static detectors or mobile detection systems

[0016] ,

[0083] . This involves evaluating existing technologies and identifying those that can effectively detect swarms of drones [3],

[0041] ,

[0072] ,

[0080] ,

[0160] RF analysis of drone communication signals is popular for detecting single or multiple drones due to its low complexity, ease of implementation, and all-weather operation [7], However, it struggles with precise positioning, only determining signal direction based on its angle of arrival

[0023] ,

[0052] ,

[0058] , Advanced RF technology with software-defined radioAttorney Docket No. 11555-017WO1UMA No. 25-006 platforms can detect small drones up to 75 meters away

[0017] ,

[0018] ,

[0055] . For dynamic detection, drones can chase others using radio targeting and a shared network within a collaborative system [20, 1, 77, 85],

[0161] The sound from drone motors and propellers, ranging from 20 Hz to 20 kHz, creates a distinct signature allowing acoustic sensors to differentiate drones using fiber-optic acoustic methods

[0024] , machine learning [4], and deep neural networks

[0075] . This method is effective only in quiet environments and short-range detection. Increasing acoustic sensors and using beam-forming techniques can locate drones with up to 80% success

[0028] ,

[0068] , Optical and thermal cameras use CNN-based algorithms for single and multi-object detection and tracking [5, 25, 45, 50], However, computer vision technologies struggle with adverse weather, lighting, background conditions, minimal detection range, and detecting small drones

[0031] ,

[0042] ,

[0071] , [86 J.

[0162] Radar can detect drones by simultaneously analyzing their Doppler signature and echo due to the rotor blades

[0043] ,

[0081] . The system transmits radio waves and detects the reflected signals to accurately determine the target’s distance and speed

[0059] , Radar position adjustments are often required for drone detection during runtime. Multi-agent reinforcement learning can adjust the radar parameters’ runtime and the radar platform’s position

[0037] , Unlike RF or acoustic sensing, radar’s detection coverage is unaffected by environmental conditions. However, this technology cannot distinguish drones from other flying objects and needs to be combined with other sensing technologies [9],

[0053] .

[0163] LiDAR offers precise detection, tracking, and mapping

[0067] , In one study, a LiDAR system with a peak power of 700 kW extended the detection range to 2 km

[0038] . Another experiment used a 3D LiDAR system on a vehicle to detect mini drones, showing high success for targets within 30 meters using sensors with a 100-meter range

[0015] ,

[0029] . The LiDAR SLAM (Simultaneous Localization and Mapping) method can detect moving objects, and point cloud analysis calculates the confidence level of the initial vehicle model

[0046] ,

[0060] ,

[0073] .

[0164] To improve 3D-SLAM accuracy, unimportant moving objects should be removed, as their presence can impact localization accuracy and mapping consistency

[0039] , Enhancing 3DSLAM accuracy further involves using deep -learning-based dynamic object filtering to label dynamic objects in the point cloud automatically

[0061] . Various SLAM frameworks like Gmapping, Karto, and Hector SLAM can detect moving objects, each with advantages depending on the environment and trajectories

[0082] .

[0165] Additional discussion. The exemplary system can recognize drones from birds and other flying objects even though both drones and birds may share similar physicalAttorney Docket No. 11555-017WO1UMA No. 25-006 characteristics (e.g., size, shape, and movement patterns) and often operate in similar environments and altitudes. Environmental conditions such as distance, lighting, weather, and background clutter can affect the accuracy of recognition algorithms. Fig. 2 shows the shapes of different drones and birds at close range, and Fig. 3 shows a point cloud of a drone and birds; they are visually indistinguishable at long distances.

[0166] Range-sensitivity consideration. Since the point cloud density of any object can differ depending on the distance of the object, developing a distance-independent tracking solution that is able to scale up to a large number of objects (i.e., tens of thousands) can be a challenging task. For example, the system receives a small number of reflected points at far distances compared to close ranges, as shown in Fig. 3C.

[0167] Complexity of trajectories of drones in a swarm consideration. Realtime tracking of drone swarms may be challenging due to their complex movement patterns, frequent path crossings, and rapid changes in direction, altitude, and speed. These factors may cause temporary occlusion and misidentification, complicating tasks like search / rescue, environmental monitoring, and tactical operations, 'fracking systems must quickly and accurately process large volumes of data despite these difficulties.

[0168] Scalability of monitoring drone swarm consideration. The scalability may be significantly challenged by computational demands, communication protocols, algorithmic accuracy, and limited research in this field. Increasing the number of drones escalates data processing complexity and communication, often surpassing current high-speed architectures. The accuracy of the tracking algorithm degrades with larger swarms, and the LiDAR sensor’s limitations in range and resolution further complicate scalability.

[0169] Conclusion

[0170] It must also be noted that, as used in the specification and the appended claims, the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as from “about” or “5 approximately” one particular value and / or to “about” or “approximately” another particular value. When such a range is expressed, other exemplary embodiments include one particular value and / or the other particular value.

[0171] By “comprising” or “containing” or “including,” is meant that at least the name compound, element, particle, or method step is present in the composition or article or method but does not exclude the presence of other compounds, materials, particles, method steps, even if the other such compounds, material, particles, method steps have the same function as what is named.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0172] In describing example embodiments, terminology will be resorted to for the sake of clarity. It is intended that each term contemplates its broadest meaning as understood by those skilled in the art and includes all technical equivalents that operate in a similar manner to accomplish a similar purpose. It is also to be understood that the mention of one or more steps of a method does not preclude the presence of additional method steps or intervening method steps between those steps expressly identified. Steps of a method may be performed in a different order than those described herein without departing from the scope of the present disclosure. Similarly, it is also to be understood that the mention of one or more components in a device or system does not preclude the presence of additional components or intervening components between those components expressly identified.

[0173] The following patents, applications, and publications, as listed below and throughout this document, are hereby incorporated by reference in their entirety herein.

[0174] The following patents, applications, and publications, as listed below and throughout this document, are hereby incorporated by reference in their entirety herein.Reference List[1] Tasnim Azad Abir, Endrowednes Kuantama, Richard Han, Judith Dawes, Rich Mildren, and Phuc Nguyen. Towards robust lidar-based 3d detection and tracking of uavs. In Proceedings of the Ninth Workshop on Micro Aerial Vehicle Networks, Systems, and Applications, DroNet ’23, page 1-7, New York, NY, USA, 2023. Association for Computing Machinery.

[0002] Leah Abraham, Sibi Biju, Felix Biju, Jithin Jose, Rakhi Kalantri, and Shahgufta Rajguru.Swarm robotics in disaster management. In 2019 International conference on innovative sustainable computational technologies (CISCT), pages 1-5. IEEE, 2019.[3] Ghulam E. Mustafa Abro, Saiful Azrin B. M. Zulkifli, Rana Javed Masood, Vijanth Sagayan Asirvadam, and Anis Laouiti. Comprehensive review of uav detection, security, and communication advancements to prevent threats. Drones, 6(10), 2022.[4] Chaudhry Awais Ahmed, Faryal Batool, Wajeeha Haider, Muhammad Asad, and Syed Hossein Raza Hamdani. Acoustic based drone detection via machine learning. In 2022 Internationa] Conference on IT and Industrial Technologies (ICIT), pages 01-06, 2022.[5] Hassan J. Al Dawasari, Muhammad Bilal, Muhammad Moinuddin, Kamran Arshad, and Khaled Assaleh. Deepvision: Enhanced drone detection and recognition in visible imagery through deep learning networks. Sensors, 23(21), 2023.Attorney Docket No. 11555-017WO1UMA No. 25-006 [6] Yaniv Altshuler, Alex Pentland, and Alfred M. Bruckstein. The Cooperative Hunters - Efficient and Scalable Drones Swarm for Multiple Targets Detection, pages 187-205. Springer International Publishing, Cham, 2018.[7] Nerya Ashush, Shlomo Greenberg, Erez Manor, and Yehuda Ben- Shimol. Unsupervised drones swarm characterization using rf signals analysis and machine learning methods. Sensors, 23(3), 2023.[8] Elena Ausonio, Patrizia Bagnerini, and Marco Ghio. Drone swarms in fire suppression activities: A conceptual framework. Drones, 5(1): 17, 2021.[9] Mohamed Barbary, Alaa S. Hafez, Mohamed H. Abd El-azeem, Tamer Soliman, Ahmed E. Elbanna, Mohamed Shalaby, Walid M. Saad, and Mohamed Mabrouk. Extended drones tracking from isar images with doppler effect and orientation based robust sub¬ random matrices algorithm. IEEE Transactions on Vehicular Technology,71(12): 12648— 12666, 2022.

[0010] Meysam Basiri, Felix Schill, Pedro Lima, and Dario Floreano. On-board relative bearing estimation for teams of drones using sound. IEEE Robotics and Automation letters, l(2):820-827, 2016.

[0011] Igor Bisio, Chiara Garibotto, Halar Haleem, Fabio Lavagetto, and Andrea Sciarrone.Rf / wifi-based uav surveillance systems: A systematic literature review. Internet of Things, page 101201, 2024.

[0012] Oscar Bjurling, Rego Granlund, Jens Alfredson, Mattias Arvola, and Tom Ziemke.Drone swarms in forest firefighting: A local development case study of multi-level human-swarm interaction. In Proceedings of the 11th Nordic Conference on Human- Computer Interaction: Shaping Experiences, Shaping Society, pages 1-7, 2020.

[0013] Vittorio Ugo Castrillo, Angelo Manco, Domenico Pascarella, and Gabriella Gigante. A review of counter-uas technologies for cooperative defensive teams of drones. Drones, 6(3), 2022.

[0014] R. Qi Charles, Hao Su, Mo Kaichun, and Leonidas J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 77-85, 2017.

[0015] Zijian Chen, Yu Miao, Dan Tang, Hao Yang, and Wenwu Pan. Effect of lidar receiver field of view on uav detection. Photonics, 9(12), 2022.

[0016] Florin-Lucian Chiper, Alexandru Martian, Calin Vladeanu, Ion Marghescu, Razvan Craciunescu, and Octavian Fratu. Drone detection and defense systems: Survey and a software-defined radio-based solution. Sensors, 22(4), 2022.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0017] Sirish Deshmukh and Jitesh Kumar. An sdr-based passive radar for detection and tracking of drones and uavs. In 2023 IEEE Microwaves, Antennas, and Propagation Conference (MAPCON), pages 1-5, 2023.

[0018] Sirish Deshmukh and Vaibhav Sharma. An sdr-based anti-drone system with detection, tracking, jamming, and spoofing capabilities. In 2022 IEEE Microwaves, Antennas, and Propagation Conference (MAPCON), pages 388-393, 2022.

[0019] Shreshtha Devgan, Sehaj Walia, Vineeta Khemchandani, and Sushil Chandra.Multi / swarm drone surveillance and monitoring system using vr simulation. In 2022 IEEE Global Conference on Computing, Power and Communication Technologies (GlobConPT), pages 1-6. IEEE, 2022.

[0020] Louis Dressel and Mykel J. Kochenderfer. Hunting drones with other drones: Tracking a moving radio target. In 2019 International Conference on Robotics and Automation (ICRA), pages 1905-1912, 2019.

[0021] H. Durrant- Whyte and T. Bailey. Simultaneous localization and mapping: part i. IEEE Robotics Automation Magazine, 13(2):99- 110, 2006.

[0022] Martin Ester, Hans-Peter Kriegel, Jorg Sander, and Xiaowei Xu. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, KDD’96, page 226-231. AAAI Press, 1996.

[0023] Martins Ezuma, Fatih Erden, Chethan Kumar Anjinappa, Ozgur Ozdemir, and Ismail Guvenc. Micro-uav detection and classification from rf fingerprints using machine learning techniques. In 2019 IEEE Aerospace Conference, pages 1-13, 2019.

[0024] Jian Fang, Yaowen Li, Philip N. Ji, and Ting Wang. Drone detection and localization using enhanced fiber-optic acoustic sensor and distributed acoustic sensing technology. Journal of Lightwave Technology, 41(3):822- 831, 2023.

[0025] Xiaofeng Fu, Guoting Wei, Xia Yuan, Yongshun Liang, and Yuming Bo. Efficient yolov7-drone: An enhanced object detection approach for drone aerial imagery.Drones, 7(10), 2023.

[0026] Ilias Gkotsis, Agavi-Christina Kousouraki, George Eftychidis, Panayiotis Kolios, and Maria Terzi. Swarm of uavs as an emergency response technology. In Risk Analysis Based on Data and Crisis Response Beyond Knowledge, pages 353-359. CRC Press, 2019.

[0027] Matthieu J Guitton. Fighting the locusts: Implementing military countermeasures against drones and drone swarms. Scandinavian Journal of Military Studies, 4( 1):26— 36, 2021.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0028] Junfeng Guo, Ishtiaq Ahmad, and KyungHi Chang. Classification, positioning, and tracking of drones by hmm using acoustic circular microphone array beamforming. EURASIP J. Wirel. Commun. Netw., 2020(1), jan 2020.

[0029] Marcus Hammer, Marcus Hebel, Bjorn Borgmann, Martin Laurenzis, and Michael Arens. Potential of lidar sensors for the detection of uavs. In Defense + Security, 2018.

[0030] Gert Herold, Adam Kujawski, Christoph Strümpfel, Svenja Huschbeck, Maarten Uijt de Haag, and Ennes Sarradj. Detection and separate tracking of swarm quadcopter drones using microphone array measurements. In Proceedings of the Berlin Beamforming Conference (BeBeC), Berlin, Germany, pages 2-3, 2020.

[0031] Bo Huang, Jianan Li, Junjie Chen, Gang Wang, Jian Zhao, and Tingfa Xu. Anti-uav410:A thermal infrared benchmark and customized scheme for tracking drones in the wild. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(5):2852- 2865, 2024.

[0032] Mauro S Innocente and Paolo Grasso. Self-organising swarms of firefighting drones:Harnessing the power of collective intelligence in decentralised multi -robot systems. Journal of Computational Science, 34:80-101, 2019.

[0033] CI Collective Intelligence, ML Machine Learning, MSP Mobile Sensing Platform, F Saffre, H Karvonen, and H Hildmann. Wild swarms: Autonomous drones for environmental monitoring and protection. New Developments and Environmental Applications of Drones: Proceedings of FinDrones 2023, page 1, 2024.

[0034] Cheolmin Jeon, Jeongsoo Ha, Hanbum Ko, Byeongman Lee, and Bo Ryu. Swarmsense:Effective and resilient drone swarm and search for disaster response and management application.

[0035] Zachary Kallenborn. Infoswarms: Drone swarms and information warfare. The US ArmyWar College Quarterly: Parameters, 52(2):87— 102, 2022.[36 J Zachary Kallenborn and Philipp C Bleek. Swarming destruction: drone swarms and chemical, biological, radiological, and nuclear weapons. The nonproliferation review, 25(5-6):523-543, 2018.

[0037] Wahab Khawaja, Qasim Yaqoob, and Ismail Guvenc. Rl-based detection, tracking, and classification of malicious uav swarms through airborne cognitive multibeam multifunction phased array radar. Drones, 7(7), 2023.

[0038] Byeong Hak Kim, Danish Khan, Ciril Bohak, Wonju Choi, Hyun Jeong Lee, and Min Young Kim. V-rbnn based small drone detection in augmented datasets for 3d ladar system. Sensors, 18(11), 2018.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0039] Jun Konno and Yoshinobu Ando. Improvement of 3d-slam accuracy by removing moving objects on 3d-lidar point cloud using image recognition in web camera. In 202222nd International Conference on Control, Automation and Systems (ICCAS), pages 527-531, 2022.

[0040] T Kumarasan, MVinoth, MDurgaram, andMBenedict. Drone-enabled wildlife monitoring system: Revolutionizing conservation efforts. NeuroQuantology, 18(8):330, 2020.

[0041] Nisha Kumari, Kevin Lee, Jan Carlo Barca, and Chathurika Ranaweera. Towards reliable identification and tracking of drones within a swarm. J. Intell. Robotic Syst., 110:84, 2024.

[0042] Nisha Kumari, Kevin Lee, Chathurika Ranaweera, and Jan Carlo Barca. Visually detecting drones in drone swarm formations topologies. In Abrar Ullah, Sajid Anwar, Davide Calandra, and Raffaele Di Fuccio, editors, Proceedings of International Conference on Information Technology and Applications, pages 21-30, Singapore, 2024. Springer Nature Singapore.

[0043] Masashi Kurosaki, Kenshi Ogawa, Ryohei Nakamura, Keiji Jimi, and Hisaya Hadama.Experimental study on multiple drone detection using a millimeter-wave fast chirp mimo radar. In 2023 IEEE Topical Conference on Wireless Sensors and Sensor Networks, pages 16-19, 2023.

[0044] Kaya Kuril, Darren Ansell, Wasiq Khan, and Halil Yetgin. Analysis and optimization of unmanned aerial vehicle swarms in logistics: An intelligent delivery platform.7:15804—15831.

[0045] Dong-Hyun Lee. Cnn-based single object detection and tracking in videos and its application to drone detection. Multimedia Tools Appl., 80(26-27): 34237-34248, nov 2021.

[0046] FeiyaLi, Chunyun Fu, SujuanWang, Yun Zhu, Zhankun Li, and Binbin Feng. Lidar¬ based vehicle detection for dynamic slam applications. In 2021 International Conference on Control, Automation and Information Sciences (ICCAIS), pages 674- 678.2021.

[0047] Kailai Li, Meng Li, and Uwe D Hanebeck. Towards high-performance solid-state-lidar- inertial odometry and mapping. IEEE Robotics and Automation Letters, 6(3):5167— 5174. 2021.

[0048] Yuhan Li, Maozhong Fu, Haixin Sun, Zhenmiao Deng, and Yunjian Zhang. Radar-based uav swarm surveillance based on a twostage wave path difference estimation method. IEEE Sensors Journal, 22(5):4268-4280, 2022.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0049] Pengfei Liu. Research on detection and attack strategy against swarm of drones. In 2023 IEEE 11th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), volume 11, pages 1648- 1653, 2023.

[0050] Kaliappan Madasamy, Vimal Shanmuganathan, Vijayalakshmi Kandasamy, Mi Young Lee, and Manikandan Thangadurai. Osddy: embedded system-based object surveillance detection system with small drone using deep yolo. J. Image Video Process., 2021(1), may 2021.

[0051] Federico Mason, Martina Capuzzo, Davide Magrin, Federico Chiariotti, Andrea Zanella, and Michele Zorzi. Remote tracking of uav swarms via 3d mobility models and lorawan communications. IEEE Transactions on Wireless Communications, 21(5):2953 -2968, 2022.

[0052] Olusiji O Medaiyese, Abbas Syed, and Adrian P Lauf. Machine learning framework for rf-based drone detection and identification system. In 2021 2nd International Conference On Smart Cities, Automation Intelligent Computing Systems (ICON¬ SONICS), pages 58-64, 2021.

[0053] Varun Mehta, Fardad Dadboud, Miodrag Bolic, and Iraj Mantegh. A deep learning approach for drone detection and classification using radar and camera sensor fusion. In 2023 IEEE Sensors Applications Symposium (SAS), pages 01-06, 2023.

[0054] Daniel Morrison, Simon Kennedy, Dennis Delic, Mehmet Rasit Yuce, and Jean-Michel Redoute. A 64 x 64 spad flash lidar sensor using a triple integration timing technique with 1.95 mm depth resolution. IEEE Sensors Journal, 21(10): 11361— 11373, 2020.

[0055] Dan Mototolea and Cornells Stolk. Detection and localization of small drones using commercial off-the-shelf fpga based software defined radio systems. In 2018 International Conference on Communications (COMM), pages 465-470, 2018.

[0056] Phuc Nguyen, Vimal Kakaraparthi, Nam Bui, Nikshep Umamahesh, Nhat Pham, Hoang Truong, Yeswanth Guddeti, Dinesh Bharadia, Richard Han, Eric Frew, el al.Dronescale: drone load estimation via remote passive if sensing. In Proceedings of the 18th Conference on Embedded Networked Sensor Systems, pages 326-339, 2020.

[0057] Phuc Nguyen, Hoang Truong, Mahesh Ravindranathan, Anh Nguyen, Richard Han, and Tam Vu. Matthan: Drone presence detection by identifying physical signatures in the drone’s rf communication. In Proceedings of the 15th annual international conference on mobile systems, applications, and services, pages 211-224, 2017.

[0058] Phuc Nguyen, Hoang Truong, Mahesh Ravindranathan, Anh Nguyen, Richard Han, and Tam Vu. Matthan: Drone presence detection by identifying physical signatures in theAttorney Docket No. 11555-017WO1UMA No. 25-006 drone’s rf communication. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys ’17, page 211-224, New York, NY, USA, 2017. Association for Computing Machinery.

[0059] Bethi Pardhasaradhi and Linga Reddy Cenkeramaddi. Gps spoofing detection and mitigation for drones using distributed radar tracking and fusion. IEEE Sensors Journal, 22(11 ): 11122-11134, 2022.

[0060] Hongrui Peng, Ziyu Zhao, and Liguan Wang. A review of dynamic object filtering in slam based on 3d lidar. Sensors, 24(2), 2024.

[0061] Patrick Pfreundschuh, Hubertus F. C. Hendrikx, Victor Reijgwart, Renaud Dube, Roland Siegwart, and Andrei Cramariuc. Dynamic object aware lidar slam based on automatic generation of training data. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 11641-11647, 2021.

[0062] Ahnirudh Y. Raj, Akshaya Venkatraman, Anish Vinodh, and Hariharank Kumar.Autonomous drone for smart monitoring of an agricultural field. In 2021 7th International Engineering Conference “Research & Innovation amid Global Pandemic" (IEC), pages 211-212.

[0063] Thinal Raj, Fazida Hanim Hashim, Aqilali Baseri Huddin, Mohd Faisal Ibrahim, and Aini Hussain. A survey on lidar scanning mechanisms. Electronics, 9(5):741, 2020.[64 J Fabrice Saffre, Hanno Hildmann, Hannu Karvonen, and Timo Lind. Monitoring and cordoning wildfires with an autonomous swarm of unmanned aerial vehicles. Drones, 6(10):301, 2022.

[0065] Fabrice Saffre, Hannu Karvonen, and Hanno Hildmann. Wild swarms: Autonomous drones for environmental monitoring and protection. In International conference on FinDrones, pages 1-32. Springer, 2023.

[0066] Shyam Sai, Istvan Kovach, Abhay Gupta, and James Wei. 2017:" lighting up the night".2017.

[0067] Meriem Salhi and Noureddine Boudriga. Multi-array spherical lidar system for drone detection. In 202022nd International Conference on Transparent Optical Networks (ICTON), pages 1-5, 2020.

[0068] Daniele Salvati, Carlo Drioli, Giovanni Fenin, and Gian LucaForesti. Acoustic source localization from multirotor uavs. IEEE Transactions on Industrial Electronics, 67(10):8618 -8628, 2020.

[0069] Andrew W Sanders. Drone swarms. US Army School for Advanced Military Studies Fort Leavenworth United States, 2017.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0070] Keyvan Sayyah, Raymond Sarkissian, Pamela Patterson, Biqin Huang, Oleg Efimov, Danny Kim, Ken Elliott, Louis Yang, and David Hammon. Fully integrated fmcw lidar optical engine on a single silicon chip. Journal of Lightwave Technology, 40(9):2763- 2772, 2022.

[0071] Fabian Schilling, Fabrizio Schiano, and Dario Floreano. Vision-based drone flocking in outdoor environments. IEEE Robotics and Automation Letters, 6(2):2954— 2961, 2021.

[0072] Ulzhalgas Seidaliyeva, Lyazzat Ilipbayeva, Kyrmyzy Taissariyeva, Nurzhigit Smailov, and Eric T. Matson. Advances and challenges in drone detection and classification techniques: A state-of-the-art review. Sensors, 24(1), 2024.

[0073] Bo-Xian Wu Shih-An Li, Yun-Chien Chen and Hsuan-Ming Feng. 3d lidar slam-based systems in object detection and navigation applications. Journal of the Chinese Institute of Engineers, 46(8):912— 925, 2023.

[0074] Petr Skobelev, Denis Budaev, Nikolay Gusev, and Georgy Voschuk. Designing multiagent swarm of UAV for precise agriculture. In Javier Bajo, Juan M. Corchado, Elena Marla Navarro Martinez, Eneko Osaba Icedo, Philippe Mathieu, Patrycja Hoffa- Dqbrowska, Elena del Vai, Sylvain Giroux, Antonio J. M. Castro, Nayat Sanchez-Pi, Vicente Julian, Ricardo Azambuja Silveira, Alberto Fernandez, Rainer Unland, and Ruben Fuentes-Fernandez, editors, Highlights of Practical Applications of Agents, Multi-Agent Systems, and Complexity: The PAAMS Collection, pages 47-59.Springer International Publishing.

[0075] Soonyong Song, Youngsung Son, and Youngil Kim. Flying drone classification based on visualization of acoustic signals with deep neural networks. In 2020 International Conference on Information and Communication Technology Convergence (ICTC), pages 546-548, 2020.

[0076] Sutthiphong Srigrarom, Niven Jun Liang Sie, Huimin Cheng, Kim Hoe Chew, Mengda Lee, and Photchara Ratsamee. Multi-camera multi-drone detection, tracking and localization with trajectory-based reidentification. In 2021 Second International Symposium on Instrumentation, Control, Artificial Intelligence, and Robotics (ICA- SYMP), pages 1-6. IEEE, 2021.

[0077] Panayiota Valianti, Panayiotis Kolios, and Georgios Ellinas. Energy-aware tracking and jamming rogue uavs using a swarm of pursuer uav agents. IEEE Systems Journal, 17(1): 1524— 1535. 2023.Attorney Docket No. 11555-017WO1UMA No. 25-006

[0078] Dinh Van Nam and Kim Gon-Woo. Solid-state lidar based-slam: A concise review and application. In 2021 IEEE International Conference on Big Data and Smart Computing (BigComp), pages 302-305. IEEE, 2021.

[0079] Federica Villa, Fabio Severini, Francesca Madonini, and Franco Zappa. Spads and sipms arrays for long-range high-speed light detection and ranging (lidar). Sensors, 21(11):3839, 2021.

[0080] Chuanyun Wang, Linlin Meng, Qian Gao, Jingjing Wang, Tian Wang, Xiaona Liu, Furui Du, Linlin Wang, and Ershen Wang. A lightweight uav swarm detection method integrated attention mechanism. Drones, 7(1), 2023.

[0081] Siying Wang and Reinhold Herschel. Fast 3d-cfar for drone detection with mimo radars.In 2021 18th European Radar Conference (EuRAD), pages 209-212, 2022.

[0082] Jimyeong Woo, Hyein Jeong, and Heoncheol Lee. Comparison and analysis of lidarbased slam frameworks in dynamic environments with moving objects. In 2021 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), pages 1-4, 2021.

[0083] Shuai Yin, Renjie He, Jingjing Id, Id Chen, and Shuangxiong Zhang. Research on the operational mode of manned / unmanned collaboratively detecting drone swarm. In 2021 IEEE International Conference on Unmanned Systems (ICUS), pages 560-564, 2021.

[0084] Xiaosheng Zhang, Jazz Pouls, and Ming CWu. Laser frequency sweep linearization by iterative learning pre-distortion for fmcw lidar. Optics express, 27(7):9965-9974, 2019.

[0085] Pengfei Zhu, Jiayu Zheng, Dawei Du, Longyin Wen, Yiming Sun, and Qinghua Hu.Multi-drone-based single object tracking with agent sharing network. IEEE Transactions on Circuits and Systems for Video Technology, 31(10):4058-4070, 2021.

[0086] Oleg Zubkov, Sergiy Sheiko, Vladimir Kartashov, and Vladimir Oleynikov. Detection of small drones in thermal infrared range using yolov5 neural networks. In 2022 IEEE 9th International Conference on Problems of Infocommunications, Science and Technology (PIC ST), pages 283-287, 2022.

[0087] Abir, T. A., Le, V., Kuantama, E., Gupta, P. S., Copley, A., Dawes, J., Islam, M., Han, R., and Nguyen, P. (2025). Detection and Tracking of Drone Swarms using LiDAR. In Proceedings of the 23rd Annual International Conference on Mobile Systems, Applications and Services, Association for Computing Machinery, New York, NY, USA, pp. 500-514. Available at: https: / / doi.org / 10.1145 / 3711875.3729156.

Claims

Attorney Docket No. 11555-017WO1UMA No. 25-006 What is claimed:

1. A system comprising:a processor; anda memory having instructions stored thereon for detecting and monitoring a drone swarm, wherein execution of the instructions by the processor causes the processor to:receive LiDAR measurement of a LiDAR scan;generate a point cloud data object for the LiDAR measurement; generate, via a computer vision operation, a cluster of the point cloud data object;to merge the cluster, or an intermediate object generated therefrom, at different time scans; andclassify, via a trained Al classifier, drones in the point cloud data object and exclude non-drone objects, wherein the classified outputs are used for detecting and monitoring a drone swarm.

2. The system of claim 1, wherein the instructions to generate the point cloud data object include:instructions to generate, via a triangulation operation, a shape object of the cluster as the intermediate object, wherein the classified outputs are generated from the merged shape object of the cluster.

3. The system of claim 1 or 2, wherein the trained Al classifier is a binary classifier.

4. The system of any one of claims 1-3, wherein the trained Al classifier was trained using the shape object of the cluster of LiDAR scans.

5. The system of any one of claims 1-4, wherein the system is a stationary platform.

6. The system of any one of claims 1-4, wherein the system is a mobile platform.

7. The system of any one of claims 1-6, wherein the instructions, when executed by the processor, further cause the processor to track individual drones within the swarm.

8. The system of claim 7, wherein the instructions to track the individual drones include:Attorney Docket No. 11555-017WO1UMA No. 25-006 instructions to:merge a plurality of the classified outputs at different time scans to generate a time sequence of the classified outputs:interpolate the time sequence of the classified outputs; andreconstruct trajectories of each classified output in the interpolated time sequence of classified outputs.

9. The system of claim 8, wherein the instructions to track the individual drones include instructions to plot the time sequence of the classified outputs and a trajectory path of the individual drones.

10. The system of any one of claims 1-9, wherein the classified outputs are used for the detection of a drone or drone swarm being in proximity to a pre-defined location, and wherein the classified outputs are used to generate an alert.

11. The system of any one of claims 1-9, wherein the classified outputs are used for the detection of a drone or drone swarm being out of formation of a pre-defined swarm formation, and wherein the classified outputs are used to generate an alert or for control of the swarm.

12. The system of any one of claims 1-9, wherein the execution of the instructions further causes the processor to:receive (i) a second LiDAR measurement of a second LiDAR scan and / or (ii) a second sensor measurement of a second sensor scan, each acquired concurrently with the LiDAR measurement, wherein the second LiDAR scan and / or the second sensor scan are time synchronized with the LiDAR measurement;generate a respective point cloud data object for the second LiDAR measurement or the second measurement;generate, via a computer vision operation, a respective cluster of the point cloud data object;generate, via a triangulation operation, a respective shape object of the respective cluster;merge the shape object of the cluster generated at different time scans and the respective shape object of the respective cluster at the same different time scans; andAttorney Docket No. 11555-017WO1UMA No. 25-006 classify, via the trained Al classifier, drones in the merged point cloud data object and exclude non-drone objects.

13. The system of claim 12, wherein the second sensor measurement or the second sensor scan is a thermal measurement by a thermal camera.

14. A method comprising:receiving LiDAR measurement of a LiDAR scan;generating a point cloud data object for the LiDAR measurement;generating, via a computer vision operation, a cluster of the point cloud data object; generating, via a triangulation operation, a shape object of the cluster; and classifying, via a trained Al classifier, drones in the point cloud data object and excluding non-drone objects, wherein the classified outputs are used for detecting and monitoring a drone swarm.

15. The method of claim 14, wherein the classification of the drones includes:merging the shape object of the cluster generated at different time scans, wherein the classified outputs are generated from the merged shape object of the cluster.

16. The method of claim 14 or 15, wherein the trained Al classifier is a binary classifier.

17. The method of any one of claims 14-16, wherein the trained Al classifier was trained using the shape object of the cluster of LiDAR scans.

18. The method of any one of claims 14-17, further comprising:tracking individual drones within the swarm.

19. The method of claim 18, wherein the tracking of the individual drones includes: merging a plurality of the classified outputs at different time scans to generate a time sequence of classified outputs;interpolating the time sequence of classified outputs; andreconstructing trajectories of each classified output in the interpolated time sequence of classified outputs.Attorney Docket No. 11555-017WO1UMA No. 25-006 20. A non-transitory computer-readable medium having instructions stored thereon, wherein execution of the instructions causes a processor to:receive LiDAR measurement of a LiDAR scan;generate a point cloud data object for the LiDAR measurement;generate, via a computer vision operation, a cluster of the point cloud data object; generate, via a triangulation operation, a shape object of the cluster; and classify, via a trained Al classifier, drones in the point cloud data object and exclude non-drone objects, wherein the classified outputs are used for detecting and monitoring a drone swarm.

21. The non-transitory computer-readable medium of claim 20, wherein the classification of the drones includes:merging the shape object of the cluster generated at different time scans, wherein the classified outputs are generated from the merged shape object of the cluster.

22. The non-transitory computer-readable medium of claim 20 or 21, wherein the trained Al classifier is a binary classifier.

23. The non-transitory computer-readable medium of any one of claims 20-22, wherein the trained Al classifier was trained using the shape object of the cluster of LiDAR scans.

24. The non-transitory computer-readable medium of any one of claims 20-23, wherein the instructions, when executed by the processor, further cause the processor to track individual drones within the swarm.

25. The non-transitory computer-readable medium of claim 24, wherein the tracking of the individual drones includes:merging a plurality of the classified outputs at different time scans to generate a time sequence of classified outputs;interpolating the time sequence of classified outputs; andreconstructing trajectories of each classified output in the interpolated time sequence of classified outputs.