Unmanned aerial vehicle tracking system and method based on intelligent lamp pole network

Abstract

The invention discloses an unmanned aerial vehicle tracking system and method based on an intelligent lamp pole network, and the system comprises a sensing layer which is disposed on a plurality of intelligent lamp poles and is used for obtaining sensing data; the network layer is used for transmitting the sensing data; the platform layer is used for receiving the sensing data and carrying out intelligent processing and decision making; wherein the sensing layer, the network layer and the platform layer are connected in sequence; the sensing layer is composed of heterogeneous multi-mode sensing units deployed on a plurality of intelligent lamp poles, and the intelligent lamp poles are deployment carriers of the system. According to the invention, full-process automatic tracking from unmanned aerial vehicle detection and identification to continuous trajectory generation is realized.

Description

Technical Field

[0001] This invention relates to the field of drone monitoring technology, and in particular to a drone tracking system and method based on a smart light pole network. Background Technology

[0002] With the popularization of drone technology and the rapid development of the low-altitude economy, its applications in logistics, surveying, entertainment and other fields are becoming increasingly widespread. At the same time, security issues such as unauthorized flights and illegal intrusions are becoming increasingly prominent. Therefore, the routine detection, identification and continuous tracking of drones, especially unauthorized drones, in a wide area has become a core technological requirement for low-altitude security and smart city management.

[0003] Although existing technical solutions have continued to evolve, from single sensors to multi-source fusion, and then to multi-station collaboration, there are still bottlenecks in system architecture, fusion depth, or deployment cost that have not been overcome, making it difficult to meet the above comprehensive requirements.

[0004] (1) The single-sensor sensing scheme has inherent defects:

[0005] Such solutions suffer from severe deficiencies in environmental adaptability and reliability due to their limited perception dimensions. Visible light camera-based solutions are constrained by lighting and weather conditions, exhibiting a sharp decline in performance at night, in foggy or bright light environments, making all-weather operation impossible. While radio frequency signal analysis-based solutions are unaffected by light, they are susceptible to electromagnetic interference and completely fail when the drone is in silent communication mode, unable to provide visual morphology and precise spatial attitude information of the target.

[0006] (2) Existing multi-source sensing and collaborative solutions have limitations in terms of architecture, fusion depth, and application areas:

[0007] To improve performance, the industry has proposed multi-source fusion and collaborative solutions. However, these solutions all have significant shortcomings in their core pathways:

[0008] a. Bottleneck of Homogeneous Sensor Network Overlay: Existing technologies include a category of solutions that focus on multi-station collaboration of homogeneous sensors. For example, patent CN111474953A discloses a multi-dynamic-view collaborative aerial target recognition method, the core of which lies in fusing image data from multiple mobile cameras through a pure visual algorithm. This type of solution is essentially an overlay of the same type of sensors, failing to address the inherent physical limitations of this type of sensor (such as the collective failure of the visual system at night). Another type of solution, such as the multi-UAV collaborative tracking disclosed in CN116127848A, focuses on the collaborative control algorithm of the mobile platform, similarly failing to change the inherently homogeneous and limited capabilities of its airborne sensors.

[0009] b. Heterogeneous fusion solutions are mostly focused on UAV autonomous control, rather than ground-side monitoring: Existing multi-source perception fusion technologies are largely applied to the autonomous navigation and control of the UAV itself. For example, patent CN120722928A discloses a UAV dynamic path control method based on multi-source perception fusion, which integrates multiple sensors onto the UAV platform to provide decision-making support for UAV flight. Patent CN120762452A also discloses a UAV real-time path planning system based on dynamic weight allocation and multi-source data fusion. A common feature of these solutions is the integration of multiple sensors onto the UAV platform, aiming to support the UAV's flight safety and mission decision-making. These solutions belong to an integrated "perception-control" autonomous system, with a focus on optimizing onboard algorithms. They completely fail to address the fundamentally different technical problem of how a ground system can detect, identify, and track an unknown UAV from scratch.

[0010] c. Heterogeneous fusion solutions for monitoring suffer from shallow fusion and centralized processing issues: Even in the few heterogeneous fusion solutions for monitoring, the fusion mode often remains at the level of post-decision fusion or simple feature overlay. This passive and shallow fusion cannot achieve real-time, proactive interaction and compensation of different modal data at the feature level, resulting in limited improvement in the system's adaptability and robustness in complex scenarios (such as the instantaneous failure of a single modality). In addition, such solutions generally adopt a centralized data processing architecture, resulting in large system response latency and high central computing load, making it difficult to meet the real-time tracking requirements of high-speed moving targets, and the system's scalability and robustness are limited by the central node.

[0011] d. Existing collaborative solutions have failed to address the core issue of low-cost grid-based deployment: whether homogeneous or heterogeneous integration solutions, their deployment typically relies on high-cost dedicated platforms (such as drone swarms and dedicated monitoring towers). They fail to provide a systematic distributed architecture that can be perfectly integrated with existing urban infrastructure such as smart streetlights, resulting in the inability to achieve economically feasible city-level wide-area coverage.

[0012] (3) The problem of blind spots and lack of coordination in integrated equipment:

[0013] In fixed deployment schemes, patent application CN119270907A provides an integrated device for drone detection, tracking, and monitoring, reflecting the concept of hardware integration. However, this type of "centralized integrated" solution has fundamental limitations:

[0014] Firstly, its coverage is strictly limited by the physical performance and installation location of individual devices, making it impossible to achieve seamless and continuous tracking over a large area of ​​airspace, resulting in numerous monitoring blind spots.

[0015] Secondly, it is a typical "information silo," lacking an effective collaboration mechanism between devices. Even if multiple such devices are deployed, they cannot be organically networked and linked together, making it difficult to form a tracking relay.

[0016] Third, deploying a large number of such high-performance integrated devices to achieve wide-area coverage would result in extremely high costs, making it difficult to achieve smart city-level grid coverage under economically feasible conditions.

[0017] In summary, existing technologies each address a specific aspect of the tracking problem, but they are constrained by multiple core contradictions, including homogeneous perception, application domain bias, insufficient depth of heterogeneous fusion, centralized processing architecture, lack of inter-device collaboration, and high deployment costs. These contradictions make it difficult to simultaneously achieve robust perception, real-time decision-making, continuous coverage, and system economy. Therefore, there is an urgent need in this field for an innovative system architecture that can fundamentally integrate heterogeneous multimodal perception, achieve distributed intelligent collaboration, and fully utilize existing urban infrastructure to build a low-cost perception network, thereby achieving efficient, reliable, and scalable drone tracking capabilities. Summary of the Invention

[0018] This invention proposes a drone tracking system and method based on a smart light pole network, aiming to solve the technical problems in existing technologies that prevent the realization of wide-area, highly reliable, and low-cost routine drone tracking due to homogeneous perception, application domain deviation, insufficient depth of heterogeneous integration, centralized processing architecture, lack of inter-device collaboration, and high-cost deployment.

[0019] On the one hand, to achieve the above objectives, the present invention provides a drone tracking system based on a smart light pole network, comprising:

[0020] Perception layer: Deployed on several smart light poles to acquire sensor data;

[0021] Network layer: used to transmit the sensed data;

[0022] Platform layer: Used to receive the sensed data and perform intelligent processing and decision-making;

[0023] The perception layer, the network layer, and the platform layer are connected in sequence; the perception layer consists of heterogeneous multimodal perception units deployed on several smart light poles, and the smart light poles serve as the deployment carriers of the system.

[0024] Preferably, the heterogeneous multimodal sensing unit includes a video acquisition module and a spectrum monitoring module;

[0025] The video acquisition module is a high-definition network camera with day / night switching function; the spectrum monitoring module is a software-defined wireless device with an operating frequency band covering 2.4GHz and 5.8GHz, and the spectrum monitoring module only receives communication signals from the drone and its remote controller.

[0026] Preferably, the platform layer includes edge computing nodes and cloud collaboration servers;

[0027] The edge computing node is embedded in the controller of the smart light pole and is used to perform front-end preprocessing and local fusion recognition on the data collected by the heterogeneous multimodal sensing unit.

[0028] The cloud-based collaborative server communicates with all edge computing nodes via TCP / IP protocol and is used to perform multi-source data fusion, 3D positioning, and trajectory generation across light poles.

[0029] Preferably, the edge computing node runs a lightweight deep learning target detection model on the video data, outputs the bounding box coordinates of the UAV in the image coordinate system, performs short-time Fourier transform and radio frequency fingerprint feature extraction on the spectrum data, identifies the UAV signal and estimates the direction of arrival.

[0030] Preferably, the cloud-based collaborative server performs multi-source data fusion, 3D positioning, and trajectory generation across light poles, including:

[0031] Perform time synchronization and spatial registration of data from multiple nodes;

[0032] When several video acquisition modules simultaneously identify the target, the three-dimensional position of the drone is calculated using a multi-view visual geometric positioning algorithm.

[0033] When several spectrum monitoring modules detect a signal, the position of the UAV is estimated by cross-positioning based on the signal arrival time difference or the direction of arrival.

[0034] By employing Kalman filtering or particle filtering algorithms and fusing visual positioning results with spectral positioning results, a continuous and smooth motion trajectory is generated.

[0035] On the other hand, to achieve the above objectives, the present invention also provides a drone tracking method based on a smart light pole network, comprising:

[0036] Video and spectrum data of the monitoring area are collected synchronously by heterogeneous multimodal sensing units deployed on several smart light poles;

[0037] The video data and the spectrum data are processed separately using edge computing nodes to identify drone targets and extract feature information.

[0038] The processed recognition results are uploaded to the cloud-based collaborative server;

[0039] Based on the cloud-based collaborative server, multi-source data is fused, and a combination of visual geometric positioning and spectrum cross-positioning is used to calculate the continuous three-dimensional position of the UAV.

[0040] The drone's trajectory is generated based on the continuous three-dimensional position, and tracking and prediction are performed.

[0041] Preferably, the edge computing node runs a lightweight deep learning target detection model on the video data and outputs the bounding box coordinates of the UAV in the image coordinate system; wherein, the lightweight deep learning detection model is a model optimized based on YOLO or SSD architecture and trained specifically for UAV targets.

[0042] Preferably, the feature analysis of the spectral data by the edge computing node includes:

[0043] The time-domain signal is converted into a time-frequency graph using short-time Fourier transform;

[0044] The radio frequency fingerprint features of the signal are extracted through the time-frequency diagram, the UAV signal is identified based on the convolutional neural network classifier, and the direction of arrival of the signal is estimated based on the MUSIC algorithm or the direction-of-arrival difference method using a multi-antenna array.

[0045] Preferably, the spectrum cross-positioning uses the TDOA algorithm, which iteratively solves the distance difference equation using the Chan algorithm or Taylor series expansion to obtain the approximate coordinates of the UAV.

[0046] Preferably, the visual geometric localization includes:

[0047] Based on the intrinsic and extrinsic parameters of several cameras, an overdetermined linear equation system is constructed.

[0048] The three-dimensional coordinates of the UAV are obtained by solving the overdetermined linear equations through singular value decomposition.

[0049] Compared with the prior art, the present invention has the following advantages and technical effects:

[0050] (1) Strong perception robustness: Through the heterogeneous multimodal fusion of video and spectrum, the perception capabilities are complementary. Vision provides rich details during the day, while spectrum ensures that the target is not lost at night and under occlusion conditions, which greatly improves the reliability and adaptability of the system in different environments.

[0051] (2) High system real-time performance: Through the collaborative computing architecture of "edge + cloud", some computing tasks are pushed down to edge nodes, which reduces the cloud load and network transmission pressure, significantly reduces the overall system response latency, and meets the real-time tracking requirements of high-speed moving targets.

[0052] (3) Wide coverage and no blind spots: The natural grid network formed by smart light poles achieves seamless coverage of the monitoring area. Multi-node collaborative work can realize tracking relay. When the target flies out of the field of view of a node, it can be automatically taken over by the next node, avoiding the blind spot problem of single-point equipment.

[0053] (4) Low deployment cost and good scalability: Make full use of the existing and densely distributed smart light poles in the city as carriers and adopt a low-cost passive spectrum monitoring scheme (non-radar), which greatly reduces the cost of infrastructure investment and makes large-scale, city-level low-altitude security deployment possible. Attached Figure Description

[0054] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0055] Figure 1 This is a schematic diagram of a drone tracking system based on a smart light pole network according to an embodiment of the present invention;

[0056] Figure 2 This is a schematic diagram of the smart light pole structure according to an embodiment of the present invention;

[0057] Figure 3 This is a flowchart of a drone tracking method based on a smart light pole network according to an embodiment of the present invention;

[0058] Among them, 100 is the smart light pole, 104 is the heterogeneous multimodal sensing unit, 105 is the edge computing node, 106 is the cloud collaborative server, 1041 is the video acquisition module, and 1042 is the spectrum monitoring module. Detailed Implementation

[0059] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0060] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0061] This embodiment proposes a drone tracking system based on a smart light pole network, such as... Figure 1 ,include:

[0062] Perception layer: Deployed on several smart light poles 100 to acquire perception data;

[0063] Network layer: used to transmit the sensed data;

[0064] Platform layer: Used to receive the sensed data and perform intelligent processing and decision-making;

[0065] The perception layer, the network layer, and the platform layer are connected in sequence. The perception layer is composed of heterogeneous multimodal perception units 104 deployed on several smart light poles 100, and the smart light poles 100 serve as the deployment carrier of the system.

[0066] This embodiment enables all-weather, highly reliable drone perception and identification, overcoming the inherent limitations of single sensors or homogeneous sensor networks. By constructing a low-latency, highly robust distributed processing architecture, it avoids the latency and load bottlenecks caused by centralized processing. This embodiment fully utilizes existing smart street light infrastructure to achieve low-cost, wide-coverage grid deployment, solving the economic issues of system deployment and realizing fully automated tracking from drone detection and identification to continuous trajectory generation.

[0067] Specifically, the perception layer is the data acquisition front end of the system, which consists of heterogeneous multimodal perception units 104 deployed on multiple smart light poles 100.

[0068] The smart light pole 100 serves as the deployment carrier of the system, providing a physical installation foundation, power supply, and communication interface for the heterogeneous multimodal sensing unit 104 and edge computing node 105. Widely distributed in public areas such as urban roads and squares, the smart light pole 100 naturally forms a densely distributed sensing network with wide coverage. Figure 2 .

[0069] Furthermore, the heterogeneous multimodal sensing unit 104 includes a video acquisition module 1041 and a spectrum monitoring module 1042;

[0070] Among them, the video acquisition module 1041 uses a high-definition network camera with more than 2 million pixels, and has automatic exposure and day and night switching functions. It is mainly used to capture continuous video streams in the monitored airspace and provide visual information of the target.

[0071] The spectrum monitoring module 1042 is a software-defined radio device (SDR, such as ADALM-PLUTO or RTL-SDR) that operates in the 2.4 GHz and 5.8 GHz frequency bands. This module operates in a passive listening mode, that is, it does not emit any electromagnetic waves itself, but only receives communication signals from the UAV and its remote controller. It is different from radar devices that actively emit electromagnetic waves and analyze the echoes.

[0072] Furthermore, the network layer serves as the data transmission link of the system. It utilizes the wired (such as fiber optic Ethernet) and wireless (such as 4G / 5G) communication networks built into the smart light pole 100 to reliably and with low latency transmit the data collected by the perception layer to the platform layer.

[0073] Furthermore, the platform layer includes an edge computing node 105 and a cloud collaboration server 106;

[0074] The edge computing node 105 is embedded in the controller of the smart light pole 100 and is used to perform front-end preprocessing and local fusion recognition on the data collected by the heterogeneous multimodal sensing unit 104.

[0075] The cloud-based collaborative server 106 communicates with all edge computing nodes 105 via TCP / IP protocol and is used to perform multi-source data fusion, 3D positioning and trajectory generation across light poles.

[0076] Specifically, edge computing node 105: This is an optional but preferred component, which can be an embedded AI computing device (such as the NVIDIA Jetson Nano series) embedded in the controller chassis of the smart light pole 100. Its main responsibility is to perform front-end preprocessing on the raw data collected by the heterogeneous multimodal sensing unit 104 to reduce the cloud load and network transmission pressure.

[0077] Cloud Collaborative Server 106: As the core processing center of the system, it can consist of one or more high-performance servers deployed in a cloud computing data center. It is responsible for receiving and aggregating data uploaded from all edge nodes and executing complex collaborative computing and fusion algorithms.

[0078] Furthermore, the edge computing node 105 runs a lightweight deep learning target detection model on the video data, outputs the bounding box coordinates of the UAV in the image coordinate system, performs short-time Fourier transform and radio frequency fingerprint feature extraction on the spectrum data, identifies the UAV signal and estimates the direction of arrival.

[0079] Furthermore, the cloud-based collaborative server 106 performs multi-source data fusion, 3D positioning, and trajectory generation across light poles, including:

[0080] Perform time synchronization and spatial registration of data from multiple nodes;

[0081] When several video acquisition modules 1041 simultaneously identify the target, the three-dimensional position of the UAV is calculated using a multi-view visual geometric positioning algorithm;

[0082] When several spectrum monitoring modules 1042 detect a signal, the position of the UAV is estimated by the signal arrival time difference or the cross-positioning of the incoming wave direction.

[0083] By employing Kalman filtering or particle filtering algorithms and fusing visual positioning results with spectral positioning results, a continuous and smooth motion trajectory is generated.

[0084] This embodiment also provides a drone tracking method based on a smart light pole network, such as... Figure 3 ,include:

[0085] The heterogeneous multimodal sensing units 104 deployed on several smart light poles 100 synchronously collect video data and spectrum data of the monitoring area;

[0086] The video data and the spectrum data are processed by the edge computing node 105 to identify the drone target and extract feature information.

[0087] The processed recognition results are uploaded to the cloud collaborative server 106;

[0088] Based on the cloud-based collaborative server 106, multi-source data is fused, and the continuous three-dimensional position of the UAV is calculated by combining visual geometric positioning and spectrum cross-positioning.

[0089] The drone's trajectory is generated based on the continuous three-dimensional position, and tracking and prediction are performed.

[0090] Specifically, it includes:

[0091] Collaborative sensing: After the system is started, the heterogeneous multimodal sensing units 104 on multiple smart light poles 100 in the monitoring area are simultaneously turned on by the central dispatch command or event trigger. The video acquisition module 1041 begins to capture high-definition video streams, and the spectrum monitoring module 1042 begins to scan and record radio signals in the 2.4GHz and 5.8GHz frequency bands.

[0092] Front-end recognition and processing: This step is executed in parallel on edge computing node 105, processing video and spectrum data respectively.

[0093] Video Recognition and Processing: Edge computing node 105 runs a lightweight deep learning object detection model (e.g., a model cropped and optimized based on the YOLOv5s architecture). This model has been trained on a dataset containing images of drones of various models, poses, and lighting conditions. The model performs real-time inference on the video stream, outputting the bounding box pixel coordinates and confidence score of the drone target in each frame. These coordinates represent the precise location of the target in the two-dimensional image plane.

[0094] Spectrum Identification and Processing: Edge computing node 105 processes the raw IQ data collected by spectrum monitoring module 1042.

[0095] Signal detection: Calculate the power spectral density of the signal. When the signal energy in a certain frequency band continuously exceeds the threshold adaptive to the ambient noise, it is determined that there is a suspicious signal.

[0096] Feature extraction and recognition: A spectrogram is generated for suspicious signals, and its radio frequency fingerprint features (such as bandwidth, center frequency, and modulation characteristics) are extracted. Subsequently, the spectrogram features are input into a pre-trained convolutional neural network (CNN) classifier for recognition. This classifier can distinguish drone signals from background interference signals such as Wi-Fi and Bluetooth.

[0097] Direction estimation: If the spectrum monitoring module 1042 integrates a directional antenna array, the direction of arrival (DOA) of the UAV signal is estimated by the MUSIC algorithm or the difference in direction of arrival (TDOA) technology.

[0098] Furthermore, the edge computing nodes employ a lightweight deep learning object detection model. As a preferred embodiment, this model is built upon a one-stage detector architecture, such as an improved YOLO (You Only Look Once) network. The model undergoes pruning and quantization to adapt to the computing power limitations of the edge computing nodes, enabling real-time inference on video frames. To improve the detection performance of small targets at long distances (drones), this embodiment employs a multi-scale feature pyramid (FPN+PAN) structure.

[0099] Through extensive training, the neural network learns the unique geometric texture features of drones (such as X-shaped frame, high-speed rotational blurring of rotors, hovering attitude, etc.). The network inference process is essentially about finding the optimal match between image regions and these features.

[0100] To quantify this degree of matching, the total loss function defined in this embodiment is... It consists of three parts:

[0101] ;

[0102] Among them, positioning loss CIoU (Complete Intersection over Union) Loss is used to measure the overlap between the predicted bounding box and the ground truth bounding box, as well as the distance between their center points, to ensure accurate coordinate output.

[0103] ;

[0104] In the formula, , , These are the balance weight coefficients for localization, confidence, and classification loss, respectively, set according to the hyperparameters during model training; Objectness Loss is used to measure the probability that the predicted bounding box contains an object. Classification Loss is used to measure the accuracy of identifying target categories (such as drone models). Distance from the center point; The coordinates of the center point of the prediction box; The coordinates of the center point of the true bounding box; The length of the diagonal of the minimum bounding rectangle; These are weighting parameters used to balance the influence of aspect ratio; Aspect ratio; Intersection over Union (IUCN) is the ratio of the area of ​​overlap between the predicted bounding box and the ground truth bounding box to the area of ​​their union.

[0105] This loss function forces the predicted bounding box to be calculated not only in terms of the overlapping area, but also in terms of the distance from the center point. and aspect ratio The above parameters are designed to approximate the actual outline of the drone as closely as possible. By minimizing this function, the drone can be accurately distinguished from interference objects such as birds and kites, and its pixel coordinates can be output. .

[0106] Furthermore, the feature analysis of the spectral data by the edge computing node 105 includes:

[0107] The time-domain signal is converted into a time-frequency graph using short-time Fourier transform;

[0108] The radio frequency fingerprint features of the signal are extracted through the time-frequency diagram, the UAV signal is identified based on the convolutional neural network classifier, and the direction of arrival of the signal is estimated based on the MUSIC algorithm or the direction-of-arrival difference method using a multi-antenna array.

[0109] Specifically, for the original time-domain signal x[n] acquired by the spectrum monitoring module 1042, the edge nodes first remove the DC component and Gaussian white noise, and then use short-time Fourier transform (STFT) to convert it into a time-frequency graph (Spectrogram).

[0110] The STFT transformation formula is defined as follows:

[0111] ;

[0112] In the formula, The transformed time-frequency complex matrix has a squared modulus. Representative spectrogram; The input is the original discrete-time domain signal sequence; These are complex exponential basis functions used to map time-domain signals to the frequency domain; For A sliding window function centered on the signal is used to extract a local portion of the signal. This represents the time shift, indicating the time axis of the frequency plot. Angular frequency represents the frequency axis of the time-frequency graph; It is a discrete-time index (TimeIndex) used to traverse signal sequences; imaginary unit .

[0113] The recognition principle of this algorithm is as follows: Drone communication (image transmission / remote control) usually adopts frequency hopping spread spectrum (FHSS) technology, that is, the signal frequency will change rapidly over time, while ordinary Wi-Fi signals are usually fixed on a certain channel.

[0114] Through the aforementioned STFT transformation, the system unfolds the one-dimensional time-domain signal into a two-dimensional time-frequency spectrum. In this spectrum, the drone's signal exhibits a unique discrete short-line or diagonal-line texture (i.e., frequency-hopping pattern), while background noise manifests as low-level noise, and Wi-Fi appears as a continuous long band. By extracting these texture features (center frequency, bandwidth, frequency-hopping period), the system can "hear" and lock onto the drone in a complex electromagnetic environment.

[0115] Furthermore, the edge computing node 105 packages and uploads the processed structured data (for video: timestamp, light pole ID, bounding box coordinates, confidence level; for spectrum: timestamp, light pole ID, signal type, DOA angle, signal strength) to the cloud collaborative server 106 via the network layer 102.

[0116] Cloud fusion and precise positioning are the core steps for achieving high-precision and robust tracking in this embodiment, and specifically include:

[0117] Data association and registration: The cloud-based collaborative server 106 first performs time synchronization and spatial coordinate system integration on data from different light poles and different modes.

[0118] Visual geometric positioning: When two or more cameras simultaneously detect the same drone, the coordinates of each two-dimensional image are back-projected onto the world coordinate system using the principle of multi-view visual stereo geometry. The precise three-dimensional position (X, Y, Z) of the drone is then calculated using triangulation. This position has the highest accuracy and serves as the "reference" for positioning.

[0119] Spectrum cross-location: When two or more spectrum modules detect the same signal, they cross-locate using their respective directions of arrival (DOA) to form an azimuth sector, or use the signal strength attenuation model (RSSI) for rough ranging, and jointly estimate the probability area where a target exists.

[0120] A Kalman filter or particle filter algorithm is used to deeply fuse the visual positioning results (high accuracy but discontinuous) with the spectral positioning results (continuous but low accuracy). The filter's state vector contains the UAV's position and velocity. When visual positioning results are available, they are used to update and correct the state vector; when the visual target is briefly lost (e.g., due to occlusion), the motion trend provided by the spectral data is used for prediction, ensuring that the system can output a continuous and smooth positioning sequence. This results in the output of continuous 3D position points.

[0121] Furthermore, the spectrum cross-positioning uses the TDOA algorithm, which iteratively solves the distance difference equation using the Chan algorithm or Taylor series expansion to obtain the approximate coordinates of the UAV.

[0122] Specifically, when the spectrum monitoring module (1042) detects the UAV signal, the time of signal reception varies slightly due to the different locations of the light poles (Time Difference of Arrival, TDOA). The system uses this difference to construct a hyperbolic equation system for coordinate calculation.

[0123] Let the spatial coordinates of the UAV (target) be... , No. The known coordinates of the lampposts are: , No. The coordinates of the lampposts are Assume the signal propagation speed is the speed of light. lamppost and lamppost The time difference of the received signal is The distance difference equation is then:

[0124] ;

[0125] Expand into spatial coordinate form:

[0126] ;

[0127] In the formula, Let the speed of electromagnetic waves in air be the speed of light. m / s); The signal measured by the spectrum monitoring module reaches the first The lamppost and the first Time difference of each light pole ; The difference in distance from the target to the two lampposts; To achieve the goal of The Euclidean distance between each lamp post; To achieve the goal of The Euclidean distance between each lamppost.

[0128] By combining data from at least three light poles (forming more than three equations), and iteratively solving using the Chan algorithm or Taylor series expansion, the approximate coordinates of the target can be calculated. It is used to guide the rotation of the camera's pan-tilt unit.

[0129] Furthermore, the visual geometric localization includes:

[0130] Based on the intrinsic and extrinsic parameters of several cameras, an overdetermined linear equation system is constructed.

[0131] The three-dimensional coordinates of the UAV are obtained by solving the overdetermined linear equations through singular value decomposition.

[0132] Specifically, this embodiment employs a method combining linear triangulation and singular value decomposition (SVD) to address the problem of multiple camera lines of sight not intersecting in three-dimensional space due to noise. The specific calculation steps are as follows:

[0133] Constructing a pinhole camera projection model:

[0134] For any camera on a smart light pole Let the pixel coordinates of the observed UAV on the image plane be... Based on the pinhole camera model, the three-dimensional points of the UAV in the world coordinate system. With pixel coordinates The mapping relationship is as follows:

[0135] ;

[0136] In the formula, , where is the scale factor, representing the distance projection from the target to the camera's optical center; For drones in the Pixel coordinates (observations) on the image plane of a camera; The three-dimensional coordinates of the UAV to be determined in the world coordinate system; The extrinsic parameter matrix of the camera, including the rotation matrix. Translation vector The angle of the pan-tilt unit is determined by the installation location of the light pole and the angle of the pan-tilt unit. The intrinsic parameter matrix of the camera:

[0137] ;

[0138] In the formula, For the camera Equivalent focal length (in pixels) along the axial direction; For the camera Equivalent focal length (in pixels) along the axial direction; , These are the pixel coordinates of the principal point (center of the optical axis) of the image.

[0139] Order No. Projection matrix of each camera Represent it as a row vector:

[0140] ;

[0141] In the formula, , , Each is a matrix The first, second, and third row vectors.

[0142] Constructing an overdetermined system of linear equations:

[0143] Eliminating unknown scale factors using the cross product property For each camera Construct two linear constraint equations:

[0144] ;

[0145] ;

[0146] in, Let be the three-dimensional homogeneous coordinate vector of the UAV to be solved. When the system has One lamppost When the target is observed simultaneously, the constraint equations for all cameras are solved concurrently, resulting in the following form: An overdetermined system of linear equations, where matrix A is:

[0147] ;

[0148] In the formula, For the first camera projection matrix The first row vector; For the first camera projection matrix The second row vector; For the first camera projection matrix The third row vector.

[0149] Optimal solution (DLT algorithm):

[0150] Perform singular value decomposition (SVD) on matrix A: .

[0151] The optimal three-dimensional homogeneous coordinate solution of the UAV is The column vectors corresponding to the minimum singular values ​​in the matrix are then converted to Euclidean coordinates:

[0152] ;

[0153] In the formula, For the final calculated 3D Euclidean coordinates of the UAV, The first three components of the homogeneous coordinate vector obtained by SVD. The fourth component (normalization factor) of the homogeneous coordinate vector obtained by SVD.

[0154] To address the jitter and momentary occlusion issues in visual positioning, Kalman filtering is used to address the aforementioned problems. Perform the optimal estimate.

[0155] State equations (predictive models):

[0156] Establish system state vector Includes position and velocity:

[0157] ;

[0158] In the formula, for The system state vector at time t. The three-dimensional position coordinates of the drone. The velocity components of the UAV in the three axes.

[0159] Predict the state at the next moment based on the uniform motion model (CV model):

[0160] ;

[0161] In the formula, Here is the state transition matrix:

[0162] ;

[0163] In the formula, for Prior state estimate (predicted value) at time 1; for The posterior state estimate at time step (the optimal value at the previous time step). This is the process noise covariance matrix, used to describe the uncertainty of the motion model; for The identity matrix; The sampling time interval (i.e., the time difference between two consecutive frames of data); for A zero matrix (where all elements are 0) indicates that position changes have no direct feedback effect on velocity.

[0164] Observation equations and adaptive updates:

[0165] Observation equations Defined as:

[0166] ;

[0167] In the formula, This is the observation matrix, used to map the state space to the observation space; for The system state vector at time t; To observe the noise covariance matrix, the system adaptively adjusts the value of the matrix based on whether the vision is locked.

[0168] Calculate Kalman gain :

[0169] ;

[0170] In the formula, For the prediction error covariance matrix, Observation matrix The transpose of the matrix;

[0171] Update the state estimate and output the final smoothed trajectory points:

[0172] ;

[0173] In the formula, for The posterior state estimate at time (the smoothed position of the final output). for Information about time The predicted state estimate at time step 1 is given by the superscript "^" indicating the estimated value and the subscript "^". "Indicates using up to Estimating based on data at time points time.

[0174] Trajectory Generation and Tracking:

[0175] The cloud-based collaborative server 106 connects continuous three-dimensional location points in chronological order to form the target's motion trajectory. Simultaneously, the system can predict its possible location at the next moment based on its current motion state (speed, direction), enabling active tracking, and can output the final trajectory information to a monitoring screen or alarm system.

[0176] Through the above-described hardware and software combined implementation method, this embodiment successfully transforms the smart light pole network into a powerful low-altitude sensing platform. Its core advantages are:

[0177] Through heterogeneous multimodal fusion, a perception effect of "1+1>2" is achieved. Vision and spectrum are deeply complementary at the algorithm level, ensuring the reliability of the system in various environments.

[0178] Through an edge-cloud collaborative computing architecture, computing power is rationally allocated, achieving a balance between low latency and high efficiency;

[0179] By utilizing existing, grid-like smart light poles, wide-area coverage that previously required huge investments has been achieved at extremely low marginal costs, making it highly valuable for commercial promotion.

[0180] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A drone tracking system based on a smart light pole network, characterized in that, include: Perception layer: Deployed on several smart light poles (100) to acquire perception data; Network layer: used to transmit the sensed data; Platform layer: Used to receive the sensed data and perform intelligent processing and decision-making; The perception layer, the network layer, and the platform layer are connected in sequence. The perception layer is composed of heterogeneous multimodal perception units (104) deployed on several smart light poles (100), and the smart light poles (100) are the deployment carriers of the system.

2. The drone tracking system based on a smart light pole network according to claim 1, characterized in that, The heterogeneous multimodal sensing unit (104) includes a video acquisition module (1041) and a spectrum monitoring module (1042). The video acquisition module (1041) is a high-definition network camera with day / night switching function; the spectrum monitoring module (1042) is a software-defined wireless device with a working frequency band covering 2.4GHz and 5.8GHz, and the spectrum monitoring module (1042) only receives communication signals from the drone and its remote controller.

3. The drone tracking system based on a smart light pole network according to claim 2, characterized in that, The platform layer includes edge computing nodes (105) and cloud collaboration servers (106). The edge computing node (105) is embedded in the controller of the smart light pole (100) and is used to perform front-end preprocessing and local fusion recognition on the data collected by the heterogeneous multimodal sensing unit (104). The cloud-based collaborative server (106) communicates with all edge computing nodes (105) via TCP / IP protocol and is used to perform multi-source data fusion, three-dimensional positioning and trajectory generation across light poles.

4. The drone tracking system based on a smart light pole network according to claim 3, characterized in that, The edge computing node (105) runs a lightweight deep learning target detection model on the video data, outputs the bounding box coordinates of the UAV in the image coordinate system, performs short-time Fourier transform and radio frequency fingerprint feature extraction on the spectrum data, identifies the UAV signal and estimates the direction of arrival.

5. The drone tracking system based on a smart light pole network according to claim 3, characterized in that, The cloud-based collaborative server (106) performs multi-source data fusion, 3D positioning, and trajectory generation across light poles, including: Perform time synchronization and spatial registration of data from multiple nodes; When several video acquisition modules (1041) simultaneously identify the target, the three-dimensional position of the UAV is calculated by a multi-view visual geometric positioning algorithm; When several spectrum monitoring modules (1042) detect a signal, the position of the UAV is estimated by the signal arrival time difference or the cross-positioning of the incoming wave direction; By employing Kalman filtering or particle filtering algorithms and fusing visual positioning results with spectral positioning results, a continuous and smooth motion trajectory is generated.

6. A drone tracking method based on a smart light pole network, used to implement the drone tracking system based on a smart light pole network as described in any one of claims 1-5, characterized in that, include: Video data and spectrum data of the monitoring area are collected synchronously by heterogeneous multimodal sensing units (104) deployed on several smart light poles (100); The video data and the spectrum data are processed by the edge computing node (105) respectively to identify the drone target and extract feature information; The processed recognition results are uploaded to the cloud collaborative server (106). Based on the cloud-based collaborative server (106), multi-source data is fused, and the continuous three-dimensional position of the UAV is calculated by combining visual geometric positioning and spectrum cross-positioning. The drone's trajectory is generated based on the continuous three-dimensional position, and tracking and prediction are performed.

7. The drone tracking method based on a smart light pole network according to claim 6, characterized in that, The edge computing node (105) runs a lightweight deep learning target detection model on the video data and outputs the bounding box coordinates of the UAV in the image coordinate system; wherein, the lightweight deep learning detection model is a model optimized based on YOLO or SSD architecture and trained for UAV targets.

8. The drone tracking method based on a smart light pole network according to claim 6, characterized in that, The feature analysis of the spectrum data by the edge computing node (105) includes: The time-domain signal is converted into a time-frequency graph using short-time Fourier transform; The radio frequency fingerprint features of the signal are extracted through the time-frequency diagram, the UAV signal is identified based on the convolutional neural network classifier, and the direction of arrival of the signal is estimated based on the MUSIC algorithm or the direction-of-arrival difference method using a multi-antenna array.

9. The drone tracking method based on a smart light pole network according to claim 6, characterized in that, The spectrum cross-positioning uses the TDOA algorithm, which iteratively solves the distance difference equation using the Chan algorithm or Taylor series expansion to obtain the approximate coordinates of the UAV.

10. The drone tracking method based on a smart light pole network according to claim 6, characterized in that, The visual geometric localization includes: Based on the intrinsic and extrinsic parameters of several cameras, an overdetermined linear equation system is constructed. The three-dimensional coordinates of the UAV are obtained by solving the overdetermined linear equations through singular value decomposition.

Images