Unmanned aerial vehicle tracking method, electronic device, and storage medium

Abstract

This application relates to the field of unmanned aerial vehicle (UAV) technology, disclosing a UAV tracking method, electronic device, and storage medium. The method includes: acquiring target observation data in a target airspace, where at least one target to be tracked exists; processing the target observation data into a synchronized spatiotemporal snapshot sequence based on a preset spatiotemporal reference; projecting the spatiotemporal snapshot sequence onto a preset global representation space, and performing temporal fusion on the multimodal feature maps obtained from the projection to determine an enhanced spatiotemporal feature map; and tracking the detected target to be tracked based on the enhanced spatiotemporal feature map to generate a tracking trajectory for the target to be tracked. This application performs pre-fusion of observation data based on a global representation space, preserving the original information of each sensor to the greatest extent and effectively improving the accuracy of UAV tracking.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a UAV tracking method, electronic device, and storage medium. Background Technology

[0002] Unmanned aerial vehicles (UAVs) are characterized by complex multimodal features, agile movement, and severe environmental interference, making their perception and tracking tasks extremely challenging. For example, UAVs typically have small radar cross-sections, weak infrared signatures, variable radio signals, and small visual dimensions, resulting in blind spots for any single sensor. They can perform complex nonlinear movements such as hovering and sharp turns, requiring high target recognition and trajectory prediction capabilities from tracking algorithms. At the same time, electromagnetic interference, false alarm targets, and physical obstructions in complex urban or near-ground environments severely affect the reliability of perception and the continuity of tracking.

[0003] In related technologies, drone tracking technology typically employs multi-source heterogeneous sensor fusion schemes. For example, it involves post-fusion of radar, photoelectric, and radio frequency data through spatiotemporal registration and extended Kalman filtering, or deep optimization of single-modal visual detection and tracking algorithms. Additionally, there are methods that preprocess data from different sensors into image formats and then perform feature extraction and classification using independent deep neural networks, as well as algorithmic pipelines involving multimodal data sequence fusion and pose estimation.

[0004] However, most of the above-mentioned solutions rely on post-fusion of sensor information, which has inherent limitations such as severe information loss, poor environmental adaptability, and insufficient tracking continuity, resulting in low accuracy in UAV tracking. Specifically, post-fusion architectures struggle to form a comprehensive and consistent target perception when processing asynchronous and sparse data; systems that rely on fixed rules or prior knowledge are ill-suited to complex and changing environments and have weak ability to identify unknown or camouflaged behaviors; furthermore, when the target is briefly occluded or the sensor data quality deteriorates, target loss or identity switching can easily occur, making it impossible to guarantee trajectory continuity. Summary of the Invention

[0005] One objective of this application is to provide a drone tracking method, electronic device, and storage medium to solve the technical problem of low accuracy in drone tracking in related technologies.

[0006] This application provides a drone tracking method in a first aspect, including: Acquire target observation data in a target airspace, where at least one target to be tracked exists; Based on a preset spatiotemporal reference, the target observation data is processed into a synchronized spatiotemporal snapshot sequence; The spatiotemporal snapshot sequence is projected onto a preset global representation space, and the multimodal feature maps obtained by the projection are temporally fused to determine the enhanced spatiotemporal feature maps; Based on the enhanced spatiotemporal feature map, the detected target is tracked, and the tracking trajectory of the target is generated.

[0007] In conjunction with the first aspect, in one possible implementation, processing the target observation data into a synchronized spatiotemporal snapshot sequence based on a preset spatiotemporal reference includes: Based on a preset spatiotemporal benchmark, a time anchor grid is constructed, which consists of multiple time anchors. A time assignment window is defined with the time corresponding to each time anchor as the center, and the target observation data within each time assignment window is aggregated to the corresponding time anchor. Through spatiotemporal correlation analysis, the target observation data at the same time anchor is aggregated to obtain the spatiotemporal snapshot sequence.

[0008] In conjunction with the first aspect, in one possible implementation, when aggregating target observation data within each time assignment window to the corresponding time anchor point, if the amount of observation data within the time assignment window is less than a preset threshold, the time assignment window is expanded to obtain observation data outside the time assignment window.

[0009] In conjunction with the first aspect, in one possible implementation, the global representation space is a world model constructed using a bird's-eye view space or voxel space, the spatiotemporal snapshot sequence includes radar point cloud data, spectrum direction finding data, and photoelectric image data, and the projection of the spatiotemporal snapshot sequence onto the preset global representation space includes: The radar point cloud data is projected onto the world model to generate feature channels containing target motion attributes at the corresponding locations; the spectrum direction finding data is projected onto the world model to generate signal features in the corresponding azimuth region; the photoelectric image data is projected onto the world model to generate visual features in the corresponding region; and the multiple features obtained by projection are stitched together in the channel dimension to form a multimodal feature map.

[0010] In conjunction with the first aspect, one possible implementation involves temporal fusion of the multimodal feature maps obtained from projection, including: Multimodal feature maps at multiple consecutive time points are input into a preset temporal fusion network. The temporal fusion network captures the dependencies of the multimodal feature maps in the time dimension and outputs the enhanced spatiotemporal feature map.

[0011] In conjunction with the first aspect, in one possible implementation, the step of tracking the detected target based on the enhanced spatiotemporal feature map and generating a tracking trajectory for the target includes: A target agent is instantiated for the detected target to be tracked, and the target agent is used to track the target. The target agent obtains feature information by querying the enhanced spatiotemporal feature map and updates its own state based on the feature information to output the tracking trajectory of the target.

[0012] In conjunction with the first aspect, in one possible implementation, each of the target agents maintains at least one set of learnable query vectors; The target agent obtains feature information by querying the enhanced spatiotemporal feature map, including: The target agent sends its maintained query vector to the enhanced spatiotemporal feature map, and queries and obtains the feature information most relevant to its own state through an attention mechanism.

[0013] In conjunction with the first aspect, in one possible implementation, the target agent maintains internal memory, which is used to store historical feature information of the target to be tracked; When the target agent fails to find valid feature information from the enhanced spatiotemporal feature map within a first preset time, it performs state deduction based on its internal memory. When the feature information of the target to be tracked is reproduced in the enhanced spatiotemporal feature map, and the similarity between the reproduced feature information and the historical feature information exceeds a preset threshold, the target agent re-associates the reproduced feature information and resumes the tracking process of the target to be tracked. If the target agent fails to re-associate the reproduced feature information within a second preset time, the internal memory of the target agent gradually decays over time. When the internal memory decays to a preset extinction threshold, the target agent is terminated.

[0014] This application provides an electronic device in a second aspect, comprising: At least one processor; and a memory communicatively connected to at least one of the processors; The memory stores instructions that can be executed by the processor to implement the method described in the first aspect.

[0015] In a third aspect, this application provides a computer-readable storage medium storing computer instructions that are executed by the computer to implement the method described in the first aspect.

[0016] This application can achieve the following technical effects: This application discloses a UAV tracking method, comprising: acquiring target observation data and processing the target observation data into a synchronous spatiotemporal snapshot sequence; projecting the spatiotemporal snapshot sequence onto a preset global representation space and performing temporal fusion on the multimodal feature maps obtained by the projection to determine an enhanced spatiotemporal feature map; instantiating a target agent for the detected target to be tracked, so as to use the target agent to track the target, wherein the target agent queries and obtains feature information according to the enhanced spatiotemporal feature map, and updates its own state based on the feature information to output the tracking trajectory of the target to be tracked.

[0017] Based on the method proposed in this application, the original information of each sensor is preserved to the greatest extent by pre-fusion based on the global representation space. Even when a single sensor (such as photoelectric) fails due to weather or obstruction, it can still rely on the information of other sensors (such as radar and spectrum) in the unified model to maintain stable perception of the target, which greatly improves the adaptability to all weather and all scenarios.

[0018] Meanwhile, this application further introduces a target agent based on memory and active query, which can actively predict, verify and maintain the target's tracking trajectory. When the target is temporarily lost or the sensor data quality deteriorates, the agent relies on its internal memory to retain the track and re-associate based on feature similarity when the target reappears. This effectively solves the common ID jump and target loss problems in traditional tracking algorithms and outputs a more reliable tracking trajectory. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A schematic diagram of a drone tracking system provided in an embodiment of this application; Figure 2 This is a flowchart illustrating a drone tracking method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a drone tracking device provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0022] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. Moreover, the terms "first," "second," and "third" used in this application do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0023] Please see Figure 1 , Figure 1 The diagram shows a drone tracking system consisting of a sensor unit 10, an electronic device 20, and an output device 30, for tracking a drone 40 in a scene. The electronic device 20 is communicatively connected to both the sensor unit 10 and the output device 30. The sensor unit 10 is used to collect sensor data from the drone 40. The electronic device 20 tracks the drone 40 based on the sensor data and transmits the tracking results to the output device 30, which then outputs the tracking results.

[0024] Sensor unit 10 can be deployed at high points in the monitoring area (such as rooftops or towers). Generally speaking, for drone tracking scenarios, the sensor unit mainly includes three types of core sensors: Radar sensors typically employ three-dimensional phased array radar or mechanically scanned radar, continuously emitting electromagnetic waves and receiving target echoes. For example, they can be connected to electronic devices 20 via an Ethernet interface to transmit point cloud data in real time using protocols such as UDP. The data packets contain parameters such as the distance, azimuth, elevation, radial velocity, and radar cross section (RCS) of each detected target. A spectrum sensor, also known as a radio spectrum monitoring device, can be equipped with a directional antenna or direction finding array to capture communication signals between the drone and the remote controller in real time. For example, a spectrum device can communicate with electronic device 20 via fiber optic or gigabit network cable and can transmit IQ data and direction finding information via protocols such as UDP, including parameters such as signal frequency, bandwidth, field strength, and angle of arrival (DOA). Optoelectronic sensors typically employ a visible light / infrared dual-light turntable system, integrating a high-resolution camera and a servo pan-tilt unit. For example, optoelectronic devices can push video streams encoded with H.264 / H.265 via protocols such as RTSP, while simultaneously transmitting attitude information such as the pan-tilt unit's azimuth, pitch, and field of view via serial port or UDP.

[0025] Optionally, all types of sensors in sensor unit 10 can be connected to an industrial-grade Ethernet switch. This switch has VLAN isolation and QoS functions to ensure the transmission priority and bandwidth guarantee of different types of data. In this scenario, all sensors synchronize with the time server via the PTP (IEEE 1588) protocol to ensure that the data carries an accurate hardware timestamp.

[0026] As the core processing platform of the drone tracking system, electronic device 20 can be deployed in the monitoring station's computer room, employing a high-performance computing server architecture to achieve the necessary data processing capabilities. Generally, it can be equipped with a multi-core CPU (such as the Intel Xeon series) for data scheduling and preprocessing, and multiple high-performance GPUs (such as the NVIDIA RTX / A100 series) for deep learning inference. In addition, electronic device 20 also requires a certain amount of data storage capacity to cache and back up system operating data.

[0027] For output device 30, it is equipped with at least a display and control terminal and a countermeasure device interface. It is connected to a splicing screen or workstation monitor via HDMI / DVI to display the drone tracking trajectory in real time. At the same time, it is connected to jammers, decoy devices, etc. via the countermeasure device interface (RS422 / RS485 serial port, fiber optic or network relay interface) to receive tracking commands.

[0028] Further, see Figure 2 , Figure 2 This is a flowchart illustrating a drone tracking method provided in an embodiment of this application, applied to... Figure 1 The electronic device 20 shown herein includes the following steps: Step S10: Obtain target observation data in the target airspace, wherein at least one target to be tracked exists in the target airspace; Step S20: Based on a preset spatiotemporal reference, the target observation data is processed into a synchronized spatiotemporal snapshot sequence; Step S30: Project the spatiotemporal snapshot sequence onto a preset global representation space, and perform temporal fusion on the multimodal feature maps obtained by the projection to determine the enhanced spatiotemporal feature maps; Step S40: Track the detected target based on the enhanced spatiotemporal feature map and generate the tracking trajectory of the target.

[0029] Specifically, this embodiment still follows the above. Figure 1 Taking the drone tracking system shown as an example, after the drone tracking system is started, the radar, spectrum monitoring equipment and photoelectric turntable installed in the sensor unit 10 begin to continuously collect sensor data (radar data, spectrum data and photoelectric data) in the target airspace as target observation data, and transmit it to the electronic device 20 in real time.

[0030] For radar, spectrum monitoring equipment, and optoelectronic turntables, each collects and transmits sensor data, for example: The radar generates a frame of point cloud data at a certain frequency (e.g., every 200ms) and sends it to port 5001 of the server via UDP protocol. Each frame of data contains multiple target points, and each target point carries a timestamp (accurate to microseconds), range R, azimuth angle θ, elevation angle φ, radial velocity v_r, and RCS value σ.

[0031] The spectrum monitoring equipment generates a frame of direction-finding data at a certain frequency (e.g., every 100ms) and sends it to port 5002 of the server via UDP protocol. Each frame of data contains multiple signal sources, and each signal source carries a timestamp, center frequency f, bandwidth B, field strength P, and angle of arrival α.

[0032] The photoelectric turntable continuously outputs a video stream at a certain frequency (e.g., 25fps), which is pushed to the server via the RTSP protocol. At the same time, it sends the gimbal attitude data (timestamp, azimuth, pitch, and field of view) via UDP.

[0033] Since the above three types of data are asynchronous in time and their spatial coordinate systems are not uniform (generally, radar uses polar coordinates, spectrum uses direction-finding coordinates, and photoelectric uses image pixel coordinates), further synchronization processing is required to obtain a synchronized spatiotemporal snapshot sequence.

[0034] For example, in this embodiment, each snapshot of the spatiotemporal snapshot sequence includes: Timestamp t_k; Scene ID (e.g., S_20250301_143000_001); Radar observation set: {(x_i,y_i,z_i,v_r_i,σ_i)}; Spectrum observation set: {(f_j,B_j,P_j,direction vector_j)}; Photoelectric image frame I_k.

[0035] In this embodiment, the preset spatiotemporal reference includes a pre-set time reference and a spatial reference.

[0036] For the time reference, all sensor data carries a hardware timestamp based on the PTP protocol for synchronization. Therefore, the global time can be based on the BeiDou / GPS time server, with a synchronization accuracy better than 1 microsecond.

[0037] For spatial reference, the ENU (East-North-Sky) Cartesian coordinate system with the radar station as the origin can be defined as a unified spatial coordinate system, and all sensor data will be converted to this coordinate system in the future.

[0038] As an optional implementation, the step of processing the target observation data into a synchronized spatiotemporal snapshot sequence based on a preset spatiotemporal reference includes: Based on a preset spatiotemporal benchmark, a time anchor grid is constructed, which consists of multiple time anchors. A time assignment window is defined with the time corresponding to each time anchor as the center, and the target observation data within each time assignment window is aggregated to the corresponding time anchor. Through spatiotemporal correlation analysis, the target observation data at the same time anchor is aggregated to obtain the spatiotemporal snapshot sequence.

[0039] In this embodiment, it is assumed that the system output frame rate is 20Hz, that is, a spatiotemporal snapshot is generated every 50ms. At this time, a time anchor grid is constructed throughout the entire tracking process, and the anchor time is t_k = t_0 + k×50ms (k=0,1,2,...).

[0040] For each time anchor point t_k, define a time assignment window [-25ms, +25ms] centered at t_k, and iterate through all observation data appearing within this window: For radar point cloud data, radar point cloud frames within a window can be merged, with each point carrying an original timestamp t_i, and its time offset relative to the anchor point Δt = |t_i - t_k| can be calculated as the basis for subsequent validity measurement. For spectrum direction finding data, spectrum frames within a window can also be merged, with each signal source carrying a timestamp and angle of arrival. For photoelectric image data, the frame with the timestamp closest to t_k within the window can be selected as the representative image of that anchor point.

[0041] Furthermore, when aggregating target observation data within each time assignment window to the corresponding time anchor point, if the amount of observation data within the time assignment window is less than a preset threshold, the time assignment window is expanded to obtain observation data outside the time assignment window.

[0042] In this embodiment, if the amount of a certain type of data in the window is insufficient (e.g., no data due to sensor failure or lack of target), the window range can be further expanded to [-50ms, +50ms] to continue trying to aggregate until the preset maximum window range is reached.

[0043] As an optional implementation, all aggregated observation data is transformed to the ENU coordinate system: For radar point clouds, polar coordinates (R,θ,φ) are converted to Cartesian coordinates (x,y,z); for spectral direction finding lines, the direction vector of the direction finding line can be determined in the ENU coordinate system using the angle of arrival α and signal parameters; and for photoelectric images, the mapping relationship between image pixel coordinates and ENU coordinates can be established by combining gimbal attitude data to prepare for subsequent projection.

[0044] For multi-target observation data belonging to the same time anchor point, scene-based association is performed through temporal overlap and spatial proximity analysis to confirm whether they point to the same physical target, and finally a synchronous spatiotemporal snapshot sequence with a unique scene ID is generated.

[0045] This embodiment predefines a global representation space, which can be represented in the form of a world model. That is, after determining the synchronous spatiotemporal snapshot sequence, this embodiment projects the features of different modes such as radar, spectrum, and photoelectric into a world model represented from a global perspective.

[0046] As an alternative implementation, a world model can be formed in the form of a bird's-eye-view (BEV) or voxel by rasterizing a uniform spatial coordinate system.

[0047] For example, a 200m × 200m bird's-eye view (BEV) grid plane can be created centered on the monitoring site, with a grid resolution of 0.5m × 0.5m, resulting in a total of 400 × 400 grid cells. Each grid cell encodes the characteristic information of that location.

[0048] As an optional implementation, projecting the spatiotemporal snapshot sequence onto a preset global representation space includes: The radar point cloud data is projected onto the global representation space to generate feature channels containing target motion attributes at the corresponding locations. The spectral direction finding data is projected onto the global representation space to generate signal features in the corresponding azimuth region; The photoelectric image data is projected onto the global representation space to generate visual features in the corresponding region; The various features obtained from projection are stitched together along the channel dimension to form a multimodal feature map.

[0049] In this embodiment, for each spatiotemporal snapshot at time anchor point t_k, radar point cloud data, spectrum direction finding data, and photoelectric image data are projected onto the BEV space respectively: Specifically, for radar point cloud data, each radar point in the snapshot is traversed, and the grid cell into which each radar point falls is determined based on the coordinates of each radar point.

[0050] More specifically, for each grid cell, the following information can be collected to form an 8-channel feature map: Channel 1: The number of point clouds falling into this grid; Channel 2: Average height z of all points; Channel 3: Maximum radial velocity v_r_max; Channel 4: Average RCS value σ_mean; Channels 5-8: Reserve for backup or encode other statistics.

[0051] More specifically, for spectrum direction finding data, the corresponding angle sector in the BEV space can be determined by traversing each spectrum direction finding signal in the snapshot and based on its angle of arrival (for example, dividing 360 degrees into 72 sectors of 5 degrees centered on the station).

[0052] For example, a Gaussian probability distribution model can be used to render the signal energy onto each grid within the sector, generating a 4-channel feature map: Channel 1: Signal energy distribution in the 2.4 GHz band; Channel 2: Signal energy distribution in the 5.8 GHz band; Channel 3: Signal energy distribution in other frequency bands; Channel 4: Overall signal field strength distribution.

[0053] For photoelectric image data, the image frame I_k from the snapshot can be input into a lightweight convolutional neural network (such as MobileNetV3) to extract semantic features and obtain a 16-channel deep feature map. Combining the gimbal attitude and camera intrinsics, the image feature map is mapped to the BEV space through inverse perspective transformation (IPM) to generate a 16-channel visual feature map.

[0054] Finally, the three types of feature maps are stitched together in the channel dimension: radar feature map (8 channels) + spectral feature map (4 channels) + visual feature map (16 channels) to form a 28-channel multimodal feature map F_k with a size of 400×400×28.

[0055] In this embodiment, temporal fusion is performed on the multimodal feature maps obtained by projection, including: Multimodal feature maps at multiple consecutive time points are input into a preset temporal fusion network. The temporal fusion network captures the dependencies of the multimodal feature maps in the time dimension and outputs the enhanced spatiotemporal feature map.

[0056] Specifically, in this embodiment, a sliding window of fixed length is constructed and maintained, for example, storing the multimodal feature maps F_{k-4}, F_{k-3}, ..., F_k at the most recent 5 time steps, and inputting these 5 feature maps into a pre-constructed temporal fusion network.

[0057] The temporal fusion network can adopt the ConvGRU (convolutional gated recurrent unit) structure. Taking ConvGRU as an example, it includes 3 layers of convolutional GRU units and 64 hidden layer channels.

[0058] Taking ConvGRU as an example, the temporal fusion network processes its input through the following process: Each ConvGRU layer receives the feature map at the current time step and the hidden state at the previous time step, and updates the hidden state at the current time step. Through gating mechanisms, the network learns the evolution of feature maps over time, enhancing features that remain consistent over time and exhibit motion consistency (i.e., the real target), while suppressing random noise and false alarms. The hidden state output from the last layer is compressed by a convolutional layer to obtain an enhanced spatiotemporal feature map E_k with a size of 400×400×32.

[0059] The enhanced spatiotemporal feature map E_k contains deep spatiotemporal context information of all potential targets in the scene at the current moment, thus providing input for the subsequent target tracking process.

[0060] As an optional implementation, this embodiment performs target detection based on the enhanced spatiotemporal feature map E_k to determine the target to be tracked.

[0061] For example, on the enhanced spatiotemporal feature map E_k, the confidence score of each grid position as the target center is calculated (e.g., it can be calculated by a convolutional neural network, etc.), a heat map H_k is generated, peak detection is performed on H_k, and local maxima points with confidence scores exceeding a threshold (e.g., 0.5) are extracted as candidate target positions.

[0062] Based on this, non-maximum suppression (NMS) can be used to remove overlapping detection results, and finally obtain the set of detected targets D_k = {d_1, d_2, ..., d_m} for the current frame. Each target d_i contains position information (x_i, y_i) and confidence s_i, which are the targets to be tracked detected in this embodiment.

[0063] In this embodiment, the detected target to be tracked is tracked based on the enhanced spatiotemporal feature map, and a tracking trajectory of the target to be tracked is generated, including: A target agent is instantiated for the detected target to be tracked, and the target agent is used to track the target. The target agent obtains feature information by querying the enhanced spatiotemporal feature map and updates its own state based on the feature information to output the tracking trajectory of the target.

[0064] Each of the target intelligent agents maintains at least one set of learnable query vectors; The target agent obtains feature information by querying the enhanced spatiotemporal feature map, including: The target agent sends its maintained query vector to the enhanced spatiotemporal feature map, and queries and obtains the feature information most relevant to its own state through an attention mechanism.

[0065] Meanwhile, the target agent maintains internal memory, which is used to store historical feature information of the target to be tracked; When the target agent fails to find valid feature information from the enhanced spatiotemporal feature map within a first preset time, state deduction is performed based on the internal memory. When the feature information of the target to be tracked is reproduced in the enhanced spatiotemporal feature map, and the similarity between the reproduced feature information and the historical feature information exceeds a preset threshold, the target agent re-associates the reproduced feature information and resumes the tracking process of the target to be tracked. If the target agent fails to re-associate the reproduced feature information within a second preset time period, the internal memory of the target agent will gradually decay over time. When the internal memory decays to a preset extinction threshold, the target agent will be terminated.

[0066] In this embodiment, for each newly emerging target to be tracked (i.e. a target not associated with any existing agent) in the target set D_k, an independent target agent A_i is instantiated for it.

[0067] Generally speaking, each agent contains the following internal states: A unique identifier, TrackID (e.g., a continuously incrementing integer); The estimated state at the current moment: position (x, y, z), velocity (v_x, v_y, v_z), acceleration (a_x, a_y, a_z); Appearance feature vector, which is a feature descriptor extracted from the enhanced feature map; Learnable query vectors are used for proactively querying feature maps; LSTM state vector: 128-dimensional, used for temporal state prediction; Internal memory stores historical state information from the most recent N frames (e.g., N=50) for recovery after target loss; Timer: Records the duration of continuous target loss.

[0068] At each time t_k, each active target agent A_i performs the following operations: The query vector it maintains is transformed linearly to generate an attention-based query; Centered on the current location, define a local window (e.g., a region with a radius of 20 grids centered on the target) on the enhanced spatiotemporal feature map E_k. Within this local window, a multi-head attention mechanism is used to calculate the relevance score between each position and the query, forming an attention weight map; The features within the window are weighted and summed according to the attention weights to obtain the aggregated feature vector f_i; Reduce the dimensionality of f_i to obtain the observation feature o_i at the current time.

[0069] In this embodiment, the target agent internally maintains an LSTM state predictor. At each time step, the following updates are performed: The hidden state from the previous time step is concatenated with the currently observed feature o_i and used as the input to the LSTM. The LSTM outputs the hidden state h_t and the cell state c_t at the current time step. The target state estimate at the current moment is output by taking h_t as input to the state decoder (which consists of two fully connected layers), including position offset, velocity, acceleration, etc. Update the target's absolute position (x, y, z) in BEV space based on the output position offset; The updated state is stored in internal memory, and the timer is updated (if a valid observation is successfully obtained, the lost timer is reset).

[0070] In this embodiment, when the signal is interrupted due to the target being blocked or flying out of the sensor coverage area, the agent cannot find a valid observation from the enhanced feature map for multiple consecutive moments (e.g., 5 consecutive frames) (i.e., the maximum value of the attention weight is lower than the threshold). At this time, it is determined that the target has entered a lost state. The agent activates its internal memory module to perform motion prediction based on historical states (position and velocity of the most recent 10 frames) (e.g., using Kalman filtering or a constant velocity model), infers the estimated position of the target at the current moment, and outputs the predicted trajectory.

[0071] In this embodiment, when the feature information of the current target to be tracked is reproduced in the enhanced spatiotemporal feature map, and the similarity between the reproduced feature information and the historical feature information exceeds a preset threshold, the target agent re-associates the reproduced feature information and resumes the tracking process of the current target to be tracked; if the target agent fails to re-associate the reproduced feature information within a second preset time, the internal memory of the target agent gradually decays over time; when the internal memory decays to a preset extinction threshold, the target agent is terminated.

[0072] In this embodiment, after the target is determined to be lost, the agent's loss timer starts to accumulate, and the feature descriptors in the internal memory remain unchanged, serving as templates for subsequent re-association. In subsequent moments, when new feature information appears in the enhanced feature map, the agent calculates the similarity (e.g., cosine similarity) between the new feature and the feature descriptors in memory. If the similarity exceeds a preset threshold (e.g., 0.7) and the location of the new feature is close to the predicted location, then the feature is determined to belong to the original target. The agent re-associates the feature, resumes tracking the target, and keeps the original TrackID unchanged. If the agent's loss timer exceeds the preset decay period (e.g., 10 seconds), the internal memory will gradually decay until the agent is terminated and the TrackID is released.

[0073] In this embodiment, each agent continuously outputs the target's tracking trajectory, and its output information includes: TrackID: A unique identifier for the target; Timestamp: The global time of the current moment; Location: Three-dimensional coordinates (x, y, z) in the ENU coordinate system (unit: meters); Velocity: Three-dimensional velocity (v_x, v_y, v_z) (unit: meters per second); Status flags: 0 indicates normal tracking, 1 indicates loss prediction, and 2 indicates a new target; Confidence level: a score between 0 and 1.

[0074] The above information is encapsulated in JSON format via UDP protocol and sent to the display and control terminal. The display and control terminal plots the target trajectory on the electronic map in real time, and triggers an audible and visual alarm when the target enters the preset warning area.

[0075] The following example, using a specific scenario, illustrates the complete workflow of this embodiment: Initial time t=0s: The UAV tracking system starts and begins to acquire target observation data, but there is no target in the airspace; t=5.23s: A drone enters the monitoring area. The radar detects the target point (distance 2000m, azimuth 90 degrees, pitch 2 degrees) in the point cloud frame at 5.20s. The spectrum monitoring equipment captures the 2.4GHz remote control signal at 5.22s with an angle of arrival of 88 degrees. At the same time, the photoelectric turntable turns towards the target direction according to the radar guidance. At t=5.25s: The data alignment module uses t=5.25s as the anchor point, aggregating radar point cloud and spectral signals to generate a spatiotemporal snapshot S_001. The feature projection module projects the data onto the BEV space, generating a multimodal feature map; the temporal fusion network (at this point, only the current frame is in the window) outputs an enhanced feature map E_5.25. The detection module detects the target at the corresponding position (x=2000m, y=0m) of E_5.25, with a confidence level of 0.85. The target agent module instantiates an agent for it and assigns TrackID=1; t=5.30s: A new spatiotemporal snapshot S_002 is generated. The agent TrackID=1 queries features on the enhanced feature map E_5.30 through an attention mechanism, successfully acquires the observation, updates the state to position (1985m, 5m, 40m) and velocity (30m / s, 10m / s, 0m / s), and continues normal tracking; t=5.35s: The drone flies behind the building and is briefly obscured. The radar loses the target echo, and the spectrum signal is also blocked. The agent TrackID=1 cannot find a valid observation on E_5.35, enters the lost state, activates the memory module to start predicting the trajectory, and outputs the predicted position (1970m, 10m, 40m); t=5.40s: The drone flies out from the other side of the building and is recaptured by the sensor. A new feature appears on the enhanced feature map E_5.40. The agent TrackID=1 compares the new feature with the feature descriptor in memory. The similarity is 0.82, which exceeds the threshold. Moreover, the location of the new feature (1955m, 15m, 40m) is close to the predicted location. It is determined to be the same target, re-associated, and tracking resumes. TrackID remains unchanged at 1. t=5.45s-5.60s: The system continues to track the target until it flies out of the monitoring area. When the target signal completely disappears for more than 10 seconds, the internal memory of the agent TrackID=1 decays to below the extinction threshold, terminating the agent.

[0076] Based on the above process, this embodiment achieves continuous and stable tracking of UAV targets, maintaining the target's identity even under brief obstruction, and outputting a complete and reliable tracking trajectory.

[0077] It should be noted that in the above embodiments, there is no necessarily a certain order between the steps. Those skilled in the art can understand from the description of the embodiments of this application that the above steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in turn, etc.

[0078] As another aspect of this application, this application provides a drone tracking device applied to the aforementioned electronic device 20. The drone tracking device can be a software module, which includes several instructions stored in a memory. A processor can access the memory, invoke the instructions, and execute them to complete the methods described in the various embodiments above.

[0079] In some embodiments, the drone tracking device can also be constructed from hardware components. For example, the drone tracking device can be constructed from one or more chips, which can work in coordination to complete the methods described in the various embodiments above. As another example, the drone tracking device can also be constructed from various logic devices, such as general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), microcontrollers, ARM (Acorn RISC Machine) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0080] Please see Figure 3 The drone tracking device includes: Data acquisition module A10 is used to acquire target observation data; The data synchronization module A20 is used to process the target observation data into a synchronized spatiotemporal snapshot sequence based on a preset spatiotemporal reference. The temporal fusion module A30 is used to project the spatiotemporal snapshot sequence onto a preset global representation space and perform temporal fusion on the multimodal feature maps obtained by the projection to determine the enhanced spatiotemporal feature maps; The process tracking module A40 is used to instantiate a target agent for a detected target to be tracked, so as to use the target agent to track the target to be tracked. The target agent queries and obtains feature information according to the enhanced spatiotemporal feature map, and updates its own state based on the feature information to output the tracking trajectory of the target to be tracked.

[0081] It should be noted that the above-described apparatus can execute the method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in the apparatus embodiments can be found in the method provided in the embodiments of this application.

[0082] See Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device can be a server, server cluster, distributed computing node, virtualized cloud host, or edge computing gateway, or other device with data processing and communication capabilities. Figure 4As shown, the electronic device includes one or more processors 31 and a memory 32. The memory 32 is connected to one or more processors 31, for example, via a bus.

[0083] Processor 31 is configured to support the electronic device in performing the corresponding functions in the methods described in the above method embodiments. Processor 31 may be a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof. The aforementioned hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0084] Memory 32 is used to store program code, etc. Memory 32 may include volatile memory (VM), such as random access memory (RAM); memory 32 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); memory 32 may also include combinations of the above types of memory.

[0085] The memory 32 can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of this application. The processor 31 executes various functional applications and data processing of the method and apparatus by running the non-volatile software programs, instructions, and modules stored in the memory 32, that is, it realizes the functions of each module or unit of the method and apparatus provided in the above method embodiments.

[0086] The memory 32 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function. The data storage area may store data created based on the use of the device. In some embodiments, the memory 32 may optionally include memory remotely located relative to the processor 31, and this remote memory may be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0087] The one or more modules are stored in the memory 32. When executed by the one or more processors 31, they execute the methods in any of the above method embodiments. For example, they execute the method steps described in the above method embodiments to realize the functions of the modules described in the above device embodiments.

[0088] This application also provides a computer-readable storage medium storing a computer program, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the method described in the foregoing embodiments.

[0089] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0090] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A method for tracking a UAV, the method comprising: include: Acquire target observation data in a target airspace, where at least one target to be tracked exists; Based on a preset spatiotemporal reference, the target observation data is processed into a synchronized spatiotemporal snapshot sequence; The spatiotemporal snapshot sequence is projected onto a preset global representation space, and the multimodal feature maps obtained by the projection are temporally fused to determine the enhanced spatiotemporal feature maps; Based on the enhanced spatiotemporal feature map, the detected target is tracked, and the tracking trajectory of the target is generated. 2.The UAV tracking method of claim 1, wherein, The process of processing the target observation data into a synchronized spatiotemporal snapshot sequence based on a preset spatiotemporal reference includes: Based on a preset spatiotemporal reference, a time anchor grid is constructed, which consists of multiple time anchor points; A time assignment window is defined with the time corresponding to each time anchor point as the center, and the target observation data within each time assignment window is aggregated to the corresponding time anchor point. By aggregating the target observation data at the same time anchor point through spatiotemporal correlation analysis, the spatiotemporal snapshot sequence is obtained. 3.The UAV tracking method of claim 2, wherein, When aggregating target observation data within each time assignment window to the corresponding time anchor point, if the amount of observation data within the time assignment window is less than a preset threshold, the time assignment window is expanded to obtain observation data outside the time assignment window. 4.The UAV tracking method of claim 1, wherein, The global representation space is a world model constructed using a bird's-eye view space or voxel space. The spatiotemporal snapshot sequence includes radar point cloud data, spectrum direction finding data, and photoelectric image data. Projecting the spatiotemporal snapshot sequence onto the preset global representation space includes: The radar point cloud data is projected onto the world model to generate feature channels containing target motion attributes at the corresponding locations; The spectrum direction finding data is projected onto the world model to generate signal features in the corresponding azimuth region; The photoelectric image data is projected onto the world model to generate visual features in the corresponding region; The various features obtained from projection are stitched together along the channel dimension to form a multimodal feature map.

5. The UAV tracking method according to claim 4, characterized in that, Temporal fusion of the multimodal feature maps obtained from projection includes: Multimodal feature maps at multiple consecutive time points are input into a preset temporal fusion network. The temporal fusion network captures the dependencies of the multimodal feature maps in the time dimension and outputs the enhanced spatiotemporal feature map.

6. The UAV tracking method according to claim 1, characterized in that, The step of tracking the detected target based on the enhanced spatiotemporal feature map and generating the tracking trajectory of the target includes: A target agent is instantiated for the detected target to be tracked, and the target agent is used to track the target. The target agent obtains feature information by querying the enhanced spatiotemporal feature map and updates its own state based on the feature information to output the tracking trajectory of the target.

7. The UAV tracking method according to claim 6, characterized in that, Each of the target agents maintains at least one set of learnable query vectors; The target agent obtains feature information by querying the enhanced spatiotemporal feature map, including: The target agent sends its maintained query vector to the enhanced spatiotemporal feature map, and queries and obtains the feature information most relevant to its own state through an attention mechanism.

8. The UAV tracking method according to claim 7, characterized in that, The target agent maintains internal memory, which is used to store historical feature information of the target to be tracked. When the target agent fails to find valid feature information from the enhanced spatiotemporal feature map within a first preset time, state deduction is performed based on the internal memory. When the feature information of the target to be tracked is reproduced in the enhanced spatiotemporal feature map, and the similarity between the reproduced feature information and the historical feature information exceeds a preset threshold, the target agent re-associates the reproduced feature information and resumes the tracking process of the target to be tracked. If the target agent fails to re-associate the reproduced feature information within a second preset time period, the internal memory of the target agent will gradually decay over time. When the internal memory decays to a preset extinction threshold, the target agent will be terminated.

9. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to at least one of the processors; The memory stores instructions that can be executed by the processor to implement the drone tracking method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are executed by a computer to implement the drone tracking method according to any one of claims 1-8.

Images