Anti-drone intelligent perception method based on low-altitude world model

Abstract

The application discloses an anti-UAV intelligent perception method based on a low-altitude world model, which comprises the following steps: preprocessing UAV video data; constructing a granular anti-interference tracking module to realize stable target tracking in a double dynamic scene; constructing a hierarchical low-altitude world model, aggregating multi-source information into unified Tokens, embedding a UAV dynamics model and low-altitude environment constraints, and outputting a structured target state sequence; outputting a UAV flight intention based on the structured target state sequence; constructing a collaborative framework to realize full-link collaboration, and outputting a target position, a state sequence and a flight intention for anti-UAV decision-making. The application solves the problems of unstable tracking in a double dynamic scene, lack of a low-altitude exclusive world model and single perception dimension, and realizes the advancement from perceiving a target to cognizing behavior.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent sensing technology, and in particular to an intelligent sensing method for anti-drone scenarios in dual dynamic scenes based on a low-altitude world model. Background Technology

[0002] With the rapid development of the low-altitude economy, drones are widely used in various fields such as environmental monitoring, logistics distribution, traffic monitoring, agricultural plant protection, and public safety. However, airspace conflicts and security threats caused by problems such as "unauthorized flights" and "illegal flights" are becoming increasingly prominent, making the demand for counter-drone technology urgent. As an important means of ensuring the safety of low-altitude airspace, counter-drone systems need to achieve accurate perception, stable tracking, and intent recognition of drones to support proactive defense decisions.

[0003] Existing anti-drone sensing technologies mainly face the following challenges:

[0004] First, there is the issue of tracking stability in dual-dynamic scenarios. Traditional anti-drone methods mostly employ single-dynamic scenarios, such as air-to-ground or ground-to-air. However, in air-to-air dual-dynamic scenarios, both the observation platform and the target drone are in high-speed motion. Accompanied by issues such as rapid scale changes, strong motion blur, and frequent viewpoint switching, coupled with the effects of partial occlusion and complex interference backgrounds, traditional target tracking methods are prone to tracking breakage and loss.

[0005] Second, there is a lack of world models specifically for low-altitude airspace. Current research on world models mainly focuses on ground-based autonomous driving scenarios, with modeling logic centered around road topology and vehicle trajectories, without customizing them for the unique attributes of low-altitude airspace. This has resulted in a long-standing lack of a unified expression system for world models specifically for low-altitude airspace, making it impossible to effectively incorporate the dynamic characteristics of UAVs and the constraints of the low-altitude environment.

[0006] Third, the perception dimensions are limited and the cognitive depth is insufficient. Traditional counter-drone methods remain at the basic level of target identification and tracking, neglecting the modeling and representation of the drone's flight intentions, and thus failing to achieve the advancement from perceiving the target to recognizing the behavior. Existing methods only focus on visual information and lack the fusion of multi-source information such as the drone's motion state, airspace semantics, and environmental constraints, making it difficult to support intelligent situational awareness and proactive defense decision-making.

[0007] To address these issues, researchers have proposed several improved methods. Deep learning-based tracking methods use convolutional neural networks or Transformer architectures to extract features, but they still lack dynamic adaptability under extreme interference in dual-dynamic scenes. Multi-object tracking methods combine detection and trajectory association, but they are prone to identity switching and trajectory breakage in complex backgrounds. Attention-based methods regulate feature weights through spatial or channel attention, but they mostly employ single-scale feature extraction and global attention mechanisms, making it difficult to adapt to the rapid scale changes and motion blur in dual-dynamic scenes, and they lack a local-to-global collaborative mechanism.

[0008] While existing methods have improved tracking performance to some extent, they still have significant limitations: they cannot effectively handle rapid scale changes and strong motion blur in air-to-air scenarios; they lack embedding of UAV dynamics models and low-altitude environmental constraints, resulting in inaccurate characterization of state evolution; and they cannot achieve the leap from visual perception to intent cognition, making it difficult to distinguish different flight intentions under similar trajectories.

[0009] It is worth noting that the world model required for low-altitude anti-drone scenarios differs fundamentally from existing ground-based autonomous driving world models: In terms of modeling objects, ground-based world models focus on road topology and vehicle trajectories in a two-dimensional plane, while low-altitude scenarios require modeling the six degrees of freedom motion of drones in three-dimensional airspace; in terms of constraints, ground-based world models are primarily constrained by human-made traffic rules, while low-altitude scenarios are constrained by the drone's own dynamic limits and the low-altitude physical environment; in terms of prediction targets, ground-based world models mainly achieve trajectory prediction, while low-altitude scenarios require not only trajectory prediction but also flight intent recognition, moving from perceiving targets to recognizing behavior. Therefore, the modeling paradigm of ground-based world models cannot be directly transferred to low-altitude anti-drone scenarios. Summary of the Invention

[0010] The technical problem to be solved by this invention is to address the shortcomings of existing anti-drone technologies, such as unstable tracking in dual-dynamic scenarios, lack of a low-altitude exclusive world model, single perception dimension, and insufficient cognitive depth. This invention proposes an intelligent anti-drone perception method that can achieve stable tracking against interference in air-to-air dual-dynamic scenarios, construct a low-altitude world model with enhanced airspace semantics, and realize the advancement from target perception to intent cognition.

[0011] To address the aforementioned technical problems, this invention first clarifies the construction requirements of a low-altitude world model: existing modeling paradigms for ground-based autonomous driving world models revolve around road topology and vehicle trajectories, which cannot be directly applied to low-altitude anti-drone scenarios. The low-altitude world model of this application is a specialized construction tailored to the characteristics of low-altitude airspace, aiming to enable visual question-answering systems to achieve physical and spatial understanding of drones. By embedding drone dynamics and low-altitude environmental constraints, it achieves physical rationality modeling and spatial semantic understanding of drone flight states. Based on this, the technical solution adopted by this invention is an anti-drone intelligent perception method based on a low-altitude world model, comprising the following steps:

[0012] Step 1: Preprocess the first-person view video data from the UAV, extract multimodal features, and align and fuse them to generate a unified multimodal feature representation; the multimodal features include visual features, text features, and state features;

[0013] Step 2: Construct a granular anti-interference tracking module, which achieves stable target tracking in dual dynamic scenes through visual-text feature dual enhancement, spatial-channel joint attention control, local attention mechanism and background interference suppression;

[0014] Step 3: Construct a hierarchical low-altitude world model with enhanced airspace semantics. The hierarchical low-altitude world model includes a multi-source input hierarchical coding layer, a physical rule hierarchical embedding layer, and a cognitive depth hierarchical output layer. Multi-source information is aggregated into unified tokens. The UAV dynamics model and low-altitude environmental constraints are embedded in the physical rule hierarchical embedding layer, and a structured target state sequence is output in the cognitive depth hierarchical output layer.

[0015] Step 4: Based on the structured target state sequence output by the hierarchical low-altitude world model, the UAV flight intention is output through intention candidate generation and confidence evaluation;

[0016] Step 5: Construct a collaborative framework for granular anti-interference tracking, hierarchical low-altitude world model, and intent perception, and achieve end-to-end collaboration through forward information transmission and feedback optimization mechanisms;

[0017] Step 6: Input the current frame video data into the trained collaborative framework, and output the target position, state sequence and flight intention for anti-drone decision-making.

[0018] Specifically, step 2 includes:

[0019] Step 2-1: Extract multi-scale visual features from video frames using a visual Transformer, generate semantic features by encoding the target description using a text encoder, and combine template features to achieve dual enhancement of visual and text features;

[0020] Step 2-2: Use the inter-layer attention mechanism to align the network to achieve progressive feature alignment between visual and textual features;

[0021] Steps 2-3: In the Backbone global detection process, a spatial-channel joint attention mechanism is introduced to dynamically adjust the feature weights for rapid scale changes and motion blur, and output attention-weighted features;

[0022] Steps 2-4: Perform global target detection based on the attention-weighted features, and combine the DeepSort multi-target tracking algorithm and confidence feedback mechanism to form a granular mechanism for global detection to local tracking, outputting the target position and tracking confidence.

[0023] Steps 2-5: Construct a positive and negative sample library, filter out interfering background features through feature matching, and output the filtered target location and tracking confidence.

[0024] Specifically, in steps 2-3, the spatial-channel joint attention mechanism includes:

[0025] Spatial attention employs a coordinate attention mechanism, aggregating features along the horizontal and vertical directions respectively to accurately locate the target's spatial position;

[0026] Channel attention employs an efficient channel attention mechanism, which captures inter-channel dependencies through one-dimensional convolution and filters discriminative feature channels;

[0027] The channel attention weights are multiplied one by one with the feature map of the spatial attention output to obtain the attention-weighted features.

[0028] Specifically, the multi-source input hierarchical encoding layer encodes visual features, text descriptions, location bounding boxes, tracking confidence, and the UAV's own state into unified tokens; the physical rule hierarchical embedding layer embeds the UAV flight dynamics model and low-altitude airspace environmental constraints respectively; and the cognitive depth hierarchical output layer outputs a structured target state sequence consisting of motion state, behavioral state, and environmental interaction state according to cognitive depth.

[0029] Specifically, step 4 includes:

[0030] Step 4-1: Based on the structured target state sequence and environmental context output by the hierarchical low-altitude world model, and combined with preset intent query instructions, generate multi-dimensional intent candidates through visual question answering;

[0031] Step 4-2: Evaluate the confidence of intent candidates using multi-dimensional indicators such as motion state matching degree, spatial position verification degree, and interference feature correlation degree;

[0032] Step 4-3: Select the intent with the highest confidence as the current intent prediction result, and output the intent category and confidence score.

[0033] Specifically, the feedback optimization mechanism includes:

[0034] When the confidence score of the intended prediction is lower than the preset threshold, the world model parameters are adaptively updated, the weight of the observed data is increased, and the state sequence is re-estimated.

[0035] When the tracking confidence level is lower than a preset threshold and remains below the threshold for multiple consecutive frames, it is determined as a tracking break. The intent perception module predicts the possible areas where the target may appear based on historical states and guides re-capture.

[0036] The beneficial effects of this invention are as follows: By proposing a granular anti-interference tracking method with joint attention control of space and channel, it specifically adapts to the rapid scale changes, strong motion blur, and frequent viewpoint switching in air-to-air scenarios, overcoming the shortcomings of traditional methods in dynamic adaptation and achieving stable tracking in dual-dynamic scenarios; by pioneering a hierarchical low-altitude world model with enhanced airspace semantics, it aggregates multi-source information tokens such as target visual features, UAV dynamic parameters, and low-altitude environmental constraints to construct a unified airspace expression system from visual semantics to physical constraints, filling the technical gap in low-altitude exclusive world models; by constructing a full-link collaborative framework of perception, modeling, and cognition, it achieves the advancement from "perceiving the target" to "cognizing the UAV's flight intention," solving the problems of single perception dimension and insufficient cognitive depth, and providing key technical support for the next generation of anti-UAV intelligent systems or equipment. Validation on anti-UAV datasets shows that compared with traditional tracking methods and multimodal large models, the tracking success rate of this invention in dual-dynamic scenarios is more than 10% higher than the state-of-the-art (SOTA), and the intention recognition accuracy is superior to existing multimodal large models. Attached Figure Description

[0037] Figure 1 This is a diagram of the overall structure of intelligent sensing for counter-drone systems based on a low-altitude world model.

[0038] Figure 2 A flowchart of intelligent anti-drone perception based on a low-altitude world model;

[0039] Figure 3 This is a diagram illustrating the effect of empty-to-empty tracking and intent perception. Detailed Implementation

[0040] This invention proposes an anti-jamming anti-drone perception method based on a spatial semantic augmented world model, aiming to improve the tracking accuracy, stability, environmental understanding, and intent recognition capabilities of anti-drone systems in dual-dynamic complex scenarios. This invention enhances the system's perception and decision support capabilities by introducing granular anti-jamming tracking, a hierarchical low-altitude world model, and intelligent intent recognition into air-to-air dual-dynamic scenarios and applying them to anti-drone systems.

[0041] The anti-drone perception system of this invention includes: data preprocessing, granular anti-interference tracking, low-altitude world model construction with enhanced airspace semantics, intelligent intent perception, and collaborative framework optimization. The system framework is as follows: Figure 1 As shown. The system implementation process is as follows. Figure 2 As shown.

[0042] The specific implementation steps are as follows:

[0043] Step 1: Data Preprocessing

[0044] The first-person view video data from the UAV is preprocessed to meet the input data requirements of the tracking model and the world model. The preprocessing process includes data format conversion, multimodal feature encoding, feature alignment, and fusion.

[0045] 1-1. Data Format Conversion

[0046] The original video data is decoded into a continuous frame image sequence, and the timestamp information is extracted to ensure temporal consistency. The image data is then converted into a unified tensor format and normalized to the [0,1] interval.

[0047] 1-2. Multimodal Feature Encoding

[0048] The video frames are visually encoded to extract multi-scale visual features; the target description text is semantically encoded to generate text feature vectors; and the UAV state information (position, velocity, attitude) is numerically encoded to generate state feature vectors.

[0049] 1-3. Feature Alignment and Fusion

[0050] Visual features, text features, and state features are aligned and fused through a cross-modal attention mechanism to generate a unified multimodal feature representation, which serves as the input for subsequent modules.

[0051] Step 2: Construction of Granularity-Based Anti-Interference Tracking Module

[0052] Due to extreme interference such as rapid scale changes, strong motion blur, and frequent viewpoint switching in air-to-air dual-dynamic scenes, this invention employs a granular anti-interference tracking architecture to achieve stable target tracking in such scenes. This architecture includes: a visual encoder, a text encoder, a feature fusion layer, an alignment layer, a spatial channel attention layer, a local attention module, a background suppression module, a DeepSort association layer, and an output layer. A combination of ReLU and GELU activation functions is used.

[0053] 2-1. Visual-Text Feature Dual Enhancement and Progressive Alignment

[0054] Video frames are encoded using a visual Transformer, and multi-scale visual features are extracted using a hierarchical feature pyramid structure. Shallow features preserve detailed information, while deep features capture semantic information. Target descriptions, such as a black quadcopter drone or a high-speed moving target, are encoded using a BERT-based text encoder to generate semantic feature vectors. Template features (target region features from the first frame) are combined to achieve dual enhancement of visual and textual features.

[0055] ;

[0056] in, For gating weights, As a visual feature, For text features, This represents the fusion feature after double enhancement.

[0057] 2-2. Progressive Alignment

[0058] Visual features guide textual attention to relevant regions, while textual features guide visual attention to semantically related regions. An inter-layer attention mechanism is used to align visual and textual features in a progressively refining network, adapting to feature differences caused by scale variations and perspective shifts through multiple layers.

[0059] 2-3. Spatial-channel joint attention regulation

[0060] The dual-enhanced fusion features output from step 2-1 As input, a spatial-channel joint attention mechanism is introduced during Backbone global detection. Spatial attention employs a coordinate attention mechanism, decomposing channel attention into two one-dimensional feature encodings, aggregating features along the horizontal and vertical directions respectively, to accurately locate the target's spatial position.

[0061] ;

[0062] ;

[0063] in, This represents the value of the input feature map at the c-th channel, height position h, and horizontal position i. The width of the feature map. This represents the feature vector of the c-th channel at height position h obtained after pooling along the horizontal direction; This represents the value of the input feature map at the c-th channel, height position j, and horizontal position w. The height of the feature map, This represents the feature vector of the c-th channel at horizontal position w obtained after pooling along the vertical direction.

[0064] After spatial attention locates the target's spatial position, its output feature map is used as the input for channel attention. Let this feature map be... 'c' represents the channel index. An efficient channel attention ECA mechanism is used to capture inter-channel dependencies and filter discriminative feature channels. The core formula for calculating channel attention weights is:

[0065] ;

[0066] in, For the input feature map, One-dimensional convolution, This represents the kernel size of a one-dimensional convolution. It is the Sigmoid activation function. Here is the channel attention weight vector, and AdaptiveAvgPool is the adaptive average pooling.

[0067] Channel attention weights Feature map of spatial attention output Multiplying each channel together yields the final attention-weighted feature. :

[0068] ;

[0069] in, This indicates multiplication by channel.

[0070] Spatial-channel joint attention dynamically adjusts feature weights to mitigate interference from rapid scale changes and motion blur. Spatial attention focuses on the spatial location of the target region, while channel attention selects discriminative feature channels. After spatial-channel joint attention modulation, the output is attention-weighted features. The characteristic response of the target region is enhanced, while the characteristic response of the interference region is suppressed.

[0071] 2-4. Local Attention and Granularity Mechanism

[0072] The attention-weighted features output from steps 2-3 As input, a granular mechanism combining global detection and local tracking is constructed.

[0073] First, based on Perform global object detection to obtain the detection bounding box and detection confidence. Simultaneously, the Kalman filter in the DeepSort multi-target tracking algorithm is used to predict the target's motion state:

[0074] ;

[0075] in, For prior state estimation, the target state in the k-th frame is predicted based on the information in the (k-1)-th frame. For posterior state estimation, the optimal state estimate of the (k-1)th frame; Here is the state transition matrix. To control the input matrix, To control the quantity.

[0076] By correlating the detection results with the predicted trajectory using the Hungarian algorithm, the trajectory update results and correlation confidence are obtained.

[0077] Introduce a confidence feedback mechanism:

[0078] When global detection confidence When the value is below a preset threshold, a local tracking mode is triggered. The local tracking mode locates the local region of the target through a coordinate attention mechanism, generates high-resolution local template features through lightweight convolutional reconstruction, and performs target localization based on template matching, forming a granular mechanism that enables global detection and local tracking.

[0079] After granular tracking processing, based on the results of global detection and local tracking, the optimized target position bounding box and tracking confidence are output. Specifically, when the global detection confidence score is higher than a threshold, the trajectory position associated using the Hungarian algorithm is used as the output; when the local tracking mode is triggered, the local template matching localization result is used as the output. The output target position bounding box and tracking confidence score are then compared. Used for background interference suppression in subsequent steps 2-5.

[0080] 2-5. Background Interference Suppression

[0081] Using the target location bounding box and tracking confidence output from steps 2-4 As input, construct a positive and negative sample database. When the value exceeds a preset threshold, the features of the current detection region are added to the sample library as positive samples; when... When the value is below a preset threshold, the features of the current detection area are added to the sample library as negative samples.

[0082] Positive samples represent target features, while negative samples represent background interference features. Interference background features are filtered out using feature matching based on cosine similarity. The calculation formula is:

[0083] ;

[0084] in, Features of the current detection area The feature vectors in the sample library, The L2 norm is used. When the similarity between the detected region features and the negative sample library exceeds a threshold, it is identified as an interference region and its response is suppressed. By updating the positive and negative sample libraries through online learning, it adapts to scene changes, improves tracking robustness in partially occluded and complex backgrounds, and effectively avoids tracking breakage in dual-dynamic scenes.

[0085] After background interference suppression, the output is the filtered target position. and tracking confidence This is used for state sequence modeling of the low-altitude world model in step 3.

[0086] Step 3: Construction of a hierarchical low-altitude world model with enhanced spatial semantics

[0087] This invention constructs the first low-altitude world model for air-to-air anti-drone scenarios, which differs from traditional world models for ground-based autonomous driving scenarios and specifically incorporates drone dynamics characteristics and low-altitude environmental constraints.

[0088] The layered low-altitude world model constructed in this step adopts a three-layer architecture:

[0089] Multi-source input hierarchical coding layer: Encodes different modal information such as visual features, text descriptions, location bounding boxes, and the drone's own state separately, maintaining the independence of each modal information and avoiding information aliasing.

[0090] The physical rules are layered and embedded in two layers: one for the UAV flight dynamics model and the other for low-altitude airspace environmental constraints. The dynamics layer adapts to typical motion modes such as cruise, hovering, acceleration, and turning; the airspace constraint layer embeds physical constraints such as no-fly zone boundaries, building distribution, and weather conditions, as well as the feasible domain for the evolution of constraint states.

[0091] Cognitive Depth Layered Output Layer: Structured results are output according to cognitive depth, including motion state layer (position, velocity, acceleration, attitude), behavior state layer (maneuvering mode), and environmental interaction state layer (distance to no-fly zone / interference source).

[0092] 3-1. Multi-source input coding layer

[0093] Design a multi-source input encoding layer to aggregate the following information into unified tokens:

[0094] Visual features: Multi-scale visual features output by the tracking module, including the dual-enhanced fusion features generated in step 2-1. and the weighted features output from steps 2-3 ;

[0095] Text description: Task description, environment description, target attribute description;

[0096] bounding box position (bbox): the filtered target position output from steps 2-5. ;

[0097] Tracking confidence: The filtered tracking confidence output from steps 2-5 ;

[0098] The drone's own status: speed, acceleration, attitude angle, angular velocity.

[0099] By mapping each modal feature to a unified dimension through linear projection, injecting temporal information through positional encoding, and distinguishing the information source through modality type encoding, a multimodal token sequence is generated.

[0100] The formula for generating tokens is:

[0101] ;

[0102] in, A multimodal token sequence, used as input to the low-altitude world model; For visual feature tokens, multi-scale visual features from the tracking module (including...) and ); It is used for semantic encoding of textual information such as text description tokens, target descriptions, task descriptions, and environment descriptions; For location bounding box tokens, For drone status token; To track confidence tokens; For timing position encoding, Encodes the modal type.

[0103] Textual description information (task description, environment description, target attribute description) is denoted as the environment context.

[0104] Used for subsequent intent-aware modules.

[0105] 3-2. Low-altitude world modeling layer

[0106] A low-altitude world modeling layer is constructed, which is based on the JEPA architecture and embeds the following prior knowledge.

[0107] The multimodal token sequence generated in step 3-1 Input the low-altitude world modeling layer. Through the Transformer's multi-head attention mechanism, deep interaction is achieved between different modal tokens, including visual features, text descriptions, location bounding boxes, tracking confidence, and the UAV's own state. Specifically, visual features and text descriptions are fused through cross-attention to enhance semantic understanding of the airspace scene; tracking confidence, as a weighting factor, influences the reliability of the corresponding frame state modeling.

[0108] UAV Dynamics Model: Establish the kinematic and dynamic equations of a quadcopter or fixed-wing UAV to describe the evolution of position, velocity, and attitude, adapting to typical motion modes such as cruise, hovering, acceleration, and turning. The core equation of UAV position kinematics is:

[0109]

[0110] In the formula, Let be the position vector of the UAV in the geocentric inertial frame. Let be a rotation matrix. This is the linear velocity vector in the body coordinate system, used to describe the positional evolution of the UAV.

[0111] Low-altitude environmental constraints: These constraints include airspace limitations (no-fly zone boundaries, altitude restrictions), interference source distribution (building locations, electromagnetic interference intensity), and meteorological conditions (wind speed vectors, visibility range), thus defining the feasible region for state evolution. Low-altitude environmental constraints limit the feasible region of the state through inequalities, ensuring that state evolution conforms to actual environmental limitations.

[0112] The attention mechanism models the long-term dependencies of state sequences. In this mechanism, textual description tokens provide semantic context, such as the target being a black quadcopter drone flying at low altitude, guiding visual features to focus on task-related regions. Tracking confidence tokens are used to dynamically adjust the weights of state predictions—high confidence enhances the observation information of the corresponding frame, while low confidence reduces the weight of observation information and enhances the weight of prediction information. A dynamic constraint layer ensures the physical rationality of state evolution, while an environmental constraint layer limits the feasible range of states.

[0113] 3-3. State Sequence Modeling and Output

[0114] Based on embedded dynamic models and environmental constraints, and combining visual features, text descriptions, location bounding boxes, and the UAV's own state in the multimodal token sequence, the target state sequence of past frames is modeled to characterize the state evolution logic. The semantic information provided by the text description helps to determine the current flight mode, such as accelerated climb corresponding to an active interference intention, and tracking confidence is used to filter credible state updates.

[0115] Output structured target state sequence ,include:

[0116] motion state Position, velocity, acceleration, attitude;

[0117] Behavioral state : Maneuvering modes (constant speed, acceleration, steering, hovering);

[0118] Environment interaction status Distance from the no-fly zone and distance from the source of interference;

[0119] Complete the digital and semantic mapping of low-altitude scenarios and drone status to support intent perception.

[0120] Step 4: Intent perception based on the low-altitude world model

[0121] Based on the state sequences output by the world model, the system advances from perceiving the target to cognitive behavior. The world model obtains the dynamic state characteristics and trajectory state characteristics of the UAV flight through video rehearsal, and VQA combines these state sequences to generate intent candidates and responses.

[0122] 4-1. Intent Candidate Generation Unit

[0123] The design intent candidate generation unit covers typical UAV behavior modes (normal cruise, area reconnaissance, target tracking, active jamming, and escape / avoidance). Combining dynamic constraints (which maneuvers are feasible) and environmental constraints (which behaviors are reasonable), it generates multi-dimensional intent candidates and their prior probabilities through VQA. The process is as follows:

[0124] (1) Input construction: The structured state sequence and environmental information output by the world model are combined with the preset intent query command to form the input triplet of Visual Question Answering (VQA). :

[0125] ;

[0126] in, For query commands, As the state context, the structured target state sequence As a state context (including motion state, behavior state, and environmental interaction state). The text description information in step 3-1 serves as the environmental context.

[0127] (2) Multimodal coding: The multimodal encoder will transmit the state context. and environment context Perform fusion encoding to obtain context-encoded features. Simultaneously, the query command will be executed. Encode the query to obtain the query encoding features. ;

[0128] (3) Cross-interaction: Integrating query coding features As a query, context encoding features As keys and values, a cross-attention mechanism enables deep interaction between queries and context;

[0129] In the attention weight calculation process, dynamics and environmental constraints are embedded: based on the UAV dynamics model (e.g., acceleration not exceeding the maximum rate of climb, turning radius limits) and low-altitude environmental constraints (e.g., location not within a no-fly zone, altitude not exceeding a height restriction zone), an attention mask matrix is ​​generated. Attention weights that do not conform to physical rules or environmental constraints are reset to negative infinity, making the softmax weights zero, thereby filtering out features that conform to physical rules. Input decoder;

[0130] (4) Decoding output: Generate intent candidate text through decoder and output the prior probability of each candidate;

[0131] (5) Candidate filtering: Based on the prior probability threshold, retain valid intent candidates to form the final candidate set.

[0132] 4-2. Confidence Assessment Unit

[0133] A confidence assessment unit is constructed, using the intent candidate set output in step 4-1 as input, to evaluate the confidence of each intent candidate. The intent candidates are ranked by confidence using multi-dimensional indicators, with the core indicators being motion state matching degree, spatial position verification degree, and interference feature correlation degree.

[0134] Motion state matching degree: Analyzes the degree of conformity between trajectory features (velocity changes, curvature, acceleration) and intent patterns. For normal cruise intent, the expected trajectory is uniform linear motion; for area reconnaissance intent, the expected trajectory is back-and-forth motion within a specific area; for escape and evasion intent, the expected trajectory is rapid departure from the current position. Dynamic Time Warping (DTW) is used for calculation. The similarity between the actual trajectory and the reference trajectories for each type of intent is also considered; the higher the similarity, the higher the matching degree.

[0135] Airspace position verification score: assesses the reasonableness of the current position relative to the intent. It is calculated using preset regions of interest (such as target buildings, no-fly zone boundaries) and spatial semantic rules of the intent: for reconnaissance intents, a higher score is given when the current position is close to the region of interest and a lower score when it is far away; for escape intents, a higher score is given when the current position is far from the no-fly zone and a lower score is given when it is close; for normal cruising intents, a higher score is given when the position is within permitted airspace and a lower score is given when it is within the no-fly zone. The distance between the position and the corresponding region is calculated and mapped to a matching score within the [0,1] interval.

[0136] Interference Feature Correlation: Determines the correlation between environmental interference and intent. This is achieved by analyzing the relationship between obstacle locations, electromagnetic interference source distribution, and intent patterns in environmental information: For evasion intents, a high correlation occurs if the target trajectory is related to obstacle locations (e.g., detour); for active interference intents, a high correlation occurs if the target is close to the interference source. A correlation score is output by calculating the correlation between the target location and environmental features.

[0137] Weighted fusion and output:

[0138] By combining multiple dimensions of indicators, the confidence score of the intent candidate is calculated through weighted fusion. :

[0139] ;

[0140] in, For motion state matching degree, For spatial location verification, For interference feature correlation, , and All weights are preset. These preset weights can be adjusted according to the actual scenario. Output the confidence score for each intent candidate. This is used for the intent decision output in step 4-3.

[0141] 4-3. Intended Decision Output

[0142] Using the confidence ranking result output in step 4-2 as input, select the intent with the highest confidence as the current intent prediction result, and output the intent category and confidence score. And the prediction basis (key feature matching degree). When the highest confidence level is lower than the threshold, "intent is uncertain" is output, triggering more intensive observations. This feedback signal is passed to the feedback optimization mechanism in step 5.

[0143] Step 5: Construction and Optimization of the Collaboration Framework

[0144] A collaborative framework for interference-resistant tracking, low-altitude world modeling, and intent awareness is constructed to achieve end-to-end collaboration. The low-altitude world model employs the Joint Embedding Prediction Architecture (JEPA) as the backbone network to learn predictable joint embedding representations.

[0145] Specifically, JEPA maps the visual features of the tracking module, the state sequence of the world model, and the prediction results of intent perception to a unified embedding space through an encoder, and learns a prediction model from historical embedding to future embedding, so that the representations of the three modules remain consistent in the embedding space, thereby achieving end-to-end collaboration.

[0146] 5-1. Forward Information Transmission

[0147] Establish forward information flow:

[0148] Tracking module → World model: Outputs target location Visual features and Tracking confidence ;

[0149] World Model → Intent Awareness: Output a structured target state sequence Context and behavioral patterns;

[0150] Intent perception → Counter-drone perception and decision-making (Step 6 implementation): Output the drone intent category and confidence score to support counter-drone decision-making, such as... Figure 3 As shown.

[0151] 5-2. Feedback Optimization Mechanism

[0152] Establish a feedback and optimization mechanism:

[0153] Intent → Tracking Feedback: When the intent is "active interference" or "escape and avoidance", increase the tracking frequency and enable high-frequency local tracking mode; when the intent is "normal cruise", reduce the tracking frequency to save computing resources.

[0154] Uncertainty → Model Feedback: When the confidence score of the intention prediction is output in step 4-3 Below the preset prediction uncertainty threshold When the uncertainty of the intention prediction is high, the world model parameters are adaptively updated, the weight of the observed data is increased, and the state is re-estimated.

[0155] The specific method for adaptive parameter updating is as follows:

[0156] When model feedback is triggered, the world model uses gradient descent, taking the current observation data as input, to increase the weights in its loss function and re-estimate the state sequence. This reduces forecast uncertainty.

[0157] Specifically, the weights of the loss function are adjusted to :

[0158] ;

[0159] in, For adaptive weighting coefficients, For observation-based loss, This is the loss based on prediction.

[0160] Tracking failure → System feedback: When the tracking confidence score output in steps 2-5... Below the preset tracking failure threshold And multiple consecutive frames below the threshold When a tracking break occurs, the intent-aware module predicts the possible areas where the target might appear based on historical data, guiding the re-capture.

[0161] Step 6: Experimental Verification and Evaluation

[0162] By employing granular anti-interference tracking, a spatial semantic augmented world model, and intelligent intent perception, the tracking stability and intent recognition capabilities of the anti-drone perception system in air-to-air scenarios are significantly improved.

[0163] To verify the effectiveness of the proposed method in target tracking and intent recognition tasks, this paper conducts comparative experiments with several mainstream baseline methods on publicly available benchmark datasets. A dual-dynamic air-to-air dataset of unmanned aerial vehicles (UAVs) is used, and the AirSim simulator is employed to supplement the missing scenarios in the original dataset (complex urban environments, extreme weather, etc.). For the target tracking task, advanced tracking methods such as OR-TrackSiamRPN++, SiamRPN++TransT, and TransT-OSTrack are selected as baselines, with mean accuracy (mACC) and multi-target tracking accuracy (MOTA) as the core evaluation metrics. For the intent recognition task, mainstream multimodal large models such as InternVL2-4B, GLM-4v-9b, and Qwen2-VL-2B-Instruct are selected as baselines, with BLEU (text generation quality) and intent recognition accuracy (ACC) as evaluation metrics.

[0164] The results of the tracker comparison experiment are shown in the table below:

[0165] method mACC MOTA ORTrack 0.2787 0.2483 SiamRPN++ 0.1954 0.1837 TransT 0.272 0.2364 This invention 0.4123 0.3145

[0166] The results of the intent recognition comparison experiment are shown in the table below:

[0167] method BLEU ACC InternVL2-4B 0.1496 0.6054 GLM-4v-9b 0.1825 0.6829 Qwen2-VL-2B-Instruct 0.2056 0.6703 This invention 0.3375 0.7852

[0168] Experiments show that the method of anti-drone perception using a spatial semantic augmented world model, as proposed in this invention, significantly improves perception accuracy and cognitive depth in complex dual-dynamic scenarios compared to traditional tracking methods and general large models. The collaborative framework effectively integrates the advantages of each module, exhibiting stronger robustness and generalization ability in complex adversarial scenarios, providing an effective technical solution for intelligent anti-drone systems.

Claims

1. An intelligent sensing method for countering unmanned aerial vehicles (UAVs) based on a low-altitude world model, characterized in that, Includes the following steps: Step 1: Preprocess the first-person view video data of the UAV, extract multimodal features and perform alignment and fusion to generate a unified multimodal feature representation; Multimodal features include visual features, text features, and state features; Step 2: Construct a granular anti-interference tracking module, which achieves stable target tracking in dual dynamic scenes through visual-text feature dual enhancement, spatial-channel joint attention control, local attention mechanism and background interference suppression; Step 3: Construct a hierarchical low-altitude world model with enhanced airspace semantics. The hierarchical low-altitude world model includes a multi-source input hierarchical coding layer, a physical rule hierarchical embedding layer, and a cognitive depth hierarchical output layer. Multi-source information is aggregated into unified tokens. The UAV dynamics model and low-altitude environmental constraints are embedded in the physical rule hierarchical embedding layer, and a structured target state sequence is output in the cognitive depth hierarchical output layer. Step 4: Based on the structured target state sequence output by the hierarchical low-altitude world model, the UAV flight intention is output through intention candidate generation and confidence evaluation; Step 5: Construct a collaborative framework for granular anti-interference tracking, hierarchical low-altitude world model, and intent perception, and achieve end-to-end collaboration through forward information transmission and feedback optimization mechanisms; Step 6: Input the current frame video data into the trained collaborative framework, and output the target position, state sequence and flight intention for anti-drone decision-making.

2. The method as described in claim 1, characterized in that, Step 2 further includes: Step 2-1: Extract multi-scale visual features from video frames using a visual Transformer, generate semantic features by encoding the target description using a text encoder, and combine template features to achieve dual enhancement of visual and text features; Step 2-2: Use the inter-layer attention mechanism to align the network to achieve progressive feature alignment between visual and textual features; Steps 2-3: In the Backbone global detection process, a spatial-channel joint attention mechanism is introduced to dynamically adjust the feature weights for rapid scale changes and motion blur, and output attention-weighted features; Steps 2-4: Perform global target detection based on the attention-weighted features, and combine the DeepSort multi-target tracking algorithm and confidence feedback mechanism to form a granular mechanism for global detection to local tracking, outputting the target position and tracking confidence. Steps 2-5: Construct a positive and negative sample library, filter out interfering background features through feature matching, and output the filtered target location and tracking confidence.

3. The method as described in claim 2, wherein the method is characterized in that, In steps 2-3, the spatial-channel joint attention mechanism includes: Spatial attention employs a coordinate attention mechanism, aggregating features along the horizontal and vertical directions respectively to accurately locate the target's spatial position; Channel attention employs an efficient channel attention mechanism, which captures inter-channel dependencies through one-dimensional convolution and filters discriminative feature channels; The channel attention weights are multiplied one by one with the feature map of the spatial attention output to obtain the attention-weighted features.

4. The method as described in claim 1, characterized in that, In step 3, the multi-source input hierarchical encoding layer encodes visual features, text descriptions, location bounding boxes, tracking confidence, and the UAV's own state into unified tokens; the physical rule hierarchical embedding layer embeds the UAV flight dynamics model and low-altitude airspace environmental constraints respectively; and the cognitive depth hierarchical output layer outputs a structured target state sequence consisting of motion state, behavior state, and environmental interaction state according to cognitive depth.

5. The method as described in claim 4, characterized in that, In step 3, the dual-enhanced fusion features and attention-weighted features output by the tracking module are used as visual feature tokens, the filtered target position output by steps 2-5 is used as a location bounding box token, the filtered tracking confidence output by steps 2-5 is used as a tracking confidence token, and the text description information is used as the environmental context.

6. The method as described in claim 1, characterized in that, Step 4 further includes: Step 4-1: Based on the structured target state sequence and environmental context output by the hierarchical low-altitude world model, and combined with preset intent query instructions, generate multi-dimensional intent candidates through visual question answering; Step 4-2: Evaluate the confidence of intent candidates using multi-dimensional indicators such as motion state matching degree, spatial position verification degree, and interference feature correlation degree; Step 4-3: Select the intent with the highest confidence as the current intent prediction result, and output the intent category and confidence score.

7. The method as described in claim 6, characterized in that, In step 4-1, dynamics and environmental constraints are embedded in the calculation of attention weights in the cross-attention mechanism. An attention mask matrix is ​​generated based on the UAV dynamics model and low-altitude environmental constraints. Attention weights that do not conform to physical rules or environmental restrictions are reset to zero, and features that conform to physical rules are selected.

8. The method as described in claim 1, characterized in that, In step 5, the collaborative framework adopts a joint embedding prediction architecture as the backbone network. The encoder maps the visual features of the tracking module, the state sequence of the world model, and the prediction results of intent perception to a unified embedding space, and learns a prediction model from historical embedding to future embedding.

9. The method as described in claim 1, characterized in that, In step 5, the feedback optimization mechanism includes: When the confidence score of the intended prediction is lower than the preset threshold, the world model parameters are adaptively updated, the weight of the observed data is increased, and the state sequence is re-estimated. When the tracking confidence level is lower than a preset threshold and remains below the threshold for multiple consecutive frames, it is determined as a tracking break. The intent perception module predicts the possible areas where the target may appear based on historical states and guides re-capture.

Images