Multi-mode fusion anti-UAV detection and identification method and device adapted to low, slow and small targets

By constructing a multi-sensor perception network, event camera scanning detection, intermodal attention mechanism, and dynamic fusion weights, the problems of insufficient fusion and environmental adaptability in the detection and identification of low, slow, and small UAV targets are solved, and efficient and robust target identification and threat assessment are achieved.

CN123069246APending Publication Date: 2026-11-10SHENZHEN EWARE INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202610997956.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-06
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing technologies for detecting and identifying low-speed, small-target UAVs suffer from insufficient multi-sensor fusion, fixed fusion weights, lack of dynamic adjustment capabilities for environmental and target states, and lack of end-to-end optimization, resulting in insufficient robustness and adaptability in identification.

Method used

A three-dimensional perception network is constructed using multiple types of sensors to achieve time synchronization and spatial coordinate unification. An event camera is used for rapid scanning detection and to drive staring imaging. Information complementarity and enhancement are achieved through intermodal attention mechanism. Combined with radar micro-Doppler analysis and temporal modeling, fusion weights are dynamically calculated and updated online and self-repaired through incremental learning strategy.

Benefits of technology

It achieves deep heterogeneous fusion of radar, optoelectronic, radio, and event camera, improving recognition robustness and system adaptability in harsh environments, establishing a full-link closed-loop evolution capability, and improving the accuracy and continuous performance of target recognition.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123069246A_ABST
    Figure CN123069246A_ABST
Patent Text Reader

Abstract

This invention belongs to the field of anti-drone detection and identification, and relates to a multi-mode fusion anti-drone detection and identification method and device adapted to low, slow, and small targets. The method includes: deploying multiple types of sensors and synchronizing them in time and space; using event camera asynchronous streams for rapid scanning detection, driving staring imaging to confirm targets; independently extracting features from each modality and enhancing them across modalities through an attention mechanism; calculating fusion weights based on real-time confidence and environmental dynamics, and identifying target types after weighted fusion; extracting micro-Doppler features from radar echo time-frequency analysis, combining trajectory prediction to determine intent, and outputting threat levels; recording full-link data to build an incremental sample library, updating the model online incrementally, and achieving self-repair through performance monitoring and anomaly detection. It achieves deep heterogeneous fusion of radar, electro-optical, radio, and event camera; introduces dynamic confidence weighting to replace fixed weights; and establishes closed-loop evolution and self-repair capabilities to improve the robustness and long-term availability of detection and identification.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of anti-drone detection and identification technology, and in particular to a multi-mode fusion anti-drone detection and identification method and apparatus adapted to low, slow and small targets. Background Technology

[0002] Low-Slow-Small (LSS) targets typically refer to aerial targets flying at altitudes below 1000 m, speeds below 55 m / s, and radar cross sections (RCS) less than 2 m². With the increasing prevalence of small drones in aerial photography, logistics, and surveying, unauthorized and erratic flights are becoming increasingly prominent, posing a serious security threat to sensitive areas such as airport airspace, nuclear power plants, large event venues, and military bases. Traditional single-sensor detection methods each have significant limitations: radar, while possessing all-weather and long-range detection capabilities, suffers a sharp decrease in detection range against low-RCS targets and is susceptible to ground clutter interference; visible light cameras offer high resolution but are severely limited by lighting conditions; infrared thermal imaging can operate day and night but suffers from low contrast when target thermal radiation is weak; and radio spectrum detection is ineffective against silent or frequency-hopping targets. Therefore, multi-mode fusion detection and identification has become the core technological direction for counter-drone systems.

[0003] Existing research has made phased progress in multi-source data fusion, AI recognition, and collaborative command and control. Open-source Anti-UAV perception and understanding libraries integrate dual-mode tracking frameworks such as SiamFusion, EDTC, and SiamDT, achieving effective fusion of visible light and infrared features. Some have constructed fully automated systems for detection-recognition-response, with CNN models achieving a recognition accuracy of 99.2%. Others have proposed LSS target detection schemes based on the collaboration of event cameras and RGB cameras, employing a design philosophy of first dynamic differential imaging scanning detection followed by traditional imaging focusing and staring recognition. However, existing research still has the following shortcomings: First, multi-sensor fusion is mostly limited to pairwise pairing, and deep heterogeneous fusion of radar, optoelectronics, radio, and events is insufficient. Second, fusion weights are mostly fixed presets, lacking the ability to dynamically adjust based on the environment and target state. Third, the entire link from detection to recognition lacks a closed-loop evolution mechanism, making it impossible to continuously optimize model performance using actual deployment data. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a multi-mode fusion anti-drone detection and identification method adapted to low, slow, and small targets, employing the following technical solution, including: Deploy multiple types of sensors to build a three-dimensional sensing network, and synchronize the data collected by each sensor in time and unify the spatial coordinates to output spatiotemporally aligned multimodal sensing data; Using the asynchronous event stream of the event camera, a fast scanning detection of moving targets is performed, and the detection results drive a traditional staring imaging system to perform high-resolution confirmation imaging of the target area, outputting the target candidate area and its location information. Each modal sensing data is extracted independently, and cross-modal information complementarity enhancement is achieved through inter-modal attention mechanism, outputting multi-scale feature maps after modal enhancement; The fusion weights are dynamically calculated based on the real-time confidence levels of each sensor and environmental conditions. The enhanced features of each modality are then weighted and fused according to the fusion weights. Target type identification is performed based on the fused comprehensive features, and target category information is output. By performing time-frequency analysis on radar echoes to extract micro-Doppler features, the target structure and motion state are inverted. At the same time, the target flight trajectory is time-series modeled and predicted to determine the flight intention. The comprehensive threat level is output by combining the target type, motion state and intention information. An incremental sample library is built by recording data from the entire process. An incremental learning strategy is used to update the model and fine-tune the parameters of each stage online. Manual intervention and system self-repair are achieved through performance monitoring and anomaly detection mechanisms.

[0005] Preferably, the steps of deploying multiple types of sensors, constructing a three-dimensional sensing network, synchronizing the data collected by each sensor in time and unifying their spatial coordinates, and outputting spatiotemporally aligned multimodal sensing data specifically include: A hybrid deployment architecture combining fixed backbone nodes and mobile blind spot filling nodes is adopted to configure radar, photoelectric, radio spectrum monitoring, acoustic sensors and event cameras to eliminate single sensor coverage blind spots; Sub-microsecond time synchronization of data collected by various sensors is achieved through a precise time protocol, and data from different sensor coordinate systems are mapped to a unified global coordinate system through joint calibration or target trajectory association methods. The system calculates the signal-to-noise ratio, target contrast, data integrity rate, and time consistency deviation of each sensor under the current environmental conditions in real time, and outputs the real-time confidence factor of each sensor.

[0006] Preferably, the step of using an asynchronous event stream from an event camera to perform rapid scanning detection of moving targets, and driving a traditional staring imaging system to perform high-resolution confirmation imaging of the target area based on the detection results, and outputting the target candidate area and its location information specifically includes: The asynchronous event stream output by the event camera within a preset time window is accumulated according to pixel position and positive information to construct an event image in the form of a regular grid tensor. The event image is input into a lightweight object detection network, which outputs a set of candidate bounding boxes and their confidence scores for the moving target within a preset delay time. The required pitch and yaw angles of the gimbal are calculated based on the pixel coordinates of the target in the event image. The visible light camera is then driven to aim at the target area for zoom staring imaging, and the staring image is subjected to high-precision target detection and confirmation.

[0007] Preferably, the steps of independently extracting features from each modal sensing data, achieving cross-modal information complementarity and enhancement through an inter-modal attention mechanism, and outputting multi-scale feature maps enhanced in each modality specifically include: Independent feature extraction networks are designed for three modalities: visible light, infrared, and event images. The channel dimension is divided into multiple feature groups for group convolution operations, and a multi-scale output structure is used to output multi-level feature maps for each modality. After concatenating the feature maps of each modality along the channel dimension, channel descriptors are extracted through global pooling. Channel attention vectors are then generated through an attention network. The attention vectors are then segmented by modality and multiplied channel by channel with the original feature maps of each modality to achieve cross-modal information enhancement along the channel dimension. Calculate the activation intensity at each spatial location of each modality feature map, take the modality with the strongest activation intensity as the guiding modality, normalize its spatial activation map into spatial attention weights and apply them to the other modality feature maps to achieve cross-modal location enhancement in spatial dimension.

[0008] Preferably, the steps of dynamically calculating the fusion weights based on the real-time confidence levels of each sensor and environmental conditions, weighting and fusing the enhanced features of each modality according to the fusion weights, and identifying the target type based on the fused comprehensive features to output target category information specifically include: The overall confidence level of each mode is calculated by combining the real-time confidence level factors of each sensor and the environmental suitability factor. Then, the dynamic fusion weight of each mode is calculated by normalizing the exponential function. The enhanced feature maps of each modality are compressed into channel feature vectors through global pooling, and then weighted and summed according to the dynamic fusion weights to obtain a comprehensive feature vector. Finally, the feature transformation layer compresses the vector into a compact comprehensive feature. The compact integrated features are input into a multi-layer fully connected classification network, and the probability distribution of the target belonging to each category is output through a normalized exponential function. The target type identification result and its credibility are output based on the probability distribution.

[0009] Preferably, the steps of extracting micro-Doppler features by performing time-frequency analysis on radar echoes to invert the target structure and motion state, simultaneously performing time-series modeling and prediction of the target flight trajectory to determine flight intent, and outputting a comprehensive threat level by integrating target type, motion state, and intent information specifically include: Time-frequency analysis of radar baseband signal is performed to obtain time-spectrum diagram. Main Doppler frequency shift and micro-Doppler sideband features are extracted from the time-spectrum diagram. Target rotor speed and blade number parameters are inverted based on micro-Doppler sideband frequency intervals. Using the target position sequence obtained through continuous tracking as input, the system captures the temporal dependencies in the flight path through the gating mechanism of the Long Short-Term Memory Network, predicts the target's flight trajectory within a preset time period in the future, and performs collision detection between the predicted trajectory and the no-fly zone boundary to determine the intrusion intent. A multi-factor threat scoring function is constructed, which includes target type factor, motion state factor and intent factor. The comprehensive threat score is obtained by weighted summation of each factor, and multi-level threat levels are output based on the comprehensive threat score.

[0010] Preferably, the steps of recording end-to-end process data to construct an incremental sample library, using an incremental learning strategy to update and fine-tune the models at each stage online, and implementing manual intervention and system self-repair through performance monitoring and anomaly detection mechanisms specifically include: Automatically record the raw data, feature vectors, model inference results and corresponding final verification labels of the entire chain from detection to recognition, and build an incremental database containing positive and negative samples; Using the new data in the incremental sample library, the elastic weight consolidation method or the hierarchical learning rate adjustment strategy are used to incrementally fine-tune and update the parameters of the deep network model at each stage. The system monitors the detection rate, false alarm rate, identification accuracy, and average tracking time in real time. It uses statistical process control methods to monitor the time series of core performance indicators. When an indicator exceeds the control limit, an alarm is triggered and manual intervention and review are initiated at each level.

[0011] To address the aforementioned technical problems, this invention also provides a multi-mode fusion anti-drone detection and identification device adapted to low, slow, and small targets, employing the following technical solution, including: The module is used to deploy various types of sensors, build a three-dimensional sensing network, synchronize the data collected by each sensor in time and unify the spatial coordinates, and output spatiotemporally aligned multimodal sensing data. The driving module is used to perform fast scanning detection of moving targets using the asynchronous event stream of the event camera, and drive the traditional staring imaging system to perform high-resolution confirmation imaging of the target area based on the detection results, and output the target candidate area and its location information. The extraction module is used to extract features independently from each modal sensing data, and to achieve cross-modal information complementarity enhancement through the inter-modal attention mechanism, outputting multi-scale feature maps of each modality enhancement; The calculation module is used to dynamically calculate the fusion weights based on the real-time confidence of each sensor and environmental conditions, weight and fuse the enhanced features of each modality according to the fusion weights, and perform target type identification based on the fused comprehensive features, and output target category information. The judgment module is used to extract micro-Doppler features by performing time-frequency analysis on radar echoes to invert the target structure and motion state. At the same time, it performs time-series modeling and prediction of the target flight trajectory to determine the flight intention. It outputs a comprehensive threat level by combining target type, motion state and intention information. The update module is used to record data from the entire process to build an incremental sample library. It uses an incremental learning strategy to update the model and fine-tune the parameters of each stage online, and realizes manual intervention and system self-repair through performance monitoring and anomaly detection mechanisms.

[0012] To address the aforementioned technical problems, the present invention also provides a computer device that employs the technical solution described below, comprising a memory and a processor. The memory stores computer-readable instructions, and the processor, when executing the computer-readable instructions, implements the steps of the aforementioned multi-mode fusion anti-UAV detection and identification method adapted to low, slow, and small targets.

[0013] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium, which employs the technical solution described below. The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the aforementioned multi-mode fusion anti-UAV detection and identification method adapted to low, slow, and small targets.

[0014] Compared with the prior art, the present invention has the following main advantages: (1) Break through the limitations of pairwise fusion and realize deep heterogeneous fusion of radar-photoelectric-radio-event camera: By unifying the asynchronous stream of event camera with multi-source data such as radar micro-Doppler and radio spectrum into the spatiotemporal alignment framework, and using event-driven fast scanning and staring imaging linkage, the blind spots of passive sensors for silent targets are effectively supplemented, and a full-link perception system covering wide-area search-precise authentication-feature inversion is constructed.

[0015] (2) Introduce a dynamic confidence weighting mechanism to replace the fixed preset weight: By adaptively calculating the fusion weight based on the real-time environmental conditions (such as illumination, visibility, electromagnetic interference) and the working status (signal-to-noise ratio, detection rate) of each sensor, the fusion result is always tilted towards the most reliable mode, which significantly improves the recognition robustness in severe weather, strong clutter and other scenarios.

[0016] (3) Establishing closed-loop evolution and self-repair capabilities: An incremental sample library is constructed by recording data from the entire process. Incremental learning is used to update the feature extraction, recognition, and tracking models online. At the same time, a performance monitoring and anomaly detection module is embedded. When model degradation or data distribution drift is detected, fine-tuning or manual intervention is automatically triggered, enabling the system to continuously evolve during continuous deployment and avoiding performance degradation caused by training freezing. These three points together endow this method with higher environmental adaptability, fusion intelligence, and long-term availability. Attached Figure Description

[0017] To more clearly illustrate the solutions in this invention, the accompanying drawings used in the description of the embodiments of this invention will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart of an embodiment of the multi-mode fusion anti-UAV detection and identification method for adapting to low, slow, and small targets according to the present invention; Figure 2 This is a schematic diagram of a structure of an embodiment of the multi-mode fusion anti-drone detection and identification device for adapting to low, slow, and small targets of the present invention; Figure 3 This is a schematic diagram of the structure of one embodiment of the computer device of the present invention. Detailed Implementation

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used herein in the specification is for the purpose of describing particular embodiments only and is not intended to limit the invention; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings are used to distinguish different objects and not to describe a particular order.

[0020] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0021] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0022] It should be noted that the multi-mode fusion anti-drone detection and identification method adapted to low, slow and small targets provided in the embodiments of the present invention is generally executed by a server / terminal device. Correspondingly, the multi-mode fusion anti-drone detection and identification device adapted to low, slow and small targets is generally set in the server / terminal device.

[0023] It should be understood that the number of terminal devices, networks, and servers is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be used. Example

[0024] Please refer to Figure 1 The flowchart illustrates an embodiment of the multi-mode fusion anti-drone detection and identification method adapted to low, slow, and small targets according to the present invention. The multi-mode fusion anti-drone detection and identification method adapted to low, slow, and small targets includes the following steps: Step S1: Deploy multiple types of sensors to build a three-dimensional sensing network, and synchronize the data collected by each sensor in time and unify the spatial coordinates to output spatiotemporally aligned multimodal sensing data.

[0025] In this embodiment, the electronic device (e.g., a server / terminal device) on which the multi-mode fusion anti-drone detection and identification method for adapting to low, slow, and small targets runs can receive the multi-mode fusion anti-drone detection and identification request for adapting to low, slow, and small targets via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAXX connections, Zigbee connections, UWB (ultrawideband) connections, and other currently known or future-developed wireless connection methods.

[0026] In this embodiment, step S1 further includes the following steps: S11 employs a hybrid deployment architecture combining fixed backbone nodes and mobile blind spot nodes, configuring radar, photoelectric, radio spectrum monitoring, acoustic sensors, and event cameras to eliminate single-sensor coverage blind spots.

[0027] A hybrid deployment architecture combining fixed backbone nodes and mobile gap-filling nodes is adopted. The backbone nodes include: S-band or millimeter-wave (77 GHz) radar for long-range (1-5 km) target search and coarse measurement; and 360° omnidirectional radio spectrum monitoring equipment (covering 300 MHz-6000 MHz) for passively detecting UAV communication and navigation signals. Gap-filling nodes include: dual-mode optoelectronic nodes consisting of a Dynamic Vision Sensor (DVS) and a visible light camera, deployed in radar near-field blind spots (<100 m) and obscured areas such as urban canyons; an infrared thermal imager for thermal radiation detection in nighttime or low-light environments; and a microphone array for acoustic-assisted perception in low-noise areas. Each node connects to an edge computing gateway via fiber optic or 5G links, forming a four-layer cloud-network-edge-device architecture. Deployment optimization employs a sensor layout optimization model based on a genetic algorithm, with the objective function being to maximize regional coverage and minimize overlap redundancy.

[0028] The purpose of step S11 is to eliminate blind spots in single-sensor coverage and increase low-altitude sensing coverage to over 90%. By deploying radar, photoelectric, radio detection, acoustic sensors, and event cameras in a grid-like, three-dimensional manner, and utilizing spatial, temporal, and modal complementarity mechanisms, blind spots such as building shadow areas, low-lying areas, and electromagnetic interference areas are eliminated.

[0029] S12 achieves sub-microsecond time synchronization of data collected by various sensors through a precise time protocol, and maps data from different sensor coordinate systems to a unified global coordinate system through joint calibration or target trajectory association methods.

[0030] Time synchronization employs the IEEE 1588 Precision Time Protocol (PTP), synchronizing the timestamps of each sensor node via a PTP clock to achieve sub-microsecond synchronization accuracy. The radar, visible light camera, infrared camera, and event camera each acquire data at an independent frame rate, with a hardware trigger signal uniformly initiating the start time of each frame's acquisition. Spatial registration is divided into two stages: coarse registration and fine registration. Coarse registration, based on the installation positions of each sensor (latitude, longitude, altitude, and orientation angle), uses rigid body transformation to convert the radar polar coordinate system, camera pixel coordinate system, and radio angle of arrival coordinate system into the East-North-Up (ENU) global coordinate system.

[0031] The precise matching criterion solves for the accurate extrinsic parameters between heterogeneous sensors through a joint calibration method. For the registration of radar and optoelectronic systems, corner reflectors are used as common calibration objects. The radar measures the spherical coordinates (R, θ, φ) of the corner reflector, and the camera detects the pixel coordinates (u, v) of the same corner reflector, establishing a radar-camera mapping relationship. .

[0032] Where K is the camera intrinsic parameter matrix (3×3), R rc The rotation matrix from the radar coordinate system to the camera coordinate system is (3×3), t rc Let be the translation vector (3×1). This equation is solved by collecting n≥6 sets of calibration points, using the PnP (Perspective-n-Point) algorithm or the Levenberg-Marquardt nonlinear optimization method. rc and t rc For passive sensors without a common calibration object (such as radio direction finding equipment and radar), soft registration is performed using target trajectory association: the target trajectory observed by each sensor is matched using the Hungarian algorithm, and the spatial offset between sensors is iteratively estimated based on the criterion of minimizing the distance between trajectories.

[0033] The purpose of step S12 is to map the data from each sensor to a unified time reference and spatial coordinate system, thereby eliminating data misalignment caused by differences in sampling frequency, sensor installation location, and coordinate system, and providing a spatiotemporally consistent input for subsequent fusion.

[0034] S13 calculates in real time the signal-to-noise ratio, target contrast, data integrity rate, and time consistency deviation of each sensor under the current environmental conditions, and outputs the real-time confidence factor of each sensor.

[0035] A multi-dimensional quality assessment index system is constructed, including indicators such as signal-to-noise ratio (SNR), target contrast, data integrity rate, and temporal consistency deviation. Radar data quality assessment uses the target SNR in the range-Doppler image as the core indicator, defining SNR as... radar =10log 10 (P s / P n ), where P s For the target echo power, P n The noise floor power. When SNR radar When the signal strength is <13dB, the radar detection confidence drops below 0.5. Photoelectric image quality assessment uses a no-reference image quality evaluation index (such as the BRISQUE algorithm), outputting a quality score between 0 and 1, comprehensively evaluating sharpness, brightness, and noise levels. The validity of radio spectrum data is assessed based on the ratio of target frequency signal strength to ambient noise floor. Event camera data quality is indicated by the event occurrence rate per unit time: too low indicates insufficient scene motion information, while too high indicates excessive noise. The real-time confidence α of each sensor... i (t) is calculated using the average quality score over a rolling time window (e.g., the most recent 10 seconds) and output to the fusion layer.

[0036] The purpose of step S13 is to evaluate the data reliability of each sensor under the current environmental conditions in real time, output the confidence factor, and provide a basis for subsequent adaptive fusion weight calculation. For example, the confidence scores for infrared and visible light should be lowered under foggy weather, while the confidence score for visible light should be higher under clear daytime conditions.

[0037] The purpose of step S1 is to address the fundamental issues of multi-sensor collaborative sensing. Multi-mode fusion detection of low-speed, small targets requires heterogeneous sensors such as radar, photoelectric, radio, acoustic, and event cameras to cover the same spatial domain, sample synchronously in time, and complement each other in data dimensions. The role of step S1 is to: form a three-dimensional sensing network through reasonable sensor array configuration; map the data from each sensor to a unified coordinate system and time reference through precise spatiotemporal registration; and pre-select usable information sources through data validity assessment, providing a spatiotemporally consistent multimodal data stream for subsequent detection and fusion.

[0038] Step S2: Using the asynchronous event stream of the event camera, perform rapid scanning detection of moving targets, and drive the traditional staring imaging system to perform high-resolution confirmation imaging of the target area based on the detection results, and output the target candidate area and its location information.

[0039] In this embodiment, step S2 further includes the following steps: S21, accumulate the event stream output by the event camera within a preset time window according to the pixel position and construct an event image in the form of a regular grid tensor.

[0040] Suppose that the event camera outputs a set of events within the time interval [t0, t0+Δt]. , where x i ,y i p represents the pixel coordinates where the event occurred. i ∈{-1,+1} represents the event polarity (ON event indicates increased brightness, OFF event indicates decreased brightness), t i The event timestamps are used. Events are accumulated over a fixed time window Δt (usually 20~50 ms) to construct an event image I. event (x,y): .

[0041] in This is an indicator function, taking a value of 1 when the condition is true and 0 otherwise. The physical meaning of this formula is: within a fixed time window, it accumulates the sum of the polarities of all events at each pixel location; a positive value indicates a net increase in brightness at that location, and a negative value indicates a net decrease in brightness. This can be further extended to a two-channel event image. ; .

[0042] The spatial distributions of positive and negative events are encoded separately. The choice of the time window Δt requires a trade-off: a shorter Δt preserves higher temporal resolution but results in sparser events, while a longer Δt increases information density but may introduce motion blur. In practice, multi-scale time windows (e.g., 10 ms, 20 ms, 50 ms) are used to generate multiple frames of event images, which are then fed into the detection network for multi-timescale feature extraction.

[0043] The purpose of step S21 is: the raw output of the event camera is an asynchronous sparse event stream, with a single event (x) i ,y i ,p i ,t i The data only contains location, polarity, and time information, which cannot be directly input into a convolutional neural network for processing. This step transforms the sparse event stream into a regular grid event image through time window accumulation, giving the event data a tensor form suitable for deep learning.

[0044] S22, the event image is input into a lightweight target detection network, and the candidate bounding box set and its confidence score of the moving target are output within a preset delay time.

[0045] Design a lightweight object detection network. The backbone network adopts lightweight architectures such as MobileNetV3 or ShuffleNetV2 to reduce computational cost. The network input is an event image. (Dual-channel ON / OFF event image), output is a set of target bounding boxes. , where x k ,y k w represents the center coordinates of the bounding box. k ,h k For width and height, c k For category confidence, s k The target scale is [scale value]. A single-stage detection head (such as YOLOv8-nano or SSD-Lite) is used, and the loss function is a weighted sum of classification loss and regression loss. .

[0046] Where FL(⋅,⋅) is Focal Loss, used to alleviate the imbalance between positive and negative samples; CIoU(⋅,⋅) is Complete Intersection over Union loss, used for bounding box regression; λ cls and λ reg represents the weight coefficients. The network is trained using a labeled event image dataset, covering drone event images under different backgrounds (sky, buildings, trees), lighting conditions (strong light, weak light, backlight), and target scale conditions.

[0047] The purpose of step S22 is to quickly detect candidate regions of moving targets in the event image, achieving preliminary localization during the scanning phase. This detector must meet low latency requirements (<50 ms) to ensure timely triggering of the gimbal pointing.

[0048] S23, calculate the required pitch and yaw angles of the gimbal based on the pixel coordinates of the target in the event image, drive the visible light camera to aim at the target area for zoom staring imaging, and perform high-precision target detection and confirmation on the staring image.

[0049] Establish a spatial mapping between the event camera coordinate system and the gimbal control coordinate system. The event camera detects the pixel coordinates (u) of the target in the event image. event , v event After that, the event camera-gimbal mapping matrix M obtained through calibration is... e→p (3×3), calculate the required pitch angle θ of the gimbal. pan and yaw angle ϕ tilt : .

[0050] The gimbal response time is typically <100 ms. The visible light camera's field of view is aligned with the target area. Subsequently, the visible light camera initiates continuous zoom, automatically adjusting the focal length according to the size of the detection frame, ensuring the target occupies an appropriate scale (approximately 50-200 pixels) on the imaging plane. This is used for visible light images. High-performance detection networks (such as YOLOv8 or RT-DETR) are used for precise detection. This detection network extracts multi-scale features using a Feature Pyramid Network (FPN) structure, enhancing sensitivity to small targets and outputting accurate bounding boxes and class labels.

[0051] The function of step S23 is to use the detection results of the event camera to drive the turntable, so that the visible light camera is aimed at the target area to perform high-resolution staring imaging, and then confirm the target's identity and precise location through a high-precision detection network, thereby realizing the staring confirmation of the scanning discovery results.

[0052] Step S2 addresses the problem of how to quickly detect targets. Traditional frame-based cameras (visible light / infrared) are limited by frame rate (typically 30~60 fps), and are prone to motion blur when the gimbal scans rapidly or the target moves at high speed; radar has insufficient detection capability for low-speed / hovering targets. Event cameras (DVS), as a novel biomimetic vision sensor, output asynchronous event streams only when pixel brightness changes exceed a threshold, offering advantages such as microsecond-level temporal resolution, approximately 120 dB dynamic range, and extremely low data redundancy. Step S2 is based on a dual-mode collaborative strategy of rapid target detection using the event camera and staring confirmation using the traditional camera. First, the event camera quickly captures moving targets within a wide field of view, triggering the gimbal to point towards the target area. Then, the high-resolution zoom imaging of the traditional camera is used for staring confirmation and detailed detection.

[0053] Step S3: Independent feature extraction is performed on the sensing data of each modality, and cross-modal information complementarity enhancement is achieved through the inter-modal attention mechanism, and multi-scale feature maps of each modality are output.

[0054] In this embodiment, step S3 further includes the following steps: S31 designs independent feature extraction networks for three modalities: visible light, infrared, and event images. It divides the channel dimension into multiple feature groups for grouped convolution operations and uses a multi-scale output structure to output multi-level feature maps for each modality.

[0055] For the infrared modality, a Feature Grouping Perception Module (FGPM) is designed, dividing the channel dimension into G groups (typically G=8 or 16). Each group performs independent convolution and pooling operations, focusing on different local features of the target (such as edges, hotspots, and spatial distribution). Finally, the features from each group are concatenated. For the visible light modality, a Spatial Feature Extraction Module (SFEM) is designed, employing Deformable Convolution (DCN) in the spatial dimension of the feature map. This allows the convolution kernel sampling points to adaptively shift according to the target shape, enhancing the feature response to irregularly shaped targets while suppressing interference from complex backgrounds. For the event modality, a Temporal Feature Aggregation (TFA) module is designed, using 3D convolution or ConvLSTM to establish temporal correlations between event images in multiple adjacent time windows, capturing information about the target's motion trajectory and velocity changes.

[0056] Each modality's independent feature extraction network employs a multi-scale output structure, using FPN or BiFPN to output feature maps at different resolution levels. Where m∈{RGB,IR,Event} represents the modality, and l∈{1,2,3} represents the scale level (high-level semantics, mid-level structure, low-level texture). Each modality feature extraction network uses the modality-specific batch normalization parameter during training to adapt to the differences in data distribution among modalities.

[0057] The purpose of step S31 is to design feature extraction networks for the three modalities of visible light, infrared, and event images, focusing on solving the difficulties of low-speed, small target imaging with few pixels and sparse features, and enhancing the feature response of small targets through a group perception mechanism.

[0058] S32, after concatenating the feature maps of each modality along the channel dimension, channel descriptors are extracted through global pooling, and channel attention vectors are generated through an attention network. The attention vectors are then segmented according to the modality and multiplied by each channel of the original feature map of each modality to achieve cross-modal information enhancement along the channel dimension.

[0059] Let F be the feature map of the same scale for the three modes. RGB , F IR , F Event ∈R C×H×W The cross-modal channel attention module first concatenates the three feature maps along the channel dimension to form F. concat =Concat(F RGB ,F IR ,F Event )∈R 3C×H× Then, global average pooling and global max pooling are used to extract global and local channel descriptors, respectively: .

[0060] Where z avg ,z max ∈R 3C The sum of the two is then fed into a two-layer channel attention network: a = σ(W2⋅δ(W1⋅(z)). avg +z max ))).

[0061] Where W1∈R (3C / r)×(3C) The weights of the dimensionality-reduced fully connected layer are given by r, which is the dimensionality reduction ratio (usually 16), and δ(⋅) is the ReLU activation function, where W2∈R. (3C)×(3C / r) For the weights of the fully connected layer in higher dimensions, σ(⋅) is the Sigmoid activation function. The output attention vector a∈R 3C Each element corresponds to a soft weight for the corresponding channel in the concatenated feature map. 'a' is divided along the channel dimension into 'a'. RGB ,a IR ,a Event ∈R CEach channel of the original feature map is multiplied by the enhanced feature map to obtain the enhanced feature map: F̃ m =a m ⊙F m , m∈{RGB,IR,Event} .

[0062] Here, ⊙ represents channel-wise scalar multiplication. This mechanism is essentially a form of soft attention, enabling the model to dynamically learn the contribution of each modality to different tasks along the channel dimension.

[0063] The purpose of step S32 is to introduce a cross-modal channel attention mechanism based on the independent feature extraction of each modality, so that the features of each modality can be adaptively enhanced or suppressed according to the complementary information provided by other modalities. For example, when the target texture is blurred in a visible light image due to backlighting, the thermal radiation information or motion edge information of the infrared or event modality can be guided by attention weights to enhance the response of the target region by the visible light features.

[0064] S33, calculate the activation intensity of each spatial location in the feature map of each modality, take the modality with the strongest activation intensity as the guiding modality, normalize its spatial activation map into spatial attention weights and apply them to the feature maps of other modalities to achieve cross-modal position enhancement in spatial dimension.

[0065] Enhanced feature maps F̃ for each modality m ∈R C×H×W Calculate the spatial activation intensity along the channel dimension: .

[0066] That is, the squared L2 norm of all channel feature vectors at each spatial location (i,j) represents the information intensity at that location. The mode with the strongest spatial activation among the three modes is taken as the guiding mode, m* = argmax. m max i,j S m For [i,j], normalize its spatial activation graph to spatial attention weights: A m* [i,j]=S m* [i,j]) / (∑ i',j' S m* [i',j'].

[0067] Then, this spatial attention weight is applied to the feature maps of the other modalities for spatial position weighting: F̃ m '[:,i,j]=A m* The mechanism [i,j]⋅F̃m enhances other modes in spatial regions where the guiding mode response is strong, while suppressing them in regions where the guiding mode response is weak. This mechanism effectively utilizes the differences in spatial responses among different modes, making the synergistic enhancement more spatially specific.

[0068] The purpose of step S33 is to address the fact that, in addition to the collaborative enhancement of the channel dimension, different modalities may also differ in spatial location—for example, the moving target region detected by the event camera may not completely overlap with the textured region in the visible light image. The spatial location collaborative guidance mechanism uses the spatial activation map of the modality with the strongest response as a guide to enhance the response of other modalities at the corresponding spatial location.

[0069] Step S3 addresses the challenge of fully extracting features from each modality and achieving cross-modal collaborative enhancement. Different sensor modalities (visible light, infrared, event, and radar) exhibit significant differences in their representation of the same target: visible light is rich in texture and color information but sensitive to illumination; infrared is sensitive to thermal radiation but dependent on ambient temperature; event sensors are sensitive to motion but lack static details; and radar provides distance and velocity information but has low spatial resolution. Within each modality, features are fully extracted through grouped convolutions and multi-scale structures. Between modalities, cross-modal attention mechanisms achieve collaborative information enhancement, ensuring that the features of each modality before fusion retain their unique information while also absorbing effective complementary information from other modalities.

[0070] Step S4: Calculate the fusion weight dynamically based on the real-time confidence of each sensor and environmental conditions, perform weighted fusion of each modal enhancement feature according to the fusion weight, and identify the target type based on the fused comprehensive feature to output the target category information.

[0071] In this embodiment, step S4 further includes the following steps: S41, calculate the comprehensive confidence of each mode by combining the real-time confidence factor of each sensor with the environmental suitability factor, and then calculate the dynamic fusion weight of each mode by normalizing the exponential function.

[0072] Environmental condition factor β env Calculations are made using real-time data acquired from meteorological and light sensors, including light intensity L. lux Visibility V vis Rainfall R rain Etc. Environmental sensitivity coefficient γ for each mode. m For a pre-calibrated constant (such as γ, which is sensitive to light intensity), RGB =0.8, infrared is sensitive to visibility γ IR =0.4, the event camera is sensitive to changes in ambient light but not to absolute illumination γ. Event =0.1). The overall confidence level of each mode is defined as: = α m (t)⋅exp(-γ m ⋅(1-β env )).

[0073] Where βenv ∈[0,1] is the normalized environmental suitability (1 represents the optimal environmental condition). Then Softmax normalization is used to calculate the fusion weight of each modality: .

[0074] where τ is the temperature coefficient (usually taken as 0.5~1.0), which controls the smoothness of the weights — the smaller τ is, the more prominent the weight of the high-confidence modality is, and the more the fusion result tends to a single modality; the larger τ is, the weights of each modality tend to be uniform. The adaptive weight ω m (t) is updated in real time and transmitted to the fusion layer.

[0075] The function of step S41 is: integrating the real-time confidence α of each sensor m (t) and the environmental condition factor β env , assigning a fusion weight ω to each modality m , so that the high-confidence modality dominates in the fusion and the low-confidence modality is suppressed.

[0076] S42, the enhanced feature map of each modality is compressed into a channel feature vector through global pooling, weighted summation is performed according to the dynamic fusion weights to obtain a comprehensive feature vector, and then the comprehensive feature vector is compressed into a compact comprehensive feature through a feature transformation layer.

[0077] The feature map F̃ of each modality m '∈R C×H×W is first compressed into a channel dimension feature vector f through global average pooling m ∈R C : .

[0078] Then weighted summation is performed according to the adaptive weights to obtain the comprehensive feature vector f fused ∈R C : f fused =∑ m∈{RGB,IR,Event} ω m (t)⋅f m .

[0079] This comprehensive feature vector encodes the category-related information of each modality at the same time, and the weight is dynamically determined by the current environment and sensor state. To further reduce feature redundancy, a learnable feature transformation layer is introduced after weighted fusion: f compact = ReLU(W proj ⋅f fused +b proj ).

[0080] where is the projection matrix, C compact < C is the compressed feature dimension, b projThe bias term is ReLU(⋅), and the activation function is ReLU(⋅). This transform layer is jointly optimized with the detection and recognition network during training to ensure that the compressed features retain the information most beneficial for classification.

[0081] The purpose of step S42 is to weight and fuse the enhanced feature maps of each modality according to adaptive weights to generate a comprehensive feature vector, providing the most information-rich and least redundant input for subsequent classification and recognition.

[0082] S43, the compact integrated features are input into a multi-layer fully connected classification network, and the probability distribution of the target belonging to each category is output through a normalized exponential function. The target type identification result and its credibility are output based on the probability distribution.

[0083] Construct a deep classification network with input f compact The output is the probability distribution of K target categories. The classification network adopts a three-layer fully connected structure with hidden layer dimensions of 512, 256, and 128, and an output layer dimension of K (the number of categories, including common drones, as well as distractors such as birds and balloons). The classifier uses the Softmax function to output probabilities: .

[0084] in Let b be the classification weight vector for the k-th class. k This is the bias term. The training objective is to minimize the cross-entropy loss. .

[0085] Where y(n,k) is the true label (one-hot encoded) of the nth sample. To predict probabilities, the training dataset must cover labeled drone data under multiple scenarios, scales, and lighting conditions, and its size should exceed 10 GB to ensure the model's generalization ability. For unknown targets, a confidence threshold T is set. conf (e.g., 0.7) When the predicted probability of all categories is lower than the threshold, the unknown target is output and manual review is triggered.

[0086] The purpose of step S43 is to classify the target type using the fused integrated feature vector, output the specific model, category and credibility of the UAV, and achieve the leap from detecting the target to identifying what the target is.

[0087] Step S4 addresses the problem of optimally fusing information from different modalities. Traditional multimodal fusion methods typically employ fixed weights or simple concatenation, ignoring the fact that sensor confidence levels change with the environment and target state. The innovation of this step lies in dynamically calculating the fusion weights based on the real-time confidence levels of each sensor, environmental conditions, and target motion state, achieving adaptive fusion that varies according to time, location, and target. The fused comprehensive feature vector is used for final target classification, recognition, and identity verification.

[0088] Step S5 involves extracting micro-Doppler features from radar echoes through time-frequency analysis to invert the target structure and motion state. Simultaneously, time-series modeling and prediction of the target flight trajectory are performed to determine flight intent. The overall threat level is output by combining target type, motion state, and intent information.

[0089] In this embodiment, step S5 further includes the following steps: S51, perform time-frequency analysis on the radar baseband signal to obtain a time-frequency spectrum, extract the main Doppler frequency shift and micro-Doppler sideband features from the time-frequency spectrum, and invert the target rotor speed and blade number parameters based on the micro-Doppler sideband frequency interval.

[0090] The radar echo signal is mixed and digitized to obtain the baseband signal s(t). A short-time Fourier transform (STFT) is performed on s(t) to obtain the time-frequency spectrum. .

[0091] Where w(τ) is the Hanning or Gaussian window function, and the window length is usually T. w =50~100 ms, to ensure a balance between time-domain and frequency-domain resolution. In the time-spectrum diagram, the main Doppler frequency shift f main It reflects the overall radial velocity of the target, while the micro-Doppler frequency shift f μ (t) reflects the modulation effect of rotating components such as the rotor, manifested as a modulation around f main The periodic sidebands. The rotor speed Ω can be determined by the frequency interval Δf of the micro-Doppler sidebands. μ Calculation: Δf μ =N blades ⋅Ω / (2π), where N blades The number of rotor blades is used. Micro-Doppler features (sideband spacing, bandwidth, modulation depth, etc.) are extracted and input into a classification network, which can distinguish between drones (significant rotor modulation features) and birds (irregular micro-Doppler) as well as different aircraft types.

[0092] The purpose of step S51 is to explain that the micro-Doppler effect generated by the rotor rotation of the UAV is a unique characteristic that distinguishes it from birds or other interfering objects. By performing time-frequency analysis on the radar echo and extracting the micro-Doppler spectrum, the structural parameters of the target, such as the number of rotors, rotation speed, and symmetry, can be retrieved to differentiate between multi-rotor UAVs, fixed-wing UAVs, and birds.

[0093] S52 takes the target position sequence obtained by continuous tracking as input, captures the temporal dependencies in the trajectory through the gating mechanism of the Long Short-Term Memory Network, predicts the target's flight trajectory in the future within a preset time period, and performs collision detection between the predicted trajectory and the no-fly zone boundary to determine the intrusion intent.

[0094] Target position sequence obtained by continuous tracking using radar and electro-optical systems As input, a Long Short-Term Memory (LSTM) network is used for time series modeling and prediction. The core unit structure of the LSTM network is as follows: i t =σ(W xi x t +W hi h t-1 +b i ); f t =σ(W xf x t +W hf h t-1 +b f ); o t =σ(W xo x t +W ho h t-1 +b o ); g t = tanh(W xg x t +W hg h t-1 +b g ); c t = f t ⊙c t-1 +i t ⊙g t h t =o t ⊙tanh(c t ).

[0095] Where x tInput (position and velocity information) at time t, h t To be in a hidden state, c t In cellular state; i t ,f t ,o t These are the activation vectors for the input gate, forget gate, and output gate, respectively; g t Let W represent the candidate cell state; W is the weight matrix of each gate; b is the bias term; σ(⋅) is the Sigmoid activation function; and tanh(⋅) is the hyperbolic tangent activation function.

[0096] LSTM networks effectively capture long-term dependencies in a trajectory through gating mechanisms, predicting future T... pred Target location of the frame: Collision detection is performed between the predicted trajectory and the no-fly zone boundary. If the predicted trajectory has a minimum distance d from the no-fly zone boundary... min Less than the safety threshold D safe If so, it is determined that there is an intent to intrude.

[0097] The purpose of step S52 is to predict the future flight trajectory of the target by performing time-series modeling of the target's historical flight path, and to determine its intrusion intent (such as entering a no-fly zone or conducting circling reconnaissance) by combining information such as no-fly zones and sensitive areas.

[0098] S53, construct a multi-factor threat scoring function that includes target type factor, motion state factor and intent factor, sum the weighted factors to obtain a comprehensive threat score, and output a multi-level threat level based on the comprehensive threat score.

[0099] Construct a multi-factor threat scoring function. Let the target type factor T be... type ∈[0,1] (the stronger the load capacity, the greater the threat), motion state factor T motion ∈[0,1] (state assignments such as speed anomaly, hovering reconnaissance, low-altitude penetration, etc.), intention factor T intent ∈[0,1] (assignment of predicted flight path relationship with no-fly zone), overall threat score: Θ=α type ⋅T type +α motion ⋅T motion +α intent ⋅T intent .

[0100] Where α type , α motion , α intent The weighting coefficient (α) type +α motion , +α intent=1), which can be dynamically adjusted according to specific scenarios (e.g., increasing α in airport scenarios). intent Increase α in security scenarios for large-scale events type The score Θ is mapped to a four-level threat level: Θ<0.25 is low risk (suspected false alarm), 0.25≤Θ<0.5 is medium risk (requires continuous monitoring), 0.5≤Θ<0.8 is high risk (requires preparation for countermeasures), and Θ≥0.8 is emergency (immediate countermeasures).

[0101] The purpose of step S53 is to integrate the target type identification results, micro-Doppler structural information, and trajectory prediction intent analysis to output a multi-level threat level (such as low risk / medium risk / high risk / emergency), providing a decision-making basis for the countermeasure system to trigger corresponding level of response measures.

[0102] Step S5 addresses the issue of how to assess the threat after identification. Target identification only determines what it is; threat assessment also requires determining its intentions. The threat level of low-altitude, slow-moving, and small targets depends on their type, payload capacity, flight trajectory, speed characteristics, and positional relationship with the no-fly zone. This step infers the target's structure and motion state through micro-Doppler feature analysis, assesses its intrusion intent through trajectory prediction and intent inference, and combines the information from both to output a comprehensive threat level, providing a basis for countermeasure decisions.

[0103] Step S6: Record the data of the entire process to build an incremental sample library, use the incremental learning strategy to update the model of each link online and fine-tune the parameters, and realize manual intervention and system self-repair through performance monitoring and anomaly detection mechanism.

[0104] In this embodiment, step S6 further includes the following steps: S61 automatically records the raw data, feature vectors, model inference results, and corresponding final verification labels of the entire chain from detection to recognition, and builds an incremental database containing positive and negative samples.

[0105] The system employs a hierarchical data storage architecture: a hot storage layer (SSD array) stores the raw data and metadata for the past 7 days for rapid playback and annotation; a warm storage layer (HDD array) stores compressed historical data packages; and a cold storage layer (cloud storage) performs long-term archiving. Each data record includes: timestamp, sensor ID, raw data pointer, preprocessed feature vector, model inference results (detection boxes, classification probabilities, tracking status, threat score), manually reviewed labels (if any), and final processing results. Data storage complies with GDPR and other privacy compliance requirements, and sensitive areas are anonymized before storage. The system supports semi-automatic annotation tools, utilizing model pre-annotation results that are manually corrected before being stored, significantly reducing annotation costs.

[0106] The purpose of step S61 is to automatically record the entire data chain from detection to identification (radar echo, event stream, visible light / infrared image, fusion features, classification results, tracking trajectory, threat score) and the corresponding final verification label (results confirmed by manual review or countermeasures), and to build an incremental database containing positive and negative samples, providing a data foundation for online learning.

[0107] S62, using the new data in the incremental sample library, the deep network model of each stage is incrementally fine-tuned and its parameters are updated by adopting the elastic weight consolidation method or the hierarchical learning rate adjustment strategy.

[0108] Incremental learning is performed using either Elastic Weight Consolidation (EWC) or a hierarchical learning rate adjustment strategy. The EWC method introduces an additional quadratic regularization term for parameters important in the old task when updating model parameters, preventing catastrophic forgetting. The loss function is defined as: .

[0109] in For the classification loss on new data, θ i For the i-th parameter, For the old model parameters, F i The i-th diagonal element of the Fisher Information Matrix measures the importance of this parameter to the old task, and λ is the regularization coefficient (usually taken as 10). 3 ~10 5 For the feature extraction layer (backbone network) of the detection network, a strategy of freezing the first L layers and only fine-tuning the parameters of the last few layers is adopted, which significantly reduces the computational cost of updates. Incremental updates are automatically triggered by background tasks during low-load periods (such as at night), and after the update is completed, the performance is verified to be no less than that of the old version through A / B testing before it is officially launched.

[0110] The purpose of step S62 is to use new data from the incremental sample library to quickly update the network weights of each stage through incremental learning or parameter-efficient fine-tuning without retraining the complete model, so as to adapt to new scenarios, new model targets and new environmental conditions.

[0111] S63 monitors the detection rate, false alarm rate, recognition accuracy and average tracking time in real time. It uses statistical process control methods to monitor the time series of core performance indicators. When the indicators exceed the control limits, an alarm is triggered and manual intervention and review are initiated at each level.

[0112] A key performance indicator (KPI) monitoring panel is set up, including: Detection Rate (DR), False Alarm Rate (FAR), Accuracy, Mean Track Duration (MTD), and Threat Assessment Response Time. Adaptive thresholds are set for each indicator—an alert is triggered when the detection rate continuously decreases by more than 5% or the false alarm rate continuously increases by more than 3%. Statistical Process Control (SPC) is used, employing a Moving Average Control Chart (MA-CC) to monitor the time series of core indicators such as the detection rate. An anomaly alarm is triggered when an indicator exceeds the 3σ control limit. The system is designed with three levels of manual intervention interfaces: Level 1 is alarm push notifications (notifying operators via SMS, email, and large screen prompts); Level 2 is remote verification (operators view raw data and model inference results through the system to confirm whether it is a false alarm or a missed detection, and mark correct results for storage); Level 3 is emergency takeover (when a system-level fault or an unknown high-threat target is confirmed, the operator manually takes over PTZ control and countermeasure decisions). This mechanism ensures that while the system has the ability to evolve automatically, it always retains the final decision-making and oversight rights of humans.

[0113] The purpose of step S63 is to monitor the performance indicators of each link of the online system in real time, automatically detect performance degradation or abnormal fluctuations, trigger manual intervention when the automatic model update fails or encounters unknown threats, and ensure the reliability and security of the system during the evolution process.

[0114] Step S6 addresses the issue of continuous evolution. Static models cannot adapt to the constantly changing environmental conditions, new drone models, and unknown adversarial strategies in deployment scenarios. This step constructs a closed-loop chain of detection → identification → tracking → evaluation → feedback → update, utilizing incremental data accumulated during actual operation to continuously optimize the model at each stage without affecting online inference, thereby achieving the system's self-evolution capability.

[0115] This invention can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0116] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0117] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0118] Example 2 Further reference Figure 2 As a response to the above Figure 1 The present invention provides an embodiment of a multi-mode fusion anti-drone detection and identification device adapted to low, slow, and small targets, based on the implementation of the method shown. This device embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0119] like Figure 2As shown, the multi-mode fusion anti-UAV detection and identification device 70 adapted to low, slow, and small targets described in this embodiment includes: a construction module 71, a driving module 72, an extraction module 73, a calculation module 74, a judgment module 75, and an update module 76. Wherein: Module 71 is used to deploy various types of sensors, build a three-dimensional sensing network, synchronize the data collected by each sensor in time and unify the spatial coordinates, and output spatiotemporally aligned multimodal sensing data. The driving module 72 is used to perform fast scanning detection of moving targets using the asynchronous event stream of the event camera, and drive the traditional staring imaging system to perform high-resolution confirmation imaging of the target area based on the detection results, and output the target candidate area and its location information. The extraction module 73 is used to extract features independently from each modal sensing data, and to achieve cross-modal information complementarity enhancement through the intermodal attention mechanism, and output multi-scale feature maps after each modal enhancement. The calculation module 74 is used to dynamically calculate the fusion weight based on the real-time confidence of each sensor and environmental conditions, perform weighted fusion of each modal enhancement feature according to the fusion weight, and perform target type identification based on the fused comprehensive feature, and output target category information. The judgment module 75 is used to extract micro-Doppler features by performing time-frequency analysis on radar echoes to invert the target structure and motion state, and at the same time perform time-series modeling and prediction of the target flight trajectory to judge the flight intention. It outputs a comprehensive threat level by combining target type, motion state and intention information. The update module 76 is used to record data from the entire process to build an incremental sample library. It uses an incremental learning strategy to update the model and fine-tune the parameters of each stage online, and realizes manual intervention and system self-repair through performance monitoring and anomaly detection mechanisms.

[0120] Example 3 To address the aforementioned technical problems, embodiments of the present invention also provide a computer device. Please refer to [link / reference needed]. Figure 3 , Figure 3 This is a basic structural block diagram of the computer device in this embodiment.

[0121] The aforementioned computer device 8 includes a memory 81, a processor 82, and a network interface 83 that are interconnected via a system bus. It should be noted that only the computer device 8 with components 81, 82, and 83 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described herein is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0122] The aforementioned computer devices can be desktop computers, laptops, handheld computers, and cloud servers, among other computing devices. These devices can facilitate human-computer interaction with users through keyboards, mice, remote controls, touchpads, or voice-activated devices.

[0123] The aforementioned memory 81 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the aforementioned memory 81 may be an internal storage unit of the aforementioned computer device 8, such as the hard disk or memory of the computer device 8. In other embodiments, the aforementioned memory 81 may also be an external storage device of the aforementioned computer device 8, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 8. Of course, the aforementioned memory 81 may also include both the internal storage unit and its external storage device of the aforementioned computer device 8. In this embodiment, the aforementioned memory 81 is typically used to store the operating system and various application software installed on the aforementioned computer device 8, such as computer-readable instructions for a multi-mode fusion anti-drone detection and identification method adapted to low, slow, and small targets. In addition, the aforementioned memory 81 can also be used to temporarily store various types of data that have been output or will be output.

[0124] In some embodiments, the processor 82 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 82 is typically used to control the overall operation of the computer device 8. In this embodiment, the processor 82 is used to execute computer-readable instructions stored in the memory 81 or to process data, for example, to execute the computer-readable instructions of the multi-mode fusion anti-UAV detection and identification method adapted to low, slow, and small targets.

[0125] The network interface 83 may include a wireless network interface or a wired network interface, which is typically used to establish a communication connection between the computer device 8 and other electronic devices.

[0126] Example 4 The present invention also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the multi-mode fusion anti-drone detection and identification method adapted to low, slow and small targets as described above.

[0127] The beneficial effects of implementing the above embodiments are as follows: (1) Break through the limitations of pairwise fusion and realize deep heterogeneous fusion of radar-photoelectric-radio-event camera: By unifying the asynchronous stream of event camera with multi-source data such as radar micro-Doppler and radio spectrum into the spatiotemporal alignment framework, and using event-driven fast scanning and staring imaging linkage, the blind spots of passive sensors for silent targets are effectively supplemented, and a full-link perception system covering wide-area search-precise authentication-feature inversion is constructed.

[0128] (2) Introduce a dynamic confidence weighting mechanism to replace the fixed preset weight: By adaptively calculating the fusion weight based on the real-time environmental conditions (such as illumination, visibility, electromagnetic interference) and the working status (signal-to-noise ratio, detection rate) of each sensor, the fusion result is always tilted towards the most reliable mode, which significantly improves the recognition robustness in severe weather, strong clutter and other scenarios.

[0129] (3) Establishing closed-loop evolution and self-repair capabilities: An incremental sample library is constructed by recording data from the entire process. Incremental learning is used to update the feature extraction, recognition, and tracking models online. At the same time, a performance monitoring and anomaly detection module is embedded. When model degradation or data distribution drift is detected, fine-tuning or manual intervention is automatically triggered, enabling the system to continuously evolve during continuous deployment and avoiding performance degradation caused by training freezing. These three points together endow this method with higher environmental adaptability, fusion intelligence, and long-term availability.

[0130] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0131] Obviously, the embodiments described above are merely some embodiments of the present invention, not all embodiments. The accompanying drawings show preferred embodiments of the present invention, but do not limit the patent scope of the present invention. The present invention can be implemented in many different forms; rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the patent protection scope of this invention.

Claims

1. A multi-mode fusion anti-UAV detection and identification method adapted to low-speed, small targets, characterized in that, Includes the following steps: Deploy multiple types of sensors to build a three-dimensional sensing network, and synchronize the data collected by each sensor in time and unify the spatial coordinates to output spatiotemporally aligned multimodal sensing data; Using the asynchronous event stream of the event camera, a fast scanning detection of moving targets is performed, and the detection results drive a traditional staring imaging system to perform high-resolution confirmation imaging of the target area, outputting the target candidate area and its location information; Each modal sensing data is extracted independently, and cross-modal information complementarity enhancement is achieved through inter-modal attention mechanism, outputting multi-scale feature maps after modal enhancement; The fusion weights are dynamically calculated based on the real-time confidence levels of each sensor and environmental conditions. The enhanced features of each modality are then weighted and fused according to the fusion weights. Target type identification is performed based on the fused comprehensive features, and target category information is output. By performing time-frequency analysis on radar echoes to extract micro-Doppler features, the target structure and motion state are inverted. At the same time, the target flight trajectory is time-series modeled and predicted to determine the flight intention. The comprehensive threat level is output by combining the target type, motion state and intention information. An incremental sample library is built by recording data from the entire process. An incremental learning strategy is used to update the model and fine-tune the parameters of each stage online. Manual intervention and system self-repair are achieved through performance monitoring and anomaly detection mechanisms.

2. The multi-mode fusion anti-UAV detection and identification method adapted to low, slow, and small targets according to claim 1, characterized in that, The steps of deploying multiple types of sensors to construct a three-dimensional sensing network, synchronizing the data collected by each sensor in time and unifying their spatial coordinates, and outputting spatiotemporally aligned multimodal sensing data specifically include: A hybrid deployment architecture combining fixed backbone nodes and mobile blind spot filling nodes is adopted to configure radar, photoelectric, radio spectrum monitoring, acoustic sensors and event cameras to eliminate single sensor coverage blind spots; Sub-microsecond time synchronization of data collected by various sensors is achieved through a precise time protocol, and data from different sensor coordinate systems are mapped to a unified global coordinate system through joint calibration or target trajectory association methods. The system calculates the signal-to-noise ratio, target contrast, data integrity rate, and time consistency deviation of each sensor under the current environmental conditions in real time, and outputs the real-time confidence factor of each sensor.

3. The multi-mode fusion anti-UAV detection and identification method adapted to low, slow, and small targets according to claim 1, characterized in that, The steps of using an asynchronous event stream from an event camera to perform rapid scanning detection of moving targets, and driving a traditional staring imaging system to perform high-resolution confirmation imaging of the target area based on the detection results, and outputting the target candidate area and its location information, specifically include: The asynchronous event stream output by the event camera within a preset time window is accumulated according to pixel position and positive information to construct an event image in the form of a regular grid tensor. The event image is input into a lightweight object detection network, which outputs a set of candidate bounding boxes and their confidence scores for the moving target within a preset delay time. The required pitch and yaw angles of the gimbal are calculated based on the pixel coordinates of the target in the event image. The visible light camera is then driven to aim at the target area for zoom staring imaging, and the staring image is subjected to high-precision target detection and confirmation.

4. The multi-mode fusion anti-UAV detection and identification method adapted to low, slow, and small targets according to claim 1, characterized in that, The steps of independently extracting features from each modal sensing data, achieving cross-modal information complementarity and enhancement through an inter-modal attention mechanism, and outputting enhanced multi-scale feature maps for each modality specifically include: Independent feature extraction networks are designed for three modalities: visible light, infrared, and event images. The channel dimension is divided into multiple feature groups for group convolution operations, and a multi-scale output structure is used to output multi-level feature maps for each modality. After concatenating the feature maps of each modality along the channel dimension, channel descriptors are extracted through global pooling. Channel attention vectors are then generated through an attention network. The attention vectors are then segmented by modality and multiplied channel by channel with the original feature maps of each modality to achieve cross-modal information enhancement along the channel dimension. Calculate the activation intensity at each spatial location of each modality feature map, take the modality with the strongest activation intensity as the guiding modality, normalize its spatial activation map into spatial attention weights and apply them to the other modality feature maps to achieve cross-modal position enhancement in spatial dimension.

5. The multi-mode fusion anti-UAV detection and identification method adapted to low, slow, and small targets according to claim 1, characterized in that, The steps of dynamically calculating the fusion weights based on the real-time confidence levels of each sensor and environmental conditions, weighting and fusing the enhanced features of each modality according to the fusion weights, and identifying the target type based on the fused comprehensive features to output target category information specifically include: The overall confidence of each mode is calculated by combining the real-time confidence factors of each sensor and the environmental suitability factor, and then the dynamic fusion weight of each mode is calculated by the normalized exponential function. The enhanced feature maps of each modality are compressed into channel feature vectors through global pooling, and then weighted and summed according to the dynamic fusion weights to obtain a comprehensive feature vector. Finally, the feature transformation layer compresses the vector into a compact comprehensive feature. The compact integrated features are input into a multi-layer fully connected classification network, and the probability distribution of the target belonging to each category is output through a normalized exponential function. The target type identification result and its credibility are output based on the probability distribution.

6. The multi-mode fusion anti-UAV detection and identification method adapted to low, slow, and small targets according to claim 1, characterized in that, The steps of extracting micro-Doppler features through time-frequency analysis of radar echoes to invert target structure and motion state, while simultaneously performing time-series modeling and prediction of target flight trajectory to determine flight intent, and outputting a comprehensive threat level by integrating target type, motion state, and intent information, specifically include: Time-frequency analysis of radar baseband signal is performed to obtain time-spectrum diagram. Main Doppler frequency shift and micro-Doppler sideband features are extracted from the time-spectrum diagram. Target rotor speed and blade number parameters are inverted based on micro-Doppler sideband frequency intervals. Using the target position sequence obtained through continuous tracking as input, the system captures the temporal dependencies in the flight path through the gating mechanism of the Long Short-Term Memory Network, predicts the target's flight trajectory within a preset time period in the future, and performs collision detection between the predicted trajectory and the no-fly zone boundary to determine the intrusion intent. A multi-factor threat scoring function is constructed, which includes target type factor, motion state factor and intent factor. The comprehensive threat score is obtained by weighted summation of each factor, and multi-level threat levels are output based on the comprehensive threat score.

7. The multi-mode fusion anti-UAV detection and identification method for adapting to low, slow, and small targets according to any one of claims 1 to 6, characterized in that, The steps of constructing an incremental sample library by recording data from the entire process, using an incremental learning strategy to update and fine-tune the models at each stage online, and implementing manual intervention and system self-repair through performance monitoring and anomaly detection mechanisms specifically include: Automatically record the raw data, feature vectors, model inference results and corresponding final verification labels of the entire chain from detection to recognition, and build an incremental database containing positive and negative samples; Using the new data in the incremental sample library, the elastic weight consolidation method or the hierarchical learning rate adjustment strategy are used to incrementally fine-tune and update the parameters of the deep network model at each stage. The system monitors the detection rate, false alarm rate, identification accuracy, and average tracking time in real time. It uses statistical process control methods to monitor the time series of core performance indicators. When an indicator exceeds the control limit, an alarm is triggered and manual intervention and review are initiated at each level.

8. A multi-mode fusion anti-UAV detection and identification device adapted to low, slow, and small targets, characterized in that, include: The module is used to deploy various types of sensors, build a three-dimensional sensing network, synchronize the data collected by each sensor in time and unify the spatial coordinates, and output spatiotemporally aligned multimodal sensing data. The driving module is used to perform fast scanning detection of moving targets using the asynchronous event stream of the event camera, and drive the traditional staring imaging system to perform high-resolution confirmation imaging of the target area based on the detection results, and output the target candidate area and its location information. The extraction module is used to extract features independently from each modal sensing data, and to achieve cross-modal information complementarity enhancement through the inter-modal attention mechanism, outputting multi-scale feature maps of each modality enhancement; The calculation module is used to dynamically calculate the fusion weights based on the real-time confidence of each sensor and environmental conditions, weight and fuse the enhanced features of each modality according to the fusion weights, and perform target type identification based on the fused comprehensive features, and output target category information. The judgment module is used to extract micro-Doppler features by performing time-frequency analysis on radar echoes to invert the target structure and motion state. At the same time, it performs time-series modeling and prediction of the target flight trajectory to determine the flight intention. It outputs a comprehensive threat level by combining target type, motion state and intention information. The update module is used to record data from the entire process to build an incremental sample library. It uses an incremental learning strategy to update the model and fine-tune the parameters of each stage online, and realizes manual intervention and system self-repair through performance monitoring and anomaly detection mechanisms.

9. A computer device, characterized in that, The method includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the multi-mode fusion anti-drone detection and identification method for low, slow and small targets as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the multi-mode fusion anti-UAV detection and identification method for adapting to low, slow, and small targets as described in any one of claims 1 to 7.