A low-altitude security-oriented unmanned aerial vehicle identification method and system

Abstract

This invention discloses a method and system for UAV identification in low-altitude security, relating to the fields of low-altitude security monitoring and target recognition. This invention utilizes motion detection, foreground separation, and prior knowledge of size and aspect ratio to eliminate a large number of background regions unrelated to the UAV in an early stage. Then, through a comprehensive weighted evaluation of three types of scores—appearance matching, motion consistency, and size prior—and an adaptive threshold strategy, it finely filters regions of interest, significantly reducing the number of candidate regions entering subsequent complex models while ensuring candidate quality. This alleviates the computational burden on graph neural networks and Transformer models from the source. Furthermore, this invention introduces a Transformer encoder-decoder at the candidate region level for global relationship modeling and decouples feature representations into category-independent shape features and category-related model features. This helps maintain strong distinguishability when multiple UAVs with similar appearances appear simultaneously, reducing misidentification caused by feature confusion.

Description

Technical Field

[0001] This invention relates to the field of low-altitude security monitoring and target recognition technology, and in particular to a method and system for identifying unmanned aerial vehicles (UAVs) for low-altitude security. Background Technology

[0002] As the application of drone technology in low-altitude security continues to expand, collaborative operation modes of multiple drones are gradually emerging in scenarios such as border patrol, urban security, and large-scale event monitoring. Multiple drones fly in relatively dense formations within limited airspace, with their flight paths intersecting and overlapping, and the appearance, size, and color characteristics of different drones are often similar. In such high-density, multi-target dynamic interaction environments, low-altitude security systems need to simultaneously detect and identify multiple drone targets. Frequent obstruction and mutual interference between targets can easily make it difficult to distinguish individual targets, thereby increasing the risk of missed detections and misidentifications.

[0003] Currently, research on multi-UAV identification and tracking largely relies on deep learning-based visual algorithms. These include convolutional neural networks or Transformer architectures that enhance feature representation through attention mechanisms, or graph neural networks that model the spatiotemporal relationships between targets and combine them with data association algorithms to achieve multi-target tracking. Some solutions also integrate visible light, infrared, or other sensor data to improve robustness in complex environments. While these methods have improved UAV target detection and tracking performance to some extent, they still have shortcomings in high-density multi-UAV swarm scenarios. On the one hand, when targets are highly similar in appearance and occlude each other, feature representations are easily confused, individual identity maintenance is unstable, and identity switching can easily occur during tracking. On the other hand, existing methods often employ global feature extraction and inference on the entire image, which significantly increases the computational load when the number of targets is large and the frame rate requirement is high, making it difficult to meet the real-time processing needs of edge computing devices and affecting the overall response efficiency of low-altitude security systems.

[0004] Furthermore, existing solutions typically lack an adaptive region-of-interest (ROI) filtering mechanism for drone targets in low-altitude security scenarios. In large-field-of-view surveillance footage, a large amount of redundant calculation is still required for areas unrelated to drones, which reduces processing efficiency and is not conducive to stably locating the position and trajectory of each drone target in complex backgrounds. Therefore, it is necessary to provide a drone identification method and system for low-altitude security scenarios that can improve identification accuracy and identity stability while reducing computational load and meeting the real-time processing requirements of edge devices. Summary of the Invention

[0005] In view of the aforementioned existing problems, the present invention is proposed.

[0006] This invention provides a method and system for identifying drones for low-altitude security, which solves the problems of frequent occlusion and similar appearance in multi-drone swarm scenarios for low-altitude security, where existing methods are prone to confusion and have high computational load.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, embodiments of the present invention provide a method for identifying unmanned aerial vehicles (UAVs) for low-altitude security, comprising: Step S1: Obtain a sequence of low-altitude area monitoring images collected by low-altitude security monitoring equipment, wherein the monitoring image sequence includes images of at least one drone; Step S2: Motion detection and / or foreground separation are performed on each frame of the monitored image sequence to determine several regions of interest (ROIs) that are suspected to contain UAV targets. Step S3: For each of the aforementioned regions of interest (ROI), crop out the candidate regions of drones from the corresponding frame images, and extract features from each candidate region of drones to obtain the feature representation of the candidate regions of drones. Step S4: Construct a spatiotemporal graph based on the candidate regions of drones and their feature representations in consecutive frames of the monitoring image sequence. Nodes in the spatiotemporal graph represent candidate regions of drones, and edges in the spatiotemporal graph represent the spatiotemporal relationships between candidate regions of drones that are correlated in spatial location and time. Step S5: Use a graph neural network to reason about the spatiotemporal graph to obtain the drone trajectory identifier of each drone candidate region; Step S6: Based on the UAV trajectory identifier, output the individual information of each UAV target, wherein the individual information includes at least the UAV's identity identifier and / or trajectory information.

[0008] As a preferred embodiment of the UAV identification method for low-altitude security described in this invention, step S2 includes: A background model is constructed based on a preset number of multi-frame surveillance images; The current frame monitoring image is compared with the background model to obtain the moving foreground region; Within the foreground region of motion, regions of interest (ROIs) that conform to the prior characteristics of the UAV are selected based on preset size and aspect ratio constraints.

[0009] As a preferred embodiment of the UAV identification method for low-altitude security described in this invention, in step S2, after obtaining the region of interest (ROI), the method further includes: Calculate the region of interest (ROI) score for each ROI. Based on the results calculated from the region of interest scores, non-maximum suppression is performed to remove overlapping regions of interest, resulting in the final UAV candidate regions. Before performing nonmaximum suppression on the region of interest, the following steps are also included: For each region of interest (ROI), a target appearance matching score, motion consistency score, and size prior score are calculated. The target appearance matching score characterizes the similarity between the appearance features of the ROI and a reference appearance template constructed based on historical high-confidence UAV samples. The motion consistency score characterizes the offset between the position of the ROI in the current frame and the position predicted by the UAV motion state in the previous frame. The size prior score characterizes the deviation of the area and aspect ratio of the ROI from the pre-statistically obtained UAV size prior distribution. The three scores are weighted and combined according to adjustable weights with a constant sum to obtain the comprehensive ROI score. A score threshold is adaptively set based on the mean and fluctuation range of the comprehensive ROI scores of all ROIs in the same frame, and only ROIs with a comprehensive ROI score not lower than the score threshold are retained as UAV candidate regions.

[0010] As a preferred embodiment of the UAV identification method for low-altitude security described in this invention, the feature extraction in step S3 includes: The Transformer encoder-decoder architecture is used to encode each UAV candidate region, and the global relationship between each UAV candidate region in the same frame and between UAV candidate regions across frames is modeled through a self-attention mechanism. The feature representation of each UAV candidate region is decoupled into category-independent shape features and category-related model features.

[0011] As a preferred embodiment of the UAV identification method for low-altitude security described in this invention, the monitoring image sequence includes a visible light image sequence and an infrared image sequence, and steps S2 and S3 include: The first region of interest (ROI) is detected in the visible light image, and the second region of interest (ROI) is detected in the infrared image. Based on the spatial relationship between the first Region of Interest (ROI) and the second Region of Interest (ROI), the corresponding ROI pairs are used as candidate regions for the UAV. Cross-modal feature fusion is performed on candidate regions of different UAV modes; The cross-modal feature fusion of candidate regions for UAVs of different modalities includes: For each pair of visible light and infrared regions of interest, visible light and infrared feature vectors are extracted respectively, and mapped to a common feature space of the same dimension through their respective linear transformations. The fusion weights of the two modes are calculated based on the quality scores of the visible light and infrared modes in the current environment. These fusion weights are obtained by normalizing the quality scores, giving the mode with the higher quality score a greater weight in the fusion result. The common space feature vectors of the two modes are weighted and summed according to the fusion weights to obtain the cross-modal fusion features of the regions of interest. Furthermore, at the feature map level, the visible light and infrared feature maps are concatenated along the channel dimension. The two modal information is further mixed spatially through one-to-one convolution and nonlinear activation. The features obtained by the above fusion are temporally aggregated across multiple frames of the same UAV trajectory and used as the feature representation of the UAV trajectory, then input into a graph neural network for trajectory association.

[0012] As a preferred embodiment of the UAV identification method for low-altitude security described in this invention, the graph neural network is a graph attention network, which adaptively models the spatiotemporal relationship between UAV candidate regions by aggregating neighborhood information based on attention weights. In the feature extraction process of step S3, a deformable convolutional layer is used to perform convolution operations on the candidate regions of the UAV, which is used to dynamically adjust the receptive field according to the shape of the UAV and the occlusion situation, focusing on the part of the UAV that is not occluded. The method further includes an incremental learning step based on online hard example mining, in which hard example samples are selected according to the UAV identification results during system operation, and the feature extraction model and / or the graph neural network are dynamically updated using the hard example samples.

[0013] Secondly, embodiments of the present invention provide a drone identification system for low-altitude security, comprising: The image acquisition module is used to acquire sequences of low-altitude area monitoring images collected by low-altitude security monitoring equipment; The region of interest (ROI) determination module is used to perform motion detection and / or foreground separation on each frame of the monitored image sequence, to determine several ROIs that are suspected to contain UAV targets, and to calculate and filter UAV candidate regions based on the ROI scores. The feature extraction module is used to extract feature representations for each candidate region of the UAV; The trajectory association module is used to construct a spatiotemporal map based on each UAV candidate region and its feature representation, and to use a graph neural network to reason about the spatiotemporal map to obtain the UAV trajectory identifier to which each UAV candidate region belongs. The output module is used to output individual information of each UAV target based on the UAV trajectory identifier.

[0014] As a preferred embodiment of the UAV identification system for low-altitude security described in this invention, the region of interest determination module is configured as follows: A background model is constructed based on a preset number of multi-frame surveillance images; The current frame monitoring image is compared with the background model to obtain the moving foreground region; Within the foreground region of motion, regions of interest (ROIs) that conform to the prior characteristics of the UAV are selected based on preset size and aspect ratio constraints.

[0015] As a preferred embodiment of the UAV identification system for low-altitude security described in this invention, the feature extraction module includes a Transformer encoder-decoder network, which is configured as follows: Each candidate region of a UAV is encoded, and the relationship between the candidate regions of each UAV is globally modeled through a self-attention mechanism. The feature representation of each UAV candidate region is decoupled into category-independent shape features and category-related model features.

[0016] As a preferred embodiment of the UAV identification system for low-altitude security described in this invention, the system is deployed on an edge computing device, which is connected to a low-altitude security monitoring device. The system reduces the computational load by limiting the number and size of Regions of Interest (ROIs) and by employing the graph neural network.

[0017] The beneficial effects of this invention are as follows: The UAV identification method and system for low-altitude security provided by this invention effectively alleviates the problems of target occlusion, similar appearance, and excessive computational load in multi-UAV swarm scenarios by introducing a hierarchical processing mechanism of interest region filtering and fine modeling in large field-of-view monitoring images.

[0018] This invention utilizes motion detection, foreground separation, and prior knowledge of size and aspect ratio to eliminate a large number of background regions unrelated to the UAV at an early stage. Then, through a comprehensive weighted evaluation of appearance matching, motion consistency, and size prior scores, along with an adaptive threshold strategy, it refines the selection of regions of interest, significantly reducing the number of candidate regions entering subsequent complex models while ensuring candidate quality. This alleviates the computational burden on graph neural networks and Transformer models from the outset. This invention introduces a Transformer encoder-decoder at the candidate region level for global relationship modeling and decouples feature representations into category-independent shape features and category-related model features. This helps maintain strong discriminative ability when multiple UAVs with similar appearances appear simultaneously, reducing misidentification due to feature confusion. By constructing a spatiotemporal graph with UAV candidate regions as nodes and spatial and temporal correlations as edges, and employing a graph attention network for trajectory association, this invention focuses on highly correlated spatiotemporal neighbors while considering local neighborhood constraints, improving the stability of individual identity maintenance under occlusion and intersecting flight scenarios. Furthermore, this invention achieves reliable target representation even in typical low-altitude security environments such as poor lighting, backlighting, or nighttime by fusing cross-modal features across regions of interest in visible and infrared images and adaptively allocating fusion weights based on modal quality scores. This significantly enhances the robustness of the overall system. The use of deformable convolution to focus on the unobstructed portion of the UAV, along with an incremental learning mechanism based on online hard example mining, enables the model to continuously optimize as the scene changes, balancing suppression of complex backgrounds with adaptability to new types of UAVs. Combined with control over the number and size of regions of interest on edge computing devices, this invention improves recognition accuracy and trajectory stability while meeting real-time processing requirements, making it suitable for various low-altitude security applications such as border patrol, urban security, and large-scale events. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation on the scope of this application.

[0020] Figure 1 This is a flowchart illustrating the drone identification method for low-altitude security in the embodiments.

[0021] Figure 2 This is a schematic diagram of the framework of the drone identification system for low-altitude security in the embodiment. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0023] All terms used in this application (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0024] For example, the terms “first” and “second” used in this application are only used to distinguish and describe similar objects, to differentiate the first object from another object, and are not used to describe a specific order or sequence, nor should they be interpreted as indicating or implying relative importance.

[0025] This application proposes a drone identification method for low-altitude security, combining... Figure 1 As shown, the method includes: Step S1: Obtain a sequence of low-altitude area monitoring images collected by low-altitude security monitoring equipment. The monitoring image sequence includes images of at least one drone. In this embodiment, the low-altitude security monitoring equipment can be a fixedly installed visible light camera and / or infrared camera. Its installation height and pitch angle are selected based on the range of the airspace to be protected and the distribution of obstacles, ensuring the monitoring image covers typical patrol routes and no-fly zones. The monitoring image sequence can be acquired via network video streaming or local storage files. The frame rate is preferably set between 20 and 30 frames per second to balance motion continuity and storage overhead. The single-frame resolution can be common high-definition specifications such as 1280 x 720 pixels or 1920 x 1080 pixels. Specifically, to meet the field-of-view requirements of different application scenarios, the correspondence between pixels and actual spatial scale can be pre-calibrated during the deployment phase based on the camera's focal length and installation position for subsequent size constraint and motion consistency calculations. Considering that drones may enter the frame at any time after monitoring begins, the acquisition module typically runs continuously, organizing continuously acquired image frames within a preset time window into a monitoring image sequence, and automatically identifying whether at least one drone image appears within it. Optionally, to improve the system's fault tolerance, when acquisition fails or single-frame decoding is abnormal, the system can skip the frame and record the error information internally without interrupting the reception and processing of subsequent frames.

[0026] Step S2: Motion detection and / or foreground separation are performed on each frame of the monitored image sequence to determine several regions of interest (ROIs) that are suspected to contain drone targets. Step S3: For each region of interest (ROI), crop out the candidate regions of drones from the corresponding frame images, and extract features from each candidate region of drones to obtain the feature representation of the candidate regions of drones. Step S4: Construct a spatiotemporal graph based on the candidate regions of drones and their feature representations in consecutive frames of the monitoring image sequence. Nodes in the spatiotemporal graph represent candidate regions of drones, and edges in the spatiotemporal graph represent the spatiotemporal relationships between candidate regions of drones that are correlated in spatial location and time. Step S5: Use graph neural networks to reason about the spatiotemporal graph to obtain the drone trajectory identifier of each drone candidate region; Step S6: Based on the drone trajectory identifier, output the individual information of each drone target. The individual information includes at least the drone's identity identifier and / or trajectory information. In one embodiment, step S2 includes: A background model is constructed based on a preset number of multi-frame surveillance images; The current frame monitoring image is compared with the background model to obtain the moving foreground region; Within the foreground region of motion, regions of interest (ROIs) that conform to the prior characteristics of the UAV are selected based on preset size and aspect ratio constraints. Specifically, motion detection and foreground separation can employ a time-sliding window-based background modeling approach. This involves statistically analyzing the brightness or grayscale of each pixel over the most recent few seconds to obtain a background estimate, and then using the difference between this estimate and the current frame's pixel value to determine the foreground region. To suppress noise and isolated bright spots, the foreground binary image can undergo morphological operations such as median filtering, dilation, and erosion to remove isolated pixels and fill small holes. Then, connected component analysis is used to extract candidate motion blocks. Size constraints can be determined based on the estimated projection size of the UAV at the target monitoring distance. In engineering practice, the shortest and longest sides of the candidate blocks can be limited to a certain percentage range of the image height or width, thereby filtering out excessively small noise spots and obviously oversized cloud clusters and building outlines. Aspect ratio constraints can be set to a certain range based on the statistical results of the shapes of common multi-rotor UAVs presented from the monitoring perspective, excluding areas that are too elongated or nearly square but have abnormally large sizes. In actual system deployment, the aforementioned size and aspect ratio thresholds can be obtained through offline analysis of historical monitoring videos and configured separately at different installation heights or lens focal lengths to ensure that the regions of interest are neither too few nor too many. Furthermore, when the moving foreground region in a frame is empty or there are no connected components that satisfy the prior constraints after screening, no region of interest is generated for that frame. The system directly outputs an empty candidate set and skips subsequent feature extraction and spatiotemporal graph construction to avoid invalid computation.

[0027] In one embodiment, after obtaining the region of interest (ROI) in step S2, the method further includes: Calculate the region of interest (ROI) score for each ROI. Based on the results of the region of interest score calculation, non-maximum suppression is performed to remove overlapping regions of interest, resulting in the final UAV candidate regions; Before performing nonmaximum suppression on the region of interest, the following steps are also included: For each region of interest (ROI), a target appearance matching score, motion consistency score, and size prior score are calculated. The target appearance matching score characterizes the similarity between the appearance features of the ROI and a reference appearance template constructed based on historical high-confidence UAV samples. The motion consistency score characterizes the offset between the position of the ROI in the current frame and the position predicted by the UAV motion state in the previous frame. The size prior score characterizes the deviation of the area and aspect ratio of the ROI from the pre-statistically obtained UAV size prior distribution. The three scores are weighted and combined according to adjustable weights with a constant sum to obtain the comprehensive ROI score. A score threshold is adaptively set based on the mean and fluctuation range of the comprehensive ROI scores of all ROIs in the same frame, and only ROIs with a comprehensive ROI score not lower than the score threshold are retained as UAV candidate regions. In the region of interest screening process, after obtaining the regions of interest, a comprehensive region of interest score is calculated for each region of interest, which is used for subsequent non-maximum suppression and UAV candidate region retention determination. In one embodiment, for the first Each region of interest has a combined score, expressed as a weighted sum. , in, Indicates the first The overall interest region score for each interest region Indicates the first The target appearance matching score for each region of interest. Indicates the first Motion consistency score for each region of interest Indicates the first Prior score of the size of each region of interest This represents the weighting coefficient of the target appearance matching score. The weighting coefficients representing the motion consistency score. The weighting coefficients represent the prior scores for size; to facilitate normalization control, each weighting coefficient satisfies a linear constraint: , in, , , The meaning is the same as above, used to limit the relative contribution ratio of the three sub-item scores to the overall score; In one embodiment, to evaluate the consistency between the region of interest and the prior appearance of the UAV, a target appearance matching score can be obtained based on the cosine similarity between the appearance feature vector of the region of interest and the features of the reference template; for the first... For each region of interest, the target appearance matching score can be constructed as follows: , in, Indicates the first The target appearance matching score for each region of interest. Indicates from the first The appearance feature vectors extracted from images cropped from regions of interest. This represents the reference appearance feature vector obtained from historical high-confidence drone samples. Representing the eigenvector The Euclidean norm, Representing the eigenvector The Euclidean norm; through this construction, the appearance matching score. Falling The larger the value in the interval, the closer the appearance of the region of interest is to the prior appearance of the drone. In low-altitude security scenarios, drones exhibit a certain degree of motion continuity, allowing for the allocation of motion consistency scores to regions of interest using position prediction results between adjacent frames; for the first... For each region of interest, its motion offset distance can be calculated first: , in, Indicates the first The motion offset distance of each region of interest in the current frame. Indicates the first The center position vector of each region of interest in the current frame This indicates the prediction of the first frame's drone motion state based on the previous frame. The center position vector of each region of interest in the current frame Representing vector difference The Euclidean norm; after obtaining the offset distance, the motion consistency score can be constructed using an exponential decay method: , in, Indicates the first Motion consistency score for each region of interest This represents the attenuation coefficient for motion offset distance, used to adjust the effect of offset distance on the score decay rate; the smaller the motion offset distance, the closer the motion consistency score is to 1, and the score decays exponentially as the offset distance increases; The degree of matching between the size characteristics of the region of interest and the prior size of the UAV can be used to suppress obviously anomalous moving targets; In one embodiment, for the first For each region of interest, a size prior score is constructed based on its area and aspect ratio: , in, Indicates the first Prior score of the size of each region of interest Indicates the first The area of ​​each region of interest on the pixel plane This represents the prior expected value of the drone target area obtained from statistical analysis of training data. This represents the standard deviation of the prior target area of ​​the drone. Indicates the first The aspect ratio of each area of ​​interest This represents the prior expected value of the aspect ratio of the UAV target obtained from statistical analysis of training data. This represents the standard deviation of the prior aspect ratio of the drone target; the closer the area and aspect ratio of the region of interest are to the prior expected value, the smaller the offset term in the corresponding index, and the higher the size prior score. because , , The values ​​of are all constructed in The interval is defined, and the weight coefficients satisfy linear constraints, to obtain the comprehensive interest region score. Within a stable range, it is convenient to use a uniform threshold for filtering; to adapt to changes in the number of candidate regions and the intensity of background interference in different frames, an adaptive threshold can be constructed based on the comprehensive score statistics of all interest regions within the same frame; let the total number of candidate regions in this frame be... Each region of interest corresponds to a comprehensive score of [number]. The average score of the frame can be written as follows: , in, This represents the average score of the integrated region of interest in the current frame. Indicates the number of regions of interest in the current frame. Indicates the first The overall interest region score for each interest region; the corresponding standard deviation is written as: , in, This represents the standard deviation of the comprehensive region of interest score in the current frame. Based on the above statistics, a comprehensive score threshold with adjustment parameters can be constructed: , in, The threshold for filtering based on the overall region of interest score. This represents a coefficient that adjusts the sensitivity of the overall score threshold; in actual judgment, conditional judgments are performed for each region of interest. , in, Indicates the first The overall interest region score for each interest region This represents the screening threshold for the comprehensive region of interest score; when the above conditions are met, the corresponding region of interest is retained as a candidate region for the UAV; when the conditions are not met, the region of interest is removed; during the system deployment phase, the coefficients can be adjusted according to target density and real-time constraints. and each weighting coefficient , , Offline calibration is performed to keep the number of remaining candidate regions within an acceptable range; Specifically, the aforementioned region of interest (ROI) scoring scheme unifies appearance matching, motion behavior, and size prior into a weighted scoring framework, solving the problem of candidate regions being easily affected by noise under a single criterion. The appearance matching score measures the similarity between the ROI image and the reference template, allowing UAV regions with typical structural textures to receive higher scores, thus highlighting regions with target appearance features when there is a lot of background clutter. The motion consistency score utilizes the position prediction relationship between consecutive frames to give exponential bonuses to ROI regions that conform to continuous motion trajectories, which is beneficial for prioritizing the retention of real moving targets when there is short-term occlusion or detection jitter. The size prior score is based on the statistical distribution of area and aspect ratio, suppressing regions that significantly deviate from the UAV's morphological prior, reducing interference from excessively large or small dynamic targets. The comprehensive score uses adjustable weights for linear combination and combines the mean and standard deviation of the score distribution within the same frame to generate an adaptive threshold, enabling the screening strategy to automatically adjust the number of candidates according to scene changes, achieving a balance between false positive and false negative rates, and providing cleaner and more stable candidate inputs for subsequent spatiotemporal map construction and trajectory association. Furthermore, in practical engineering implementation, the weighting coefficients of target appearance matching score, motion consistency score, and size prior score can be determined by offline testing of different parameter combinations on labeled monitoring data collected under multiple time periods and weather conditions, and comparing recognition accuracy and false alarm rates. Initially, all three can be set to the same order of magnitude, and then gradually fine-tuned based on the statistical results of false positive and false negative rates during trial operation. The coefficients related to exponential decay and the sensitivity of the comprehensive score threshold can be set in conjunction with the typical flight speed of the UAV, the camera's field of view, and the allowable detection delay, so that the motion consistency score quickly approaches the upper limit under normal flight trajectories, while maintaining sufficient suppression of sudden high-speed noise targets. In this embodiment, the statistics of the comprehensive region of interest score are calculated independently within each frame. When the number of regions of interest is one or all regions of interest have the same score, resulting in a score standard deviation of zero, the system can degenerate to using only the average score of that frame as the screening threshold, thereby avoiding division by zero errors. For example, if the overall score in a frame is low, the selection threshold can still be calculated according to preset rules. However, when all overall scores are simultaneously below a preset minimum confidence level, the system can consider that the frame lacks reliable candidates and discard it entirely to avoid introducing purely noisy trajectories. Similarly, to ensure statistical stability, when the number of regions of interest is extremely small, the adaptive threshold update for that frame can be skipped, and the threshold configuration of the previous frame or historical window can be used instead, thereby avoiding drastic threshold fluctuations in cases of very few targets or occasional noise. The reference appearance template can be derived from training images collected and labeled according to typical UAV models before system deployment, or it can be slowly updated from multiple high-confidence detection results during operation, allowing the template to gradually adapt to the lighting and background characteristics of the actual scene.

[0028] In one embodiment, the feature extraction in step S3 includes: The Transformer encoder-decoder architecture is used to encode each UAV candidate region, and the global relationship between each UAV candidate region in the same frame and between UAV candidate regions across frames is modeled through a self-attention mechanism. The feature representations of each candidate region of UAVs are decoupled into category-independent shape features and category-related model features; In one embodiment, the monitoring image sequence includes a visible light image sequence and an infrared image sequence, and steps S2 and S3 include: The first region of interest (ROI) is detected in the visible light image, and the second region of interest (ROI) is detected in the infrared image. Based on the spatial relationship between the first Region of Interest (ROI) and the second Region of Interest (ROI), corresponding ROI pairs are used as candidate regions for UAVs. Cross-modal feature fusion is performed on candidate regions of drones of different modalities to improve the stability of drone recognition under complex lighting conditions; Cross-modal feature fusion for candidate regions of different UAV modalities includes: For each pair of visible light and infrared regions of interest, visible light and infrared feature vectors are extracted respectively, and mapped to a common feature space of the same dimension through their respective linear transformations. The fusion weights of the two modes are calculated based on the quality scores of the visible light and infrared modes in the current environment. The fusion weights are obtained by normalizing the quality scores, giving the mode with the higher quality score a greater weight in the fusion result. The common space feature vectors of the two modes are weighted and summed according to the fusion weights to obtain the cross-modal fusion features of the regions of interest. Furthermore, at the feature map level, the visible light and infrared feature maps are concatenated along the channel dimension. The two modal information is further mixed spatially through one-to-one convolution and nonlinear activation. The features obtained by the above fusion are temporally aggregated across multiple frames of the same UAV trajectory and used as the feature representation of the UAV trajectory, then input into a graph neural network for trajectory association. Visible light and infrared images have been paired at the Region of Interest (ROI) level, allowing for cross-modal feature fusion on each corresponding ROI pair. This ensures that the instability of a single modality under complex lighting conditions is compensated for by the other modality. For the... For visible light-infrared regions of interest, fusion can be performed gradually at three levels: vector level, spatial channel level, and temporal level. In one embodiment, for the first For the region of interest, feature vectors are extracted from the visible light image and the infrared image, respectively. and Then, it is first mapped to a common feature space of the same dimension, and then fused; the mapping process can be written as: , in, Indicates the first For the common spatial feature vectors of visible light modes in the region of interest, This represents the linear transformation matrix that maps visible light modal features to a common feature space. Indicates the first For the feature vectors of the region of interest in the visible light image, Indicates the first For the common spatial feature vectors of infrared modes in the region of interest, This represents the linear transformation matrix that maps infrared modal features to a common feature space. Indicates the first The feature vectors of the region of interest in the infrared image. An index representing a pair of regions of interest; After completing the public space mapping, cross-modal fusion features are constructed using a weighted summation method: , in, Indicates the first Cross-modal fusion of feature vectors for regions of interest. Indicates the first The fusion weighting coefficients for visible light modal features in the region of interest. Indicates the first For the common spatial feature vectors of visible light modes in the region of interest, Indicates the first The fusion weighting coefficients for infrared modal features in the region of interest. Indicates the first For the common spatial feature vectors of infrared modes in the region of interest, An index representing a pair of regions of interest; To adaptively adjust the importance of the two modes under complex lighting or low contrast conditions, a weighting coefficient is calculated based on the quality score of each mode; let the first... Quality scores are given for the visible light and infrared modes of the region of interest, respectively. and The modal weights are constructed as follows: , in, Indicates the first The fusion weighting coefficients for visible light modal features in the region of interest. This indicates the visible light modal quality score. The value after performing natural exponentiation. Indicates the first The quality score of the visible light mode in the region of interest is used to measure the reliability of that mode in the current environment. Indicates the infrared modal quality score The value after performing natural exponentiation. Indicates the first Quality scoring of infrared modes in the region of interest. Indicates the first The fusion weighting coefficients for infrared modal features in the region of interest. An index representing a pair of regions of interest. The above form represents the natural exponential function; it maps the quality scores of the two modes to weights that sum to 1 using a normalized exponential function. This is used to automatically increase the weight of the infrared mode when the visible light is too dim or the glare is severe, and to increase the weight of the visible light mode when the infrared noise is too high. After cropping the region of interest to a uniform size, finer-grained fusion can be performed at the feature map level; in one embodiment, for a time frame index of... A frame of an image, in spatial location At this point, visible light and infrared feature maps are stitched together along the channel dimension: , in, Indicates that the time frame index is Time and space Cross-modal concatenation of feature vectors at the location Indicates that the time frame index is Time and space The channel vector extracted from the visible light region of interest feature map. Indicates that the time frame index is Time and space The channel vector extracted from the infrared region of interest feature map. This indicates the frame index in the monitored image sequence. Represents the pixel coordinates of the region of interest in the horizontal direction. Represents the pixel coordinates of the region of interest in the vertical direction; After obtaining the splicing features, it can be done through... Convolution performs linear mixing along the channel dimension, while simultaneously superimposing nonlinear activations to achieve cross-modal feature fusion at the spatial location level: , in, Indicates that the time frame index is Time and space Cross-modal fusion feature vector at the location, This represents a non-linear activation function applied element-wise to the input vector. This indicates that a linear transformation is performed along the channel dimension. Convolution weight matrix, Indicates that the time frame index is Time and space Cross-modal concatenation of feature vectors at the location Indicates and The corresponding bias vector, This indicates the frame index in the monitored image sequence. Represents the pixel coordinates of the region of interest in the horizontal direction. Represents the pixel coordinates of the region of interest in the vertical direction; After the feature map-level fusion described above, a pooling operation is performed on the spatial dimension to... Aggregate the region of interest in the first position The fused feature vectors under the frame are passed as input to the subsequent Transformer feature decoupling and graph neural network trajectory association modules, realizing the structural division of labor of first fusing in the channel dimension and then modeling in the temporal dimension; When the same UAV forms a trajectory across multiple frames of images, a trajectory-level representation can be constructed based on multi-frame cross-modal fusion features to enhance recognition stability under short-term occlusion conditions; let the... The drone trajectory is in continuous If observed in a frame, the trajectory-level fusion feature can be written as: , in, Indicates the first The time-series average fused feature vector of the drone trajectory. Indicates the first The number of observation frames contained in a drone trajectory. Indicates the first The drone trajectory on the first Cross-modal fusion feature vectors on each observation frame This indicates the frame number within the trajectory. An index representing the drone's trajectory; this trajectory-level representation can supplement the node features in a graph neural network, and be used to consider cross-frame consistency information during the trajectory association stage; Specifically, the aforementioned cross-modal feature fusion implementation uniformly processes visible light and infrared information at the region of interest (ROI) level. Through common feature space mapping and weighted fusion, the two modalities are aligned in dimension and formed into a single vector representation. In scenes with stable lighting and clear textures, the visible light modality has a higher quality score, and the fusion weights shift towards the visible light side, which is beneficial for using detailed textures to distinguish model and structural differences. In nighttime or backlit scenes, the infrared modality has a better quality score, and the fusion result relies more on thermal radiation information, thereby reducing the probability of missed detection due to insufficient exposure. Feature map-level channel splicing and one-to-one convolution further perform linear and nonlinear mixing of the two modalities at a fine-grained spatial location, which can comprehensively use texture and hotspot information at the same location to suppress false targets and background interference. Trajectory-level temporal fusion aggregates multi-frame fusion features, so that even when single-frame detection is unstable, it can still rely on the statistical characteristics of the entire trajectory to complete identification and association. Thus, in low-altitude security scenarios with complex lighting and dense interference targets, it improves the stability of UAV identification, reduces the false alarm rate, and reduces the burden of subsequent graph neural network inference. For example, the quality score for the visible light modality can be calculated based on image statistical indicators such as local and global brightness, contrast, saturation, and noise levels. This includes statistically analyzing the grayscale histogram distribution of each region of interest, estimating the signal-to-noise ratio, and detecting overexposed or underexposed areas, mapping these to a quality score using a linear or piecewise function. The quality score for the infrared modality can be obtained based on indicators such as temperature contrast, hotspot clarity, and background noise density. The calculation of these quality scores can be performed using existing feature maps during feature extraction. Furthermore, to ensure spatial alignment between the two modalities, this embodiment preferably completes geometric calibration during the camera installation phase, transforming the imaging planes of the visible light camera and the infrared camera to a unified coordinate system through calibration. During runtime, the two images are synchronized according to timestamps, ensuring that each pair of regions of interest has a pixel-level correspondence. Optionally, when a modality experiences short-term data loss or severe interference, causing its quality score to drop to near its minimum, the normalized fusion weights will automatically approach zero. In this case, the fusion features are mainly contributed by the other modality, achieving a degenerate working mode of single-modality recognition without adding new switching logic to the system structure. The weights and bias parameters of the one-to-one convolution can be learned along with other network parameters through the conventional deep learning training process during the large-scale offline training phase. They remain fixed during actual deployment, so that the cross-modal fusion process is only executed in forward inference and does not increase the complexity of the model structure during runtime.

[0029] In one embodiment, the graph neural network is a graph attention network, which adaptively models the spatiotemporal relationship between candidate regions of the UAV by aggregating neighborhood information based on attention weights. In the feature extraction process of step S3, a deformable convolutional layer is used to perform convolution operations on the candidate regions of the UAV, which is used to dynamically adjust the receptive field according to the shape of the UAV and the occlusion situation, focusing on the part of the UAV that is not occluded. The method also includes an incremental learning step based on online hard example mining, in which hard example samples are selected based on the UAV recognition results during system operation, and the feature extraction model and / or graph neural network are dynamically updated using hard example samples; Optionally, online hard example mining can be triggered at fixed time intervals or based on cumulative frame counts during system operation. Each trigger iterates through the recognition results of the most recent period, storing candidate drone regions with low classification confidence, frequent identity switching during trajectory association, or those manually labeled as false detections as hard example samples in a buffer. To avoid excessive impact on online inference, the model update process can be executed during periods of low processing load or in a background thread, limiting the number of samples and iterations involved in each update to keep inference latency relatively stable. In this embodiment, the parameter initialization of the feature extraction model and graph neural network comes from offline training on the labeled training set before deployment. Online incremental learning only performs small-step fine-tuning around the original parameters to reduce the risk of catastrophic forgetting. Furthermore, when recent network anomalies or insufficient hardware resources are detected, the system can temporarily freeze parameter updates, retaining only the hard example sample cache and updating it again after resources are restored, thereby ensuring the continuity and reliability of the main drone recognition process.

[0030] This application proposes a drone identification system for low-altitude security, combining... Figure 1 As shown, the system includes: an image acquisition module, used to acquire low-altitude area monitoring image sequences collected by low-altitude security monitoring equipment; The Region of Interest (ROI) determination module is used to perform motion detection and / or foreground separation on each frame of the monitoring image sequence, to determine several ROIs that are suspected to contain UAV targets, and to calculate and filter UAV candidate regions based on the ROI scores. The feature extraction module is used to extract feature representations for each candidate region of the UAV; The trajectory association module is used to construct a spatiotemporal map based on each UAV candidate region and its feature representation, and to use a graph neural network to reason about the spatiotemporal map to obtain the UAV trajectory identifier to which each UAV candidate region belongs; The output module is used to output individual information of each UAV target based on the UAV trajectory identifier; In one embodiment, the region of interest determination module is configured as follows: A background model is constructed based on a preset number of multi-frame surveillance images; The current frame monitoring image is compared with the background model to obtain the moving foreground region; Within the foreground region of motion, regions of interest (ROIs) that conform to the prior characteristics of the UAV are selected based on preset size and aspect ratio constraints. In one embodiment, the feature extraction module includes a Transformer encoder-decoder network, which is configured as follows: Each candidate region of a UAV is encoded, and the relationship between the candidate regions of each UAV is globally modeled through a self-attention mechanism. The feature representations of each candidate region of a UAV are decoupled into category-independent shape features and category-related model features; In one embodiment, the system is deployed on an edge computing device, which is connected to a low-altitude security monitoring device. The system completes the identification and tracking of multiple drones while meeting preset real-time constraints by limiting the number and size of regions of interest (ROIs) and using graph neural networks to reduce the computational load. Similarly, in terms of hardware implementation, edge computing devices can utilize industrial control hosts or embedded computing platforms equipped with general-purpose central processing units (CPUs) and graphics processing units (GPUs). The processor's clock speed, memory capacity, and storage bandwidth are configured based on the number and resolution of cameras at the target monitoring point to ensure the entire process from region of interest (ROI) determination and feature extraction to graph neural network inference is completed at the target frame rate. For example, when a single camera acquires high-definition images at 20 to 30 frames per second and the number of ROIs per frame is controlled to within a few dozen, a GPU with moderate computing power can complete a forward inference and trajectory update within milliseconds, thus meeting real-time alarm requirements. Furthermore, to ensure that preset real-time constraints are met during long-term operation, the system can periodically analyze the actual processing latency and the average number of ROIs. When the latency is found to be continuously approaching the allowable upper limit, the ROI score threshold can be appropriately tightened or the input image resolution reduced without changing the overall algorithm flow. This keeps the number of candidate regions entering the subsequent network within a controllable range, thereby stably achieving multi-UAV identification and tracking functions with limited hardware resources.

[0031] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0032] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of this application and form different embodiments. For example, all the embodiments above can be used in any combination. The information disclosed in this background section is intended only to enhance the understanding of the general background of this application and should not be construed as an admission or in any way implying that such information constitutes prior art known to those skilled in the art.

Claims

1. A method for identifying unmanned aerial vehicles (UAVs) for low-altitude security, characterized in that, include: Step S1: Obtain a sequence of low-altitude area monitoring images collected by low-altitude security monitoring equipment, wherein the monitoring image sequence includes images of at least one drone; Step S2: Motion detection and / or foreground separation are performed on each frame of the monitored image sequence to determine several regions of interest (ROIs) that are suspected to contain UAV targets. Step S3: For each of the aforementioned regions of interest (ROI), crop out the candidate regions of drones from the corresponding frame images, and extract features from each candidate region of drones to obtain the feature representation of the candidate regions of drones. Step S4: Construct a spatiotemporal graph based on the candidate regions of drones and their feature representations in consecutive frames of the monitoring image sequence. Nodes in the spatiotemporal graph represent candidate regions of drones, and edges in the spatiotemporal graph represent the spatiotemporal relationships between candidate regions of drones that are correlated in spatial location and time. Step S5: Use a graph neural network to reason about the spatiotemporal graph to obtain the drone trajectory identifier of each drone candidate region; Step S6: Based on the UAV trajectory identifier, output the individual information of each UAV target, wherein the individual information includes at least the UAV's identity identifier and / or trajectory information.

2. The UAV identification method for low-altitude security as described in claim 1, characterized in that, Step S2 includes: A background model is constructed based on a preset number of multi-frame surveillance images; The current frame monitoring image is compared with the background model to obtain the moving foreground region; Within the foreground region of motion, regions of interest (ROIs) that conform to the prior characteristics of the UAV are selected based on preset size and aspect ratio constraints.

3. A drone identification method for low-altitude security as described in claim 1 or 2, characterized in that, In step S2, after obtaining the region of interest (ROI), the following steps are also included: Calculate the region of interest (ROI) score for each ROI. Based on the results calculated from the region of interest scores, non-maximum suppression is performed to remove overlapping regions of interest, resulting in the final UAV candidate regions. Before performing nonmaximum suppression on the region of interest, the following steps are also included: For each region of interest (ROI), a target appearance matching score, motion consistency score, and size prior score are calculated. The target appearance matching score characterizes the similarity between the appearance features of the ROI and a reference appearance template constructed based on historical high-confidence UAV samples. The motion consistency score characterizes the offset between the position of the ROI in the current frame and the position predicted by the UAV motion state in the previous frame. The size prior score characterizes the deviation of the area and aspect ratio of the ROI from the pre-statistically obtained UAV size prior distribution. The three scores are weighted and combined according to adjustable weights with a constant sum to obtain the comprehensive ROI score. A score threshold is adaptively set based on the mean and fluctuation range of the comprehensive ROI scores of all ROIs in the same frame, and only ROIs with a comprehensive ROI score not lower than the score threshold are retained as UAV candidate regions.

4. A drone identification method for low-altitude security as described in any one of claims 1 to 3, characterized in that, The feature extraction in step S3 includes: The Transformer encoder-decoder architecture is used to encode each UAV candidate region, and the global relationship between each UAV candidate region in the same frame and between UAV candidate regions across frames is modeled through a self-attention mechanism. The feature representation of each UAV candidate region is decoupled into category-independent shape features and category-related model features.

5. A drone identification method for low-altitude security as described in any one of claims 1 to 4, characterized in that, The monitoring image sequence includes a visible light image sequence and an infrared image sequence. Steps S2 and S3 include: The first region of interest (ROI) is detected in the visible light image, and the second region of interest (ROI) is detected in the infrared image. Based on the spatial relationship between the first Region of Interest (ROI) and the second Region of Interest (ROI), the corresponding ROI pairs are used as candidate regions for the UAV. Cross-modal feature fusion is performed on candidate regions of different UAV modes; The cross-modal feature fusion of candidate regions for UAVs of different modalities includes: For each pair of visible light and infrared regions of interest, visible light and infrared feature vectors are extracted respectively, and mapped to a common feature space of the same dimension through their respective linear transformations. The fusion weights of the two modes are calculated based on the quality scores of the visible light and infrared modes in the current environment. These fusion weights are obtained by normalizing the quality scores, giving the mode with the higher quality score a greater weight in the fusion result. The common space feature vectors of the two modes are weighted and summed according to the fusion weights to obtain the cross-modal fusion features of the regions of interest. Furthermore, at the feature map level, the visible light and infrared feature maps are concatenated along the channel dimension. The two modal information is further mixed spatially through one-to-one convolution and nonlinear activation. The features obtained by the above fusion are temporally aggregated across multiple frames of the same UAV trajectory and used as the feature representation of the UAV trajectory, then input into a graph neural network for trajectory association.

6. A drone identification method for low-altitude security as described in any one of claims 1 to 5, characterized in that, The graph neural network is a graph attention network, which adaptively models the spatiotemporal relationship between candidate regions of the UAV by aggregating neighborhood information based on attention weights. In the feature extraction process of step S3, a deformable convolutional layer is used to perform convolution operations on the candidate regions of the UAV, which is used to dynamically adjust the receptive field according to the shape of the UAV and the occlusion situation, focusing on the part of the UAV that is not occluded. The method further includes an incremental learning step based on online hard example mining, in which hard example samples are selected according to the UAV identification results during system operation, and the feature extraction model and / or the graph neural network are dynamically updated using the hard example samples.

7. A drone identification system for low-altitude security, based on the drone identification method for low-altitude security according to any one of claims 1 to 6, characterized in that, include: The image acquisition module is used to acquire sequences of low-altitude area monitoring images collected by low-altitude security monitoring equipment; The region of interest (ROI) determination module is used to perform motion detection and / or foreground separation on each frame of the monitored image sequence, to determine several ROIs that are suspected to contain UAV targets, and to calculate and filter UAV candidate regions based on the ROI scores. The feature extraction module is used to extract feature representations for each candidate region of the UAV; The trajectory association module is used to construct a spatiotemporal map based on each UAV candidate region and its feature representation, and to use a graph neural network to reason about the spatiotemporal map to obtain the UAV trajectory identifier to which each UAV candidate region belongs. The output module is used to output individual information of each UAV target based on the UAV trajectory identifier.

8. The UAV identification system for low-altitude security as described in claim 7, characterized in that, The region of interest determination module is configured as follows: A background model is constructed based on a preset number of multi-frame surveillance images; The current frame monitoring image is compared with the background model to obtain the moving foreground region; Within the foreground region of motion, regions of interest (ROIs) that conform to the prior characteristics of the UAV are selected based on preset size and aspect ratio constraints.

9. A drone identification system for low-altitude security as described in claim 7 or 8, characterized in that, The feature extraction module includes a Transformer encoder-decoder network, which is configured as follows: Each candidate region of a UAV is encoded, and the relationship between the candidate regions of each UAV is globally modeled through a self-attention mechanism. The feature representation of each UAV candidate region is decoupled into category-independent shape features and category-related model features.

10. A drone identification system for low-altitude security as described in any one of claims 7 to 9, characterized in that, The system is deployed on an edge computing device, which is connected to a low-altitude security monitoring device. The system reduces the computational load by limiting the number and size of Regions of Interest (ROIs) and by employing the graph neural network.

Images